Architecture¶

OpenNebula is a single-daemon cloud management platform. Unlike OpenStack's distributed microservice approach, OpenNebula's core functionality runs in a single process (oned) with a modular driver framework for hypervisors, storage, networking, and authentication. This architectural simplicity is a key differentiator.

Component Overview¶

graph TD
    subgraph Frontend["Frontend Node"]
        ONED["oned<br/>(Core Daemon)"]
        SCHED["Scheduler<br/>(mm_sched)"]
        SUNSTONE["Sunstone<br/>(Web UI)"]
        FIREDGE["FireEdge<br/>(Guacamole Proxy)"]
        ONEFLOW["OneFlow<br/>(Service Orchestration)"]
        ONEGATE["OneGate<br/>(VM REST API)"]
        HEM["Hook Execution<br/>Manager (HEM)"]
    end

    subgraph Drivers["Driver Framework"]
        VM_DRV["VM Drivers<br/>(KVM, LXC, Firecracker)"]
        NET_DRV["Network Drivers<br/>(bridge, 802.1Q, OVSwitch, VXLAN)"]
        STO_DRV["Storage Drivers<br/>(FS, Ceph, LVM, vCenter)"]
        AUTH_DRV["Auth Drivers<br/>(SSH, LDAP, x509)"]
    end

    subgraph Hosts["Hypervisor Hosts"]
        KVM1["KVM Host 1"]
        KVM2["KVM Host 2"]
        LXC1["LXC Host"]
    end

    subgraph Datastores["Datastores"]
        SYS_DS["System DS<br/>(host-local SSH)"]
        IMG_DS["Image DS<br/>(shared/ceph)"]
        FILE_DS["File DS<br/>(kernel/initrd)"]
    end

    SUNSTONE --> ONED
    FIREDGE --> ONED
    ONEFLOW --> ONED
    ONEGATE --> ONED
    ONED --> SCHED
    ONED --> Drivers
    VM_DRV --> KVM1
    VM_DRV --> KVM2
    VM_DRV --> LXC1
    NET_DRV --> KVM1
    NET_DRV --> KVM2
    STO_DRV --> Datastores

Core Daemon: oned¶

oned is the central service of the OpenNebula platform. It is installed as the opennebula package and managed by the opennebula systemd service.

Key responsibilities:

Manage cluster nodes (hosts), virtual networks, and datastores
Manage users, groups, and Virtual Data Centers (VDCs)
Manage VM lifecycle (create, deploy, migrate, terminate)
Expose the XML-RPC API for all external access
Coordinate with the scheduler for VM placement
Execute drivers for hypervisor, storage, network, and authentication operations

The daemon configuration lives in /etc/one/oned.conf, which defines datastore types, supported drivers, monitoring intervals, and default VM settings.

Scheduler¶

The OpenNebula scheduling framework is a modular system responsible for resource allocation:

Integrated with oned: The scheduler manager runs as part of the core daemon and requests placement plans from external scheduler processes
Scheduler Plan Manager: Executes placement and optimization plans. It deploys or migrates VMs.
External Schedulers: Generate placement and optimization plans using configurable policies

The scheduler allocates three resource types:

Resource	Purpose
Hosts	Determines which physical host runs each VM
System Datastores	Selects storage location for VM disk images
Virtual Networks	Assigns networks to VM interfaces (when set to auto mode)

Scheduling operations include:

Initial VM placement: Based on capacity, compatibility, affinity rules, and constraints
VM re-scheduling: On-demand re-placement triggered by admins or automation
Cluster-wide load balancing: Generates optimization plans to distribute workloads per defined policies

Built-in schedulers include the Rank Scheduler (scores hosts with user-defined expressions) and the OpenNebula DRS (Distributed Resource Scheduler) for automated load balancing.

User Interfaces¶

Sunstone (Web UI)¶

Sunstone is the primary web management interface for OpenNebula. It communicates with oned via the XML-RPC API and provides:

VM, template, image, and network management
User and group administration
Virtual Data Center (VDC) management
Role-based views (cloud admin, group admin, end user)
Cloud-init and contextualization support

Sunstone authentication supports its own login mechanism as well as integration with external identity providers.

FireEdge¶

FireEdge provides remote console access to VMs through a web browser. It integrates with Guacamole to offer:

VNC, RDP, and SSH console access without client-side plugins
Token-based session management
Direct integration with Sunstone

OneFlow¶

OneFlow is a RESTful service for orchestrating multi-VM application stacks:

Define services as groups of interconnected VMs with deployment dependencies
Supports elasticity rules (auto-scale based on metrics)
Manages VM startup ordering and shutdown sequencing
Exposes a REST API independent of the core XML-RPC API

OneGate¶

OneGate is a RESTful service that enables running VMs to communicate with OpenNebula:

VMs can pull contextualization data and push custom metrics
Enables application-level monitoring inside the guest
Useful for auto-scaling triggers in OneFlow

Driver Framework¶

OpenNebula uses a pluggable driver architecture to support heterogeneous infrastructure:

VM Drivers (Virtualization)¶

Driver	Description
KVM	Primary hypervisor. Uses libvirt to manage QEMU/KVM guests
LXC	OS-level container virtualization (v7.2+ improved support)
Firecracker	MicroVM-based isolation for lightweight workloads
vCenter	VMware vCenter integration (primarily for migration)

Network Drivers¶

Driver	Description
bridge	Default Linux bridge networking
802.1Q	VLAN-tagged networking with per-VLAN bridges
OVSwitch	Open vSwitch integration
VXLAN	Overlay networking without switch configuration
AWS	Amazon VPC network integration
ebtables	Firewall rules at the Ethernet layer

Storage Drivers¶

Driver	Description
fs	Filesystem-based (shared via NFS, for example)
ceph	Ceph RBD integration for block storage
lvm	LVM volume groups
vcenter	VMware datastore integration
dummy	No transfer (useful for specialized setups)

Authentication Drivers¶

Driver	Description
core	Built-in user/password with SHA256 hashing
ssh	SSH key-based authentication
ldap	LDAP/Active Directory integration
x509	Client certificate authentication
server_cipher / server_x509	Service-to-service authentication

Datastores¶

OpenNebula uses three types of datastores:

Type	Purpose	Example
System DS	VM disk images and runtime files. Typically host-local (SSH) or shared (NFS, Ceph)	`/var/lib/one/datastores/0/`
Image DS	Stores VM disk images (templates, ISOs). Used for image registration and cloning	`/var/lib/one/datastores/1/`
File DS	Stores kernel, initrd, and context files	`/var/lib/one/datastores/2/`

The storage driver handles the transfer from Image DS to System DS. For example, Ceph drivers clone RBD images. SSH drivers copy files over SSH.

VM Lifecycle¶

stateDiagram-v2
    [*] --> Pending : onevm create
    Pending --> Hold : on hold
    Hold --> Pending : release
    Pending --> Prolog : scheduler assigns host
    Prolog --> Boot : transfer image to host
    Boot --> Running : guest OS boots
    Running --> Migrate : live migration
    Migrate --> Running : migration complete
    Running --> Saves : onevm suspend
    Saves --> Suspended : saved to disk
    Suspended --> Boot : onevm resume
    Running --> Shutdown : onevm shutdown
    Running --> Poweroff : onevm poweroff
    Poweroff --> Boot : onevm resume
    Shutdown --> Epilog : guest shuts down
    Epilog --> Done : cleanup
    Boot --> Failed : boot error
    Running --> Failed : monitor error
    Done --> [*]

System Interfaces¶

Interface	Protocol	Purpose
XML-RPC API	XML-RPC over HTTP	Primary API for all resource management
OneFlow API	REST (JSON)	Multi-VM service orchestration
OneGate API	REST (JSON)	VM-to-platform communication
Hooks (ZeroMQ)	Pub/Sub	Event-driven automation
Sunstone	HTTP/HTTPS	Web management interface
Prometheus Exporter	HTTP	Metrics export for monitoring

Language bindings are available for Ruby, Java, Go, and Python via the XML-RPC API.

Key File Locations¶

Path	Purpose
`/etc/one/oned.conf`	Core daemon configuration
`/etc/one/sched.conf`	Scheduler configuration
`/etc/one/sunstone-server.conf`	Sunstone web UI configuration
`/etc/one/fireedge-server.conf`	FireEdge console configuration
`/etc/one/oneflow-server.conf`	OneFlow service configuration
`/var/lib/one/datastores/`	Datastore directories
`/var/lib/one/vms/`	VM deployment files
`/var/log/one/oned.log`	Core daemon log
`/var/log/one/sched.log`	Scheduler log

References¶

How It Works¶

Internal mechanisms, VM lifecycle, the centralized daemon model, and the AI-powered DRS.

Centralized Architecture Model¶

Unlike OpenStack's distributed services, OpenNebula uses a centralized daemon model where oned is the single source of truth and orchestration. This dramatically simplifies operations, troubleshooting, and upgrades.

flowchart TB
    subgraph Frontend["Front-End Node"]
        ONED["oned\n(core daemon)"]
        Sched["Scheduler"]
        OneDRS["OneDRS\n(AI DRS)"]
        DB["MySQL / SQLite"]
        Sunstone["Sunstone UI"]
        OneFlow["OneFlow"]
        OneGate["OneGate"]
    end

    subgraph Host1["KVM Host 1"]
        Libvirt1["libvirtd"]
        VMs1["VMs"]
    end

    subgraph Host2["KVM Host 2"]
        Libvirt2["libvirtd"]
        VMs2["VMs"]
    end

    ONED -->|"SSH + drivers"| Host1
    ONED -->|"SSH + drivers"| Host2
    Sched --> ONED
    OneDRS --> ONED
    Sunstone --> ONED
    ONED --> DB

    style Frontend fill:#00758f,color:#fff

VM Provisioning Flow¶

sequenceDiagram
    participant User as User / Sunstone
    participant API as oned (XML-RPC / gRPC)
    participant DB as MySQL
    participant Sched as Scheduler
    participant Host as KVM Host

    User->>API: vm.allocate (template)
    API->>DB: Store VM record (PENDING)
    Sched->>DB: Read pending VMs
    Sched->>Sched: Match requirements → host
    Sched->>API: Deploy VM on Host X
    API->>Host: SSH: deploy script
    Host->>Host: Create libvirt domain
    Host->>Host: Attach disks, configure NICs
    Host->>Host: Start QEMU/KVM
    Host-->>API: VM RUNNING
    API->>DB: Update state → RUNNING

OneDRS (Distributed Resource Scheduler)¶

flowchart LR
    Metrics["Host Metrics\n(CPU, RAM, I/O)"] --> DRS["OneDRS Engine"]
    History["Historical Data\n(trends, patterns)"] --> DRS
    Policies["Placement Policies\n(packing, striping, load-aware)"] --> DRS
    DRS -->|"migration\nrecommendations"| ONED["oned"]
    ONED -->|"live migrate"| Hosts["KVM Hosts"]

    style DRS fill:#ff6f00,color:#fff

Storage Data Path¶

Driver	Access Pattern	Best For
Shared FS (NFS)	Front-end exports, hosts mount	Simple, live migration works natively
SSH	Front-end copies images via SSH	No shared storage needed
Ceph (RBD)	Direct RBD access from each host	Scale, performance, live migration
LVM	Local LVM on each host	Performance (local I/O)
iSCSI / FC	SAN target from each host	Enterprise, existing SAN infrastructure

Networking Model¶

flowchart TB
    subgraph VM["Virtual Machine"]
        VNIC["vNIC\n(virtio)"]
    end

    subgraph Host["Host Network"]
        Bridge["Linux Bridge\nor OVS"]
        VLAN["802.1Q VLAN\nor VXLAN"]
        PF["Physical NIC"]
    end

    VNIC --> Bridge --> VLAN --> PF

    subgraph SecurityGroups["Security"]
        FW["iptables / nftables\n(security groups)"]
    end

    Bridge --> FW

    style Host fill:#00758f,color:#fff

Sources¶

Benchmarks¶

Scope

Performance characteristics, scaling limits, and resource consumption for OpenNebula.

VM Performance¶

Hypervisor	vCPU Overhead	Memory Overhead	Network
KVM	< 3%	100-200MB per VM	Near-native
LXC	< 1%	50MB per container	Native
Firecracker	< 1%	30MB per microVM	Near-native

Scaling Limits¶

Dimension	Single Frontend	HA (3 nodes)
VMs	2,000-5,000	10,000+
Hosts	200	500+
Networks	4,096 (VLAN)	16M (VXLAN)
Storage images	10,000+	10,000+

Scheduler Performance¶

VMs to Place	Scheduling Time
10	< 1s
100	1-5s
1,000	5-30s

Sourcing Status¶

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources¶

Security¶

Authentication Model¶

OpenNebula uses a pluggable authentication driver system configured in /etc/one/oned.conf under the AUTH_MAD section:

Auth Driver	Method	Use Case
SSH Key	SSH key pair challenge	Developer/admin access
X.509	Client certificate	Enterprise PKI integration
LDAP	Bind against LDAP/AD	Corporate directory integration
SAML 2.0	Federated identity	SSO with identity providers
OpenID Connect	OAuth2/OIDC flow	Keycloak, Okta, Azure AD
Plain	Username/password	Basic local auth

Multiple drivers can be chained — a user authenticating via LDAP can also have an SSH key for API access.

graph TB
    subgraph Users
        Admin["Admin\n(Sunstone UI)"]
        Dev["Developer\n(CLI / API)"]
        CI["CI/CD\n(XML-RPC)"]
    end
    subgraph AuthN["Authentication Layer"]
        LDAP["LDAP Driver"]
        SSH["SSH Key Driver"]
        OIDC["OIDC Driver"]
        X509["X.509 Driver"]
    end
    subgraph OpenNebula["OpenNebula Core"]
        ACL["ACL Engine"]
        oned["oned Daemon"]
    end
    Admin --> OIDC
    Admin --> LDAP
    Dev --> SSH
    CI --> X509
    LDAP --> ACL
    SSH --> ACL
    OIDC --> ACL
    X509 --> ACL
    ACL --> oned

Authorization: ACL System¶

OpenNebula implements a granular ACL (Access Control List) system managed via oneacl CLI or the Sunstone GUI:

ACL Rule Syntax¶

USER/GROUP  RESOURCES  RIGHTS

Object Types¶

Object	Identifier	Examples
VM	`VM`	Running instances
Template	`TEMPLATE`	VM definitions
Image	`IMAGE`	Disk images
Network	`NET`	Virtual networks
Host	`HOST`	Physical hosts
Cluster	`CLUSTER`	Host groups
Datastore	`DATASTORE`	Storage backends

Rights Levels¶

Right	Description
USE	Read and consume the resource
MANAGE	Modify the resource
ADMIN	Full administrative control
CREATE	Create new resources of this type

VDC (Virtual Data Centers)¶

VDCs provide tenant-level isolation — each VDC maps to a subset of hosts, networks, and datastores:

Resources are exclusively allocated to a VDC
Users in one VDC cannot see or access resources in another
VDCs can span multiple OpenNebula zones (federation)

Network Security¶

Security Groups¶

OpenNebula provides distributed firewall rules applied per NIC:

# Create a security group with HTTP/HTTPS rules
onesecgroup create my-sg <<EOF
RULE = [ PROTOCOL = TCP, RULE_TYPE = INBOUND, RANGE = 80:443 ]
RULE = [ PROTOCOL = TCP, RULE_TYPE = INBOUND, RANGE = 22 ]
RULE = [ PROTOCOL = ICMP, RULE_TYPE = INBOUND ]
EOF

Security groups support 5-tuple filtering: protocol, source/destination IP, source/destination port.

Network Isolation Methods¶

Method	Level	Description
VLANs	L2	802.1Q tagged isolation
VXLANs	L2 overlay	Multi-tenant overlay networks
Open vSwitch	L2/L3	Programmable flow-based switching
WireGuard	L3	Encrypted tunnel between hosts
Security Groups	L3/L4	Per-NIC firewall rules

VM Isolation¶

Hypervisor-Level Isolation¶

Hypervisor	Isolation Mechanism
KVM (default)	Hardware virtualization (VT-x/AMD-V), sVirt (SELinux/AppArmor)
LXC	Container namespaces, cgroups, AppArmor profiles
Firecracker	Minimal microVM with strict JAIL separation
vCenter	VMware ESXi isolation, vSphere security policies

Anti-Affinity and Host Pinning¶

Anti-affinity rules: Make sure that VMs from the same group run on different physical hosts (fault tolerance and security isolation)
Host pinning: Restrict VMs to specific hosts for compliance (dedicated tenancy)
Host reserved resources: Reserve CPU/memory for system processes

Encryption¶

In Transit¶

Sunstone UI: HTTPS with configurable TLS certificates
XML-RPC API: SSL/TLS encryption for all API communication
Inter-host communication: WireGuard or IPsec tunnels for VXLAN traffic
Libvirt: TLS for migration and remote connections

At Rest¶

Disk encryption: LUKS-encrypted datastores for VM images
Ceph RBD: Native encryption via Ceph dmcrypt on OSDs
QCOW2: Encrypted qcow2 images with passphrases

Sunstone Web UI Security¶

Sunstone (the web management interface) security configuration:

Enable HTTPS via server configuration (/etc/one/sunstone-server.conf)
Configure SSL/TLS certificates (Let's Encrypt or corporate CA)
Enable SSO integration (OIDC/SAML)
Restrict access via network-level firewall rules
Session timeout configuration
CSRF protection enabled by default

Hardening Checklist¶

Known Pitfalls¶

Pitfall	Risk	Mitigation
Default `oneadmin` unchanged	Privilege escalation	Restrict to jump hosts, use LDAP for daily operations
Unencrypted VXLAN traffic	Network sniffing across hosts	Enable WireGuard/IPsec encryption
Broad ACL rules	Unauthorized resource access	Use least-privilege ACLs per group
No security groups	VMs exposed to all network traffic	Apply default-deny security groups to all NICs
Unpatched KVM/QEMU	Hypervisor escape vulnerabilities	Regular host OS and hypervisor updates

Architecture¶

Component Overview¶

Core Daemon: oned¶

Scheduler¶

User Interfaces¶

Sunstone (Web UI)¶

FireEdge¶

OneFlow¶

OneGate¶

Driver Framework¶

VM Drivers (Virtualization)¶

Network Drivers¶

Storage Drivers¶

Authentication Drivers¶

Datastores¶

VM Lifecycle¶

System Interfaces¶

Key File Locations¶

References¶

How It Works¶

Centralized Architecture Model¶

VM Provisioning Flow¶

OneDRS (Distributed Resource Scheduler)¶

Storage Data Path¶

Networking Model¶

Sources¶

Benchmarks¶

VM Performance¶

Scaling Limits¶

Scheduler Performance¶

Sourcing Status¶

Sources¶

Security¶

Authentication Model¶

Authorization: ACL System¶

ACL Rule Syntax¶

Object Types¶

Rights Levels¶

VDC (Virtual Data Centers)¶

Network Security¶

Security Groups¶

Network Isolation Methods¶

VM Isolation¶

Hypervisor-Level Isolation¶

Anti-Affinity and Host Pinning¶

Encryption¶

In Transit¶

At Rest¶

Sunstone Web UI Security¶

Hardening Checklist¶

Known Pitfalls¶

Related Notes¶