Architecture

OpenStack is a distributed cloud operating system composed of independent services that communicate through a shared message bus (RabbitMQ) and a central identity service (Keystone). Each service owns its database and exposes a REST API. This microservices architecture enables incremental adoption but increases operational complexity compared to single-daemon platforms like OpenNebula.

See also: infrastructure/openstack/index, infrastructure/openstack/architecture, infrastructure/openstack/operations, infrastructure/openstack/security

Component Overview

graph TD
    subgraph Users["User Interfaces"]
        HORIZON["Horizon<br/>(Web Dashboard)"]
        CLI["OpenStack CLI"]
    end

    subgraph Core["Core Services"]
        KEYSTONE["Keystone<br/>(Identity)"]
        NOVA["Nova<br/>(Compute)"]
        NEUTRON["Neutron<br/>(Networking)"]
        CINDER["Cinder<br/>(Block Storage)"]
        GLANCE["Glance<br/>(Image)"]
        SWIFT["Swift<br/>(Object Storage)"]
    end

    subgraph Orchestration["Orchestration"]
        HEAT["Heat<br/>(Orchestration)"]
        MAGNUM["Magnum<br/>(Container Infra)"]
    end

    subgraph Infrastructure["Shared Infrastructure"]
        MQ["RabbitMQ<br/>(Message Bus)"]
        DB["MariaDB / Galera<br/>(Database)"]
        MEMCACHED["Memcached<br/>(Cache)"]
    end

    HORIZON --> KEYSTONE
    CLI --> KEYSTONE
    KEYSTONE --> DB
    NOVA --> MQ
    NOVA --> DB
    NEUTRON --> MQ
    NEUTRON --> DB
    CINDER --> MQ
    CINDER --> DB
    GLANCE --> DB
    SWIFT --> DB
    HEAT --> MQ
    HEAT --> DB
    NOVA --> NEUTRON
    NOVA --> CINDER
    NOVA --> GLANCE
    MAGNUM --> HEAT

Core Services

Keystone (Identity)

Keystone is the central authentication, authorization, and service catalog for OpenStack:

• Authentication: Validates user identity via passwords, tokens, or federated identity (SAML, OIDC)
• Authorization: Role-Based Access Control (RBAC) with per-project policies
• Service Catalog: Registry of all OpenStack service endpoints (each service registers its API URL)
• Multi-domain: Supports multiple identity backends (SQL, LDAP, Active Directory)
• Token types: Fernet tokens (lightweight, symmetric-key) or JWT tokens
• Every API request in OpenStack starts with Keystone token validation

Nova (Compute)

Nova manages the lifecycle of virtual machine instances:

• nova-api: Accepts and validates REST API requests
• nova-scheduler: Selects compute hosts based on filters and weights (similar to Kubernetes scheduling)
• nova-conductor: Mediates database access for compute nodes (security proxy)
• nova-compute: Runs on each hypervisor node, manages VM lifecycle via libvirt (KVM) or other drivers
• nova-novncproxy / nova-serialproxy: Console access to instances
• Supports live migration, resize, snapshot, and evacuate operations
• Integrates with Neutron for networking, Cinder for volumes, Glance for images

Neutron (Networking)

Neutron provides network connectivity as a service between interface devices managed by other OpenStack services:

• Networks and subnets: Virtual L2 segments with IP address management
• Routers: Virtual L3 routing between subnets, with SNAT for external access
• Floating IPs: One-to-one NAT from external networks to VMs
• Security groups: Per-port stateful firewall rules
• Plugins: ML2 (Modular Layer 2) with drivers for Open vSwitch, Linux bridge, SR-IOV, and vendor SDNs
• Agents: L2 agent (per compute node), L3 agent (routing), DHCP agent, metadata agent

Cinder (Block Storage)

Cinder provides persistent block storage volumes to instances:

• Volume creation, attachment, snapshot, and cloning
• Backend drivers: LVM (local), Ceph RBD, NetApp, Dell EMC, AWS EBS, and dozens more
• Volume types with extra specs (performance tiers, replication)
• Availability zone awareness for volume placement
• Backup to external backends (Swift, Ceph, POSIX)

Glance (Image)

Glance manages VM disk images:

• Stores, discovers, and retrieves bootable disk images
• Supports multiple formats: QCOW2, RAW, VHD, VMDK, ISO, AKI/ARI (kernel/ramdisk)
• Backend stores: local filesystem, Swift, Ceph, S3, HTTP
• Image properties and metadata for scheduling hints
• Image sharing across projects

Swift (Object Storage)

Swift provides highly available, eventually consistent object storage:

• Stores unstructured data objects organized in containers and accounts
• Designed for durability and horizontal scalability (no single point of failure)
• Ring-based data placement across storage nodes
• Supports large objects (segmented uploads), versioning, and expiration
• Independent of the SQL database (uses its own consistency layer)

Orchestration and Supporting Services

Heat (Orchestration)

Heat provisions OpenStack resources using declarative templates (HOT format -- Heat Orchestration Template):

• Define stacks of interconnected resources (instances, networks, volumes, etc.)
• Supports auto-scaling via Ceilometer alarms
• Handles dependency ordering and rollback on failure
• Environment files separate parameter values from templates

Horizon (Dashboard)

Horizon is the web-based management interface:

• Dashboard for managing instances, volumes, networks, and images
• Project and user administration via Keystone integration
• Extensible via Django plugins for additional services
• Serves as the primary GUI for cloud operators and tenants

Magnum (Container Infrastructure)

Magnum provisions and manages Kubernetes, Swarm, and Mesos clusters on OpenStack:

• Uses Heat templates to orchestrate cluster creation
• Integrates with Neutron for cluster networking
• Manages cluster lifecycle (create, update, delete)
• Certificates and node group management

Shared Infrastructure

RabbitMQ (Message Bus)

All OpenStack services communicate asynchronously through RabbitMQ:

• Implements AMQP 0-9-1 protocol
• Topics-based routing (e.g., notifications.info, nova)
• Supports quorum queues for durability
• HA via mirrored queues or quorum queues across a cluster

MariaDB / Galera (Database)

Each service runs its own database, typically MariaDB with Galera clustering for HA:

• Galera provides multi-master synchronous replication
• Each service has its own database schema (nova_api, nova_cell0, neutron, cinder, glance, keystone, heat)
• Connection routed through a load balancer (HAProxy) with health checks

Memcached

Memcached provides caching for Keystone tokens and service catalog lookups:

• Reduces database load for repeated authentication
• Deployed as a pool of instances behind a load balancer
• All services use memcache_servers configuration for shared caching

Request Flow: Instance Creation

sequenceDiagram
    actor User
    participant Horizon as Horizon / CLI
    participant Keystone as Keystone
    participant Nova as Nova API
    participant Sched as Nova Scheduler
    participant Compute as Nova Compute
    participant Glance as Glance
    participant Neutron as Neutron
    participant MQ as RabbitMQ

    User->>Horizon: Launch instance
    Horizon->>Keystone: authenticate (get token)
    Keystone-->>Horizon: token
    Horizon->>Nova: POST /servers (token + flavor + image)
    Nova->>Keystone: validate token
    Nova->>MQ: publish create_instance message
    Nova-->>Horizon: instance BUILD status

    Sched->>MQ: consume schedule message
    Sched->>Sched: filter + weigh compute hosts
    Sched->>MQ: publish selected host

    Compute->>MQ: consume build message for this host
    Compute->>Glance: download image
    Glance-->>Compute: image data
    Compute->>Neutron: create ports + allocate IPs
    Neutron-->>Compute: port info (MAC, IP)
    Compute->>Compute: create VM via libvirt/KVM
    Compute->>Nova: update instance status ACTIVE
    Nova-->>Horizon: instance ACTIVE
    Horizon-->>User: instance ready

Service Port Reference

Service	Port	Protocol
Keystone	5000	HTTP (API v3)
Nova API	8774	HTTP
Neutron	9696	HTTP
Cinder	8776	HTTP
Glance	9292	HTTP
Swift	8080	HTTP (proxy)
Heat	8004	HTTP
Horizon	80/443	HTTP/HTTPS
RabbitMQ	5672	AMQP
MariaDB	3306	MySQL
Memcached	11211	TCP

References

• OpenStack architecture
• OpenStack services
• Nova architecture
• Neutron architecture
• Security guide

---

How It Works

Internal mechanisms, VM lifecycle, network data paths, and service interactions.

VM Provisioning Flow

sequenceDiagram
    participant User as User / Horizon
    participant KS as Keystone
    participant Nova_API as Nova API
    participant Sched as Nova Scheduler
    participant MQ as RabbitMQ
    participant Compute as Nova Compute
    participant Neutron as Neutron
    participant Glance as Glance
    participant Cinder as Cinder

    User->>KS: Authenticate (token)
    KS-->>User: Token
    User->>Nova_API: POST /servers (flavour, image, network)
    Nova_API->>KS: Validate token
    Nova_API->>Glance: Check image exists
    Nova_API->>Neutron: Allocate port
    Nova_API->>MQ: Schedule request
    MQ->>Sched: Pick host
    Sched->>Sched: Filter: RAM, CPU, disk, AZ
    Sched->>Sched: Weigh: balance, spread
    Sched->>MQ: Host selected
    MQ->>Compute: Build instance
    Compute->>Glance: Download image
    Compute->>Cinder: Attach volume (if any)
    Compute->>Neutron: Plug port → OVN
    Compute->>Compute: Launch KVM/QEMU domain
    Compute->>Nova_API: Instance ACTIVE

Neutron Networking (OVN)

flowchart TB
    subgraph Tenant["Tenant Network"]
        VM1["VM 1\n(10.0.0.2)"]
        VM2["VM 2\n(10.0.0.3)"]
    end

    subgraph OVN["OVN Data Path"]
        LS["Logical Switch\n(tenant subnet)"]
        LR["Logical Router\n(inter-subnet)"]
        GW["Gateway Router\n(external)"]
        SNAT["SNAT\n(floating IP)"]
    end

    subgraph Physical["Physical Network"]
        ExtNet["External Network\n(provider bridge)"]
    end

    VM1 --> LS
    VM2 --> LS
    LS --> LR
    LR --> GW
    GW --> SNAT
    SNAT --> ExtNet

    style OVN fill:#ef3e42,color:#fff

Ceph Integration

OpenStack Service	Ceph Layer	Purpose
Cinder	RBD (RADOS Block Device)	Persistent VM volumes
Glance	RBD or RGW (Object)	VM image storage
Nova	RBD (ephemeral disks)	Live migration enabler
Swift	RGW (RADOS Gateway)	S3-compatible object store
Manila	CephFS	Shared file systems

Multi-Region Architecture

flowchart TB
    subgraph Global["Global Services"]
        KS_G["Keystone\n(shared identity)"]
    end

    subgraph Region1["Region 1"]
        Nova1["Nova"]
        Neutron1["Neutron"]
        Cinder1["Cinder"]
        Compute1["Compute Hosts"]
    end

    subgraph Region2["Region 2"]
        Nova2["Nova"]
        Neutron2["Neutron"]
        Cinder2["Cinder"]
        Compute2["Compute Hosts"]
    end

    KS_G --> Region1
    KS_G --> Region2

Sources

• Nova Architecture
• Neutron OVN
• Ceph Integration

---

Benchmarks

Scope

Performance characteristics, scaling limits, and resource consumption for OpenStack.

Nova (Compute) Performance

Operation	Time	Notes
VM boot (local)	10-30s	Depends on image size
VM boot (Ceph)	15-45s	RBD clone is fast
Live migration	5-60s	Depends on memory size
Snapshot	10-120s	Copy-on-write with Ceph

Scaling Benchmarks

Dimension	Tested	Notes
Compute nodes	500+	Production deployments
VMs per compute	100+	Depends on host resources
Total VMs	50,000+	Large cloud deployments
API requests/sec	1,000+	Keystone auth bottleneck

Neutron (Networking)

Feature	OVS Performance	OVN Performance
Port creation	5-10s	2-5s
Network create	1-3s	1-2s
Floating IP	2-5s	1-3s
East-west throughput	8-9 Gbps	9-10 Gbps

Sourcing Status

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources

• OpenStack Documentation
• OpenStack GitHub