Skip to content

Architecture

Related Notes

secrets/vault/index | secrets/vault/architecture | secrets/vault/operations | secrets/vault/security

Overview

HashiCorp Vault is an identity-based secrets and encryption management system. It centralizes secret storage, rotates credentials, generates dynamic secrets on demand, encrypts application data, and maintains detailed audit logs. Vault provides a unified HTTP API to any secret type while enforcing strict access control through path-based ACL policies.

Component Architecture

graph TD
    subgraph "Client Layer"
        CLI["Vault CLI"]
        API["HTTP API Client"]
        SDK["Vault SDK / Agent"]
    end

    subgraph "Vault Server"
        LISTENER["Listener (TCP)"]
        CORE["Vault Core / Router"]
        POLICY["Policy Store"]
        EXPIRY["Expiration Manager"]
        TOKEN["Token Store"]
        ROLLBACK["Rollback Manager"]
        AUDIT["Audit Broker"]

        AUTH["Auth Methods"]
        SECRET["Secrets Engines"]
        AUDIT_DEV["Audit Devices"]

        BARRIER["Encryption Barrier (AES-256-GCM)"]
    end

    subgraph "Storage Backends"
        RAFT["Integrated Raft Storage"]
        CONSUL["Consul"]
    end

    subgraph "Seal Mechanism"
        SHAMIR["Shamir's Secret Sharing"]
        KMS["Auto-Unseal (AWS KMS / GCP CKMS / Azure KV / Transit)"]
    end

    CLI --> LISTENER
    API --> LISTENER
    SDK --> LISTENER
    LISTENER --> CORE
    CORE --> AUTH
    CORE --> SECRET
    CORE --> AUDIT_DEV
    CORE --> POLICY
    CORE --> EXPIRY
    CORE --> TOKEN
    CORE --> ROLLBACK
    CORE --> AUDIT
    CORE --> BARRIER
    BARRIER --> RAFT
    BARRIER --> CONSUL
    SHAMIR -->|provides unseal key| BARRIER
    KMS -->|provides unseal key| BARRIER

Encryption Barrier

The barrier is Vault's cryptographic boundary. Every piece of data that passes between the Vault core and the storage backend is encrypted using AES-256-GCM. The storage backend is considered untrusted -- even if an attacker gains full access to the storage backend, they only see opaque ciphertext.

Key properties: - All data written to storage is encrypted before leaving the barrier. - All data read from storage is decrypted after entering the barrier. - The barrier key never leaves Vault's memory. - Vault uses mlock system calls to prevent the operating system from swapping encrypted memory pages to disk.

Seal and Unseal

When a Vault server starts, it begins in a sealed state. In this state, the encryption keys needed to operate the barrier are not resident in memory, and Vault cannot process any requests except seal-status queries.

sequenceDiagram
    participant Init as Initialization
    participant Vault as Vault Server
    participant Storage as Storage Backend
    participant KMS as Auto-Unseal KMS

    Init->>Vault: vault operator init
    Vault->>Vault: Generate master key
    Vault->>Vault: Split master key via Shamir (N shares, K threshold)
    Vault->>Storage: Write encrypted root key to storage
    Init-->>Init: Output unseal key shares (offline storage)

    Note over Vault: Server starts in sealed state

    rect rgb(240, 240, 240)
        alt Manual Unseal (Shamir)
            loop Provide K of N shares
                Operator->>Vault: vault operator unseal <share>
            end
            Vault->>Vault: Reconstruct master key from shares
        else Auto-Unseal (Cloud KMS)
            Vault->>KMS: Request decrypt of wrapped root key
            KMS-->>Vault: Decrypted root key
        end
    end

    Vault->>Vault: Derive barrier key from root key
    Vault->>Storage: Read and decrypt all data through barrier
    Vault->>Vault: Load auth methods, secrets engines, audit devices
    Note over Vault: Vault is now UNSEALED

Key Hierarchy

graph BT
    SHARES["Unseal Key Shares<br/>(Shamir: N shares)"]
    MK["Master Key<br/>(reconstructed from K-of-N shares)"]
    RK["Root Key"]
    BK["Barrier Key (AES-256-GCM)"]
    DATA["Encrypted Data in Storage"]

    SHARES -->|K shares reconstruct| MK
    MK -->|decrypts| RK
    RK -->|derives| BK
    BK -->|encrypts/decrypts| DATA

    style MK fill:#f96,stroke:#333,stroke-width:2px
    style BK fill:#6cf,stroke:#333,stroke-width:2px
    style DATA fill:#ccc,stroke:#333

Auto-Unseal

Vault supports auto-unseal using trusted cloud key management systems, eliminating the need for operators to manually provide unseal key shares on every restart:

Auto-Unseal Backend Mechanism
AWS KMS Vault stores the root key encrypted by a KMS key; on startup, it calls KMS Decrypt
GCP CKMS Uses Google Cloud KMS to unwrap the root key
Azure Key Vault Uses Azure Key Vault to decrypt the root key
Transit (another Vault) Uses the Transit secrets engine of a separate Vault cluster
PKCS#11 HSM Uses a hardware security module via the PKCS#11 interface

Storage Backends

Vault does not store data internally. It relies on a pluggable storage backend for durable persistence. All data in the storage backend is encrypted by the barrier.

Backend Status Notes
Integrated Raft Recommended Built-in consensus; no external dependency. Default since Vault 1.4+.
Consul Supported HashiCorp Consul as storage. Requires separate Consul deployment.
Filesystem Non-production Single-node only; no HA support.
In-Memory Testing only Data lost on restart.

Integrated Raft Storage

Raft is the recommended storage backend for new deployments. It provides: - Built-in consensus protocol for HA (leader election, log replication). - No external dependency on Consul. - Better network performance (no extra network hop). - BoltDB for on-disk persistence with fsync for durability.

storage "raft" {
  path    = "/vault/data"
  node_id = "vault-node-1"
}
cluster_addr = "https://vault-node-1:8201"

Auth Methods

Auth methods verify client identity and issue Vault tokens with associated policies. They are plugins enabled at specific paths.

Method Path Use Case
Token token/ Default method; direct token authentication (root, periodic, service tokens)
AppRole approle/ Machine-to-machine auth; role_id + secret_id pair
Kubernetes kubernetes/ Pods authenticate via ServiceAccount JWT tokens
JWT/OIDC jwt/, oidc/ Federated identity (Okta, Auth0, Azure AD)
LDAP ldap/ Corporate directory integration
Cert cert/ Mutual TLS client certificate authentication
Userpass userpass/ Simple username/password (primarily for operators)
GitHub github/ GitHub personal access token authentication
AWS aws/ AWS IAM or EC2 identity-based authentication

Authentication workflow: 1. Client submits credentials to an auth method. 2. Auth method validates credentials and returns a list of associated policies. 3. Vault core creates a client token with the associated policies and a lease duration. 4. Client uses the token for subsequent requests.

Secrets Engines

Secrets engines are plugins that store, generate, or encrypt data. Each is enabled at a unique path and operates independently.

graph LR
    CORE["Vault Core"] --> KV["KV (v1/v2)<br/>secret/"]
    CORE --> TRANSIT["Transit<br/>transit/"]
    CORE --> DB["Database<br/>database/"]
    CORE --> PKI["PKI<br/>pki/"]
    CORE --> AWS["AWS<br/>aws/"]
    CORE --> SSH["SSH<br/>ssh/"]
    CORE --> ID["Identity<br/>identity/"]
    CORE --> TOTP["TOTP<br/>totp/"]

    KV -->|Static secrets| APP1["Applications"]
    TRANSIT -->|Encrypt/decrypt| APP2["Encryption-as-a-Service"]
    DB -->|Dynamic creds| DB_INST["PostgreSQL / MySQL / etc."]
    PKI -->|TLS certs| PKI_CLIENTS["TLS Clients"]
    AWS -->|IAM creds| AWS_ACCT["AWS Account"]
Engine Type Description
KV (v1/v2) Static Key-value secret storage. v2 adds versioning and metadata.
Transit Encryption-as-a-Service Encrypt/decrypt data without storing it. Applications offload cryptographic operations.
Database Dynamic Generates short-lived database credentials on demand (PostgreSQL, MySQL, MongoDB, etc.)
PKI Dynamic Full Certificate Authority; issues and manages X.509 certificates.
AWS Dynamic Generates dynamic AWS IAM credentials and STS tokens.
SSH Dynamic Issues one-time SSH keys or signed certificates.
Identity Internal Entity and alias management across auth methods; groups and group aliases.
TOTP Dynamic Time-based one-time password generation.

Dynamic vs Static Secrets

Static secrets (KV) are stored and retrieved as-is. Dynamic secrets (Database, AWS, PKI) are generated on demand with a lease. When the lease expires, Vault automatically revokes the credential, ensuring no long-lived credentials exist.

ACL Policies

Vault uses path-based ACL policies to control access. Policies are written in HCL or JSON and define which paths a token can access and which operations (create, read, update, delete, list, sudo) are permitted.

# Example: read-only access to KV v2
path "secret/data/myapp/*" {
  capabilities = ["read"]
}

# Example: full access to database credentials
path "database/creds/myapp-role" {
  capabilities = ["read"]
}

# Deny all other paths (implicit deny is default)

Vault operates on a default-deny model: all actions are forbidden unless explicitly granted by a policy. When a token has multiple policies, the permissions are unioned (most permissive wins).

Audit Devices

Audit devices log every request and response to Vault. They are critical for compliance and security monitoring.

Device Type Backend
File Writes JSON audit logs to a file
Syslog Sends audit logs to syslog
Socket Streams audit logs to a TCP/Unix socket

Multiple audit devices can be enabled simultaneously. The audit broker distributes requests to all configured devices. Audit logs can be configured to omit sensitive data (using hmac for values) while preserving the audit trail.

Replication (Enterprise)

Vault Enterprise supports two replication modes for disaster recovery and performance scaling:

Mode Purpose Behavior
DR (Disaster Recovery) Business continuity Replicates all data (including tokens and leases) to a secondary cluster. Promoted during primary failure.
Performance Latency reduction Replics secrets to local clusters for fast reads. Writes still go to the primary. Supports filtered replication by path.

Both modes use a primary-secondary architecture with encrypted streaming over TLS.

Request Processing Flow

sequenceDiagram
    participant Client
    participant Listener
    participant Core as Vault Core
    participant Auth as Auth Method
    participant Policy as Policy Store
    participant Engine as Secrets Engine
    participant Audit as Audit Broker
    participant Barrier as Encryption Barrier
    participant Storage as Storage Backend

    Client->>Listener: HTTP request with token
    Listener->>Core: Route request
    Core->>Audit: Log request (all audit devices)
    Core->>Policy: Resolve token to policies
    Policy-->>Core: Return applicable policies
    Core->>Core: Check ACL permissions
    alt Authorized
        Core->>Engine: Forward to secrets engine
        Engine->>Barrier: Read/Write encrypted data
        Barrier->>Storage: Persist ciphertext
        Storage-->>Barrier: Confirm
        Barrier-->>Engine: Plaintext data
        Engine-->>Core: Return secret
        Core->>Audit: Log response
        Core-->>Client: HTTP response with secret
    else Unauthorized
        Core->>Audit: Log denied request
        Core-->>Client: 403 Forbidden
    end

Deployment Topology

A typical production Vault deployment:

  • 3 or 5 node Raft cluster for consensus and HA.
  • Load balancer (e.g., AWS ALB) distributing client traffic across nodes.
  • Auto-unseal via cloud KMS to avoid manual unseal after node restarts.
  • Audit devices logging to file and/or external SIEM.
  • Integrated storage (Raft) preferred over Consul for new deployments.
                 Load Balancer
                 (active-active)
                /       |       \
         Node 1      Node 2      Node 3
        (leader)    (follower)  (follower)
         Raft        Raft        Raft
         storage     storage     storage

How It Works

Seal/unseal mechanics, encryption barrier, auth flow, and dynamic secret generation.

Seal / Unseal Process

sequenceDiagram
    participant Admin as Admin (unseal keys)
    participant Vault as Vault Server
    participant Barrier as Encryption Barrier
    participant Storage as Storage Backend (Raft)

    Note over Vault: Server starts SEALED
    Admin->>Vault: Unseal key 1/3
    Admin->>Vault: Unseal key 2/3
    Admin->>Vault: Unseal key 3/3
    Vault->>Vault: Reconstruct master key (Shamir's Secret Sharing)
    Vault->>Barrier: Decrypt encryption key with master key
    Barrier->>Storage: Can now read/write encrypted data
    Note over Vault: Server is UNSEALED ✅

Authentication & Token Flow

sequenceDiagram
    participant App as Application
    participant Auth as Auth Method (K8s/OIDC/AppRole)
    participant Vault_H as Vault Core
    participant SE as Secret Engine
    participant Cloud_V as Cloud API (AWS/GCP)

    App->>Auth: Authenticate (SA token / OIDC / role_id + secret_id)
    Auth->>Vault_H: Validate identity
    Vault_H->>Vault_H: Check policies
    Vault_H-->>App: Vault token (with TTL + policies)
    App->>Vault_H: Read secret (with token)
    Vault_H->>SE: Generate dynamic credential
    SE->>Cloud_V: Create IAM user / DB role
    Cloud_V-->>SE: Credentials
    SE-->>App: Dynamic credentials (TTL: 1h)
    Note over SE: Vault auto-revokes after TTL

Secret Lease Lifecycle

stateDiagram-v2
    [*] --> Active: Generate credential
    Active --> Active: Renew (extend TTL)
    Active --> Revoked: TTL expires
    Active --> Revoked: Manual revoke
    Revoked --> [*]: Credential deleted from target system

Storage Backend and Raft

Vault uses Raft consensus for its integrated storage backend (recommended for production):

Component Role
Leader Processes all write requests, replicates to followers
Follower Accepts writes forwarded from leader, serves read requests
Candidate Node soliciting votes during leader election

Raft guarantees: - Linearizable reads: Reads from the leader always return the most recent data - Strong consistency: All writes go through the leader and require acknowledgment from a quorum (majority) of nodes - Automatic leader election: If the leader fails, remaining nodes elect a new leader within seconds

The recommended cluster size is 3 or 5 voting nodes. 3 nodes tolerate 1 failure; 5 nodes tolerate 2 failures.

Seal/Unseal and Key Shards

Vault uses Shamir's Secret Sharing to split the master key into shards:

  1. Initialization: Vault generates a master key and splits it into N shards (default: 5 shards, threshold: 3)
  2. Sealed state: On startup, Vault cannot decrypt its storage because the master key is not in memory
  3. Unseal: An operator provides at least threshold shards (e.g., 3 of 5). Vault reconstructs the master key and decrypts the encryption key
  4. Auto-unseal: With cloud KMS (AWS KMS, GCP KMS, Azure Key Vault), Vault can automatically unseal by retrieving the master key from the KMS service

Transit Auto-unseal

Vault can also use another Vault instance's Transit engine for auto-unseal, creating a hierarchy where a "root" Vault unseals "leaf" Vault instances.

Sources

Benchmarks

Scope

Performance characteristics, scaling limits, and resource consumption for HashiCorp Vault.

Request Performance

Operation Single Node 3-Node HA (Raft) Notes
KV read 5,000-10,000 req/s 5,000-8,000 req/s Read on leader
KV write 2,000-5,000 req/s 1,500-3,000 req/s Raft consensus overhead
PKI sign 500-1,000 req/s 400-800 req/s CPU-intensive
Transit encrypt 3,000-7,000 req/s 2,500-5,000 req/s AES-GCM
Token create 2,000-5,000 req/s 1,500-3,000 req/s Storage write

Resource Sizing

Deployment CPU Memory Storage Concurrent Clients
Dev 1 1Gi 10Gi < 50
Small prod 2 4Gi 50Gi 50-200
Medium prod 4 8Gi 100Gi 200-1,000
Large prod 8 16Gi+ 500Gi 1,000+

Scaling Limits

Dimension Limit Notes
Active secrets 1M+ Storage backend dependent
Secret engines 1,000+ Impacts startup time
Policies 10,000+ Policy evaluation < 1ms
Leases 300,000 default max_lease_count tunable
Raft cluster 5 voting nodes 3-5 recommended

Sourcing Status

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources