Skip to content

Architecture

Apache Pulsar's three-tier architecture: stateless brokers, BookKeeper bookies for storage, and ZooKeeper (or alternatives) for metadata.

Component Overview

flowchart TB
    Client["Producer / Consumer"]
    subgraph Brokers["Pulsar Brokers (stateless)"]
        Broker["PulsarBroker\n- ManagedLedger\n- ManagedCursor\n- NamespaceBundle owner\n- Subscriptions\n- Functions runtime"]
    end
    subgraph BK["Apache BookKeeper"]
        Bookie["BookKeeperBookie\n- Journal disk\n- Ledger disk\n- Garbage collection"]
        AutoRecovery["BK AutoRecovery\n(Auditor + Replication Worker)"]
    end
    subgraph Metadata["Metadata Layer"]
        ZK["ZooKeeper / etcd / RocksDB"]
        ConfigStore["Configuration Store\n(per-instance, global)"]
    end
    subgraph Tiered["Tiered Storage"]
        Offloader["TieredStorageOffloader"]
        Cloud["S3 / GCS / Azure Blob"]
    end
    Client --> Broker
    Broker --> Bookie
    Broker --> ZK
    Bookie --> ZK
    AutoRecovery --> ZK
    AutoRecovery --> Bookie
    Broker --> Offloader
    Offloader --> Cloud
    Broker -.cluster info.- ConfigStore

Layer responsibilities

Layer Holds State?
Broker Topic ownership (via NamespaceBundles), subscriptions, ManagedLedger / ManagedCursor handles, Pulsar Functions instances Stateless w.r.t. messages
BookKeeper bookies Persistent message data as ledgers (sequences of entries) Stateful
ZooKeeper / etcd Cluster topology, ledger metadata, broker–bundle ownership, schemas Stateful (small)
Configuration Store Tenants, namespaces, policies, geo-replication topology Stateful (small)
Tiered storage Offloaded ledgers Stateful (large)

Tenant / Namespace / Topic Hierarchy

my-tenant/
   ├── ns-prod/
   │     ├── persistent://my-tenant/ns-prod/orders   (partitioned 12 ways)
   │     └── persistent://my-tenant/ns-prod/audit-log
   └── ns-staging/
         └── non-persistent://my-tenant/ns-staging/debug
  • Tenant — security boundary. Managed by an admin.
  • Namespace — policy boundary (retention, replication, schema, dispatch quotas).
  • Topic — actual message stream. persistent://... (durable) or non-persistent://... (best-effort).

Topic Storage Model — Ledgers and Segments

The data of a topic is stored as a chain of ledgers (BookKeeper entities), each ledger split into segments. The ManagedLedger abstraction of the brokers manages ledger creation, rollover, and trimming.

flowchart LR
    subgraph Topic["persistent://my-tenant/ns/orders"]
        L1["Ledger 1\n(closed)"]
        L2["Ledger 2\n(closed)"]
        L3["Ledger 3\n(open)"]
    end
    subgraph Bookies["BookKeeper bookies"]
        BkA["Bookie A"]
        BkB["Bookie B"]
        BkC["Bookie C"]
    end
    L1 --> BkA
    L1 --> BkB
    L1 --> BkC
    L2 --> BkA
    L2 --> BkB
    L2 --> BkC
    L3 --> BkA
    L3 --> BkB
    L3 --> BkC
    OffStore["Tiered storage S3"]
    L1 -. offload .-> OffStore

Write quorum / Ack quorum / Ensemble size

BookKeeper writes use three numbers (Eq, Wq, Aq):

Param Meaning
Ensemble size (Eq) Number of bookies that can hold this ledger.
Write quorum (Wq) Number of bookies the write is striped across.
Ack quorum (Aq) Number of bookies that must ack a write before it is considered durable.

Common production values: Eq=3, Wq=3, Aq=2. Performance and durability trade-offs flow from these settings.

Subscription Types

flowchart LR
    Topic["Topic"]
    Sub1["Subscription S1\n(Exclusive)"]
    Sub2["Subscription S2\n(Failover)"]
    Sub3["Subscription S3\n(Shared)"]
    Sub4["Subscription S4\n(Key_Shared)"]
    Topic --> Sub1
    Topic --> Sub2
    Topic --> Sub3
    Topic --> Sub4
    C1["Consumer A"]
    C2["Consumer B"]
    C3["Consumer C"]
    Sub1 --> C1
    Sub2 --> C1
    Sub2 -. standby .- C2
    Sub3 --> C1
    Sub3 --> C2
    Sub3 --> C3
    Sub4 -- "key=k1" --> C1
    Sub4 -- "key=k2" --> C2
    Sub4 -- "key=k3" --> C3
Type Behavior
Exclusive Only one consumer at a time. The second consumer is rejected.
Failover One active, others standby. New active picked on failure.
Shared (Round-robin) Messages distributed across all consumers.
Key_Shared Messages with the same key always route to the same consumer (sticky hash, auto-split).

Geo-Replication

flowchart LR
    subgraph US["us-east cluster"]
        BrokerUS["broker"]
        BookieUS["bookie"]
        ZkUS["ZK"]
    end
    subgraph EU["eu-west cluster"]
        BrokerEU["broker"]
        BookieEU["bookie"]
        ZkEU["ZK"]
    end
    subgraph APAC["ap-southeast cluster"]
        BrokerAP["broker"]
        BookieAP["bookie"]
        ZkAP["ZK"]
    end
    Global["Configuration Store\n(global ZK)"]
    ZkUS -.-> Global
    ZkEU -.-> Global
    ZkAP -.-> Global
    BrokerUS <-->|geo-replicate| BrokerEU
    BrokerEU <-->|geo-replicate| BrokerAP
    BrokerUS <-->|geo-replicate| BrokerAP

A namespace can be configured with a list of clusters. The brokers run a replicator subscriber that produces published messages into the topic of the remote cluster. Each cluster keeps its own copy.

Mechanism summary:

  • Per-namespace replication clusters set via pulsar-admin namespaces set-clusters.
  • Replicated cursor tracks per-cluster delivery position.
  • Conflict resolution: last-write-wins on identical message ids. In practice, design topics to be partitioned per-source-cluster.

Pulsar Functions

flowchart LR
    Input["Input topic A"]
    Func["PulsarFunction\n(user code)"]
    Output["Output topic B"]
    DLQ["Dead-letter topic"]
    Input --> Func
    Func --> Output
    Func -. on error .-> DLQ

Functions can run inside the broker (lightweight) or as a separate Functions Worker cluster (production). Source/sink IO connectors live in the same runtime.

Schemas & Transactions

  • Schema Registry is per-namespace. Supports Avro, JSON Schema, Protobuf, and key-value composites.
  • Transactions (Pulsar transactions) span produces + acks across multiple topics. The transaction coordinator runs in a system topic.

Performance Characteristics

Workload Notes
Single broker, R3 ledger, NVMe bookies ~150–300 MB/s sustained (varies wildly with hardware)
Cluster-wide aggregate Linear with broker + bookie counts
Tiered-storage cold read Adds object-store latency to first byte
Geo-replication lag Typically WAN RTT + replicator commit interval
Pulsar Functions overhead ~1–10 ms/event for moderate transforms

OpenMessaging Benchmark publishes apples-to-apples Kafka/Pulsar/RabbitMQ comparisons. Trust on-hardware measurement above any vendor blog.

Comparison Hooks

  • vs Kafka — Pulsar's compute/storage split is its big differentiator. Kafka's single-binary model is simpler ops.
  • vs Redpanda — Redpanda is Kafka-API and single-binary. Pulsar is multi-tenant and multi-protocol but heavier.
  • vs NATS — both have multi-tenant designs. NATS uses accounts, Pulsar uses tenants/namespaces. Pulsar wins on durable-stream scale, NATS on latency.

Security

Apache Pulsar's security model spans broker authentication, BookKeeper authentication, ZooKeeper hardening, and per-namespace authorization with role-based and tenant-isolated access.

Authentication

Mechanism Use Case
TLS client certificate (x509) Cert-based auth. CN/SAN maps to a Pulsar role.
JWT tokens Stateless tokens signed by a configured key. Supports symmetric (HS256) and asymmetric (RS256).
OAuth 2.0 / OIDC Token Exchange via IdP — Keycloak, Auth0, Okta, AWS Cognito.
Athenz Yahoo's identity service. Predates OAuth in Pulsar's history.
SASL / Kerberos For environments standardizing on Kerberos.
HTTP basic Dev only.

Configure JWT authentication

# broker.conf
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
tokenSecretKey=file:///etc/pulsar/jwt/secret.key
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
brokerClientAuthenticationParameters=file:///etc/pulsar/jwt/broker.token

Configure TLS (broker)

tlsEnabled=true
tlsCertificateFilePath=/etc/pulsar/certs/server.crt
tlsKeyFilePath=/etc/pulsar/certs/server.key
tlsTrustCertsFilePath=/etc/pulsar/certs/ca.crt
tlsRequireTrustedClientCertOnConnect=true

Authorization

Pulsar authorizes by role. Roles are derived from the auth principal. The AuthorizationService consults the AuthorizationProvider to allow/deny.

Per-namespace permissions

pulsar-admin namespaces grant-permission my-tenant/ns-prod \
  --role orders-svc \
  --actions produce,consume

pulsar-admin namespaces revoke-permission my-tenant/ns-prod \
  --role orders-svc

Per-topic permissions

pulsar-admin topics grant-permission persistent://my-tenant/ns-prod/orders \
  --role downstream-svc \
  --actions consume

Tenant administrators

# broker.conf
superUserRoles=admin,break-glass

A tenant-admin role can create namespaces and grant permissions inside its own tenant.

Subscription-auth modes

subscriptionAuthMode policy values:

  • None — any role can create or use a subscription.
  • Prefix — subscription name must start with role name.

This is useful in shared topics where multiple consumers must isolate cursors.

Encryption

In transit

  • TLS 1.2 / 1.3 on all listeners: client–broker (6651), broker–broker, broker–bookie, broker–ZK.
  • mTLS supported on every listener.
  • Cipher suites configurable (tlsCiphers, tlsProtocols).

End-to-end (E2E)

Pulsar supports per-message E2E encryption:

  • Producer encrypts message payload with a symmetric key.
  • Symmetric key is wrapped with the public RSA/ECDSA key of the consumer and embedded in the message metadata.
  • Brokers and bookies see only the ciphertext.
  • Multiple consumers can each have their own key wrapper.
producer.newMessage()
  .addEncryptionKey("my-app-key")
  .cryptoKeyReader(new RawFileKeyReader("public-key.pem", "private-key.pem"))
  .value(payload)
  .send();

This protects against a compromised broker reading data.

At rest

Bookie ledger files are not encrypted by Pulsar itself. Use OS-level dm-crypt or cloud-managed disk encryption. For tiered storage, configure SSE-S3 or SSE-KMS on the offload bucket.

# offload to S3 with KMS
managedLedgerOffloadDriver=aws-s3
s3ManagedLedgerOffloadBucket=pulsar-cold
s3ManagedLedgerOffloadRegion=us-east-1
s3ManagedLedgerOffloadServerSideEncryption=SSE-KMS
s3ManagedLedgerOffloadServerSideEncryptionKMSKeyId=alias/pulsar-cold-cmk

Audit Logging

  • Broker logs record auth attempts and authorization denials.
  • Pulsar 4.x adds optional structured audit log topic that records admin operations as JSON.
  • Forward to your SIEM via Pulsar IO Sink or a Functions-based exporter.

Threat Model

Threat Mitigation
Broker compromise reading data Use end-to-end encryption for sensitive payloads.
Bookie ledger leakage At-rest disk encryption. Restrict bookie host access.
Geo-replication credential theft Per-cluster JWT issuer. Rotate it. mTLS for cross-cluster connections.
Function code injection Validate Function jars. Require signed jars. Isolate the Function Worker network.
ZK metadata tampering ZK ACLs (digest/sasl). ZK on private network only.
Cross-tenant subscription poisoning subscriptionAuthMode=Prefix. Tenant-scoped roles.
Schema poisoning is_allow_auto_update_schema=false. Require admin to register schemas.
Stale-token replay Short JWT TTLs. Revocation list. Refresh-token rotation.
Broker-bookie spoofing mTLS between broker and bookies. Bookie auth via BookKeeper SASL.
MITM on ZK ZK SASL + TLS (3.6+). ZK on private network.
Tiered-storage bucket misconfig Bucket policies block public access. CloudTrail + GuardDuty (or equivalents).
DoS via unbounded subscription Per-namespace dispatch quota + backlog quota.

CVE History (selected)

CVE Year Affected Summary
CVE-2024-23114 2024 Apache Pulsar Functions Worker Improper input validation can allow unauthorized access to Function metadata. Fix in 3.0.4 / 3.1.3 / 3.2.1.
CVE-2023-37579 2023 Pulsar Functions Worker Authorization bypass on the Function admin API.
CVE-2023-31994 2023 Pulsar Proxy Auth bypass via crafted request when proxy is in OAuth2 mode.
CVE-2023-31993 2023 Pulsar broker Auth header parsing issue.
CVE-2023-30474 2023 Pulsar TLS hostname validation can be bypassed in client.

Subscribe to Apache Security Advisories and the [email protected] list.

Hardening Checklist

  • TLS 1.3 on every listener. mTLS for broker-broker and broker-bookie.
  • JWT or OAuth 2.0 with short token TTLs and audience pinning.
  • Tenant-scoped roles. superUserRoles minimal.
  • subscriptionAuthMode=Prefix in any shared-tenant topic.
  • Schema auto-update disabled in prod. Admin-only schema registration.
  • Tiered-storage bucket: SSE-KMS with CMK. Bucket-policy blocks public access.
  • ZK ACLs and TLS configured. No anonymous access.
  • BookKeeper bookie auth (SASL or TLS-cert). Bookies on private network.
  • End-to-end encryption for sensitive payloads.
  • Pulsar Functions jars reviewed/signed. Worker isolated from production network.
  • Audit log topic shipped to SIEM.
  • Subscribed to Apache Pulsar Security Advisories.

Cross-references