Streaming Brokers: Kafka vs Redpanda vs Pulsar
A side-by-side comparison of the three log-structured event streaming platforms in this vault.
At a Glance
| Dimension |
Apache Kafka |
Redpanda |
Apache Pulsar |
| Language |
Java (JVM) |
C++ (Seastar) |
Java (JVM) |
| License |
Apache-2.0 |
BSL 1.1 (→ Apache after 4y) |
Apache-2.0 |
| Architecture |
Broker = compute + storage |
Broker = compute + storage (single binary) |
Broker (stateless) + BookKeeper (storage) |
| Consensus |
KRaft (metadata) + ISR (data) |
Raft per partition + Raft controller |
Raft (BookKeeper) + ZK (metadata) |
| Multi-tenancy |
ACLs + quotas (no native tenants) |
ACLs + quotas (no native tenants) |
Tenants → Namespaces → Topics (native) |
| Geo-replication |
MirrorMaker 2 / Cluster Linking (Confluent) |
MirrorMaker 2 compatible |
Built-in per-namespace flag |
| Tiered storage |
KIP-405 (GA in 3.6+) |
Native (S3/GCS/Azure) |
Native offloader (S3/GCS/Azure) |
| Schema Registry |
Confluent SR (separate) |
Built-in (Confluent-compatible) |
Built-in |
| Stream processing |
Kafka Streams / ksqlDB / Flink |
WASM data transforms (in-broker) |
Pulsar Functions (in-broker or worker) |
| Wire protocol |
Kafka protocol |
Kafka protocol (compatible) |
Pulsar protocol + KoP/MoP/AoP plugins |
| Operational complexity |
Medium (KRaft simplifies) |
Low (single binary) |
High (broker + bookie + ZK) |
| Tail latency |
~5–15 ms p99 (JVM GC) |
sub-5 ms p99 (no GC) |
~5–20 ms p99 (JVM GC) |
When to Choose
Choose Kafka when
- You need the broadest ecosystem (Connect, Streams, ksqlDB, Flink, Confluent Cloud).
- Your team already has Kafka operational expertise.
- You want Apache-2.0 licensing with no restrictions.
- You need Confluent's commercial features (Cluster Linking, RBAC, Audit Log).
Choose Redpanda when
- You want Kafka API compatibility with lower operational overhead.
- Tail latency matters (fintech, gaming, ad tech).
- You want a single binary with no JVM tuning.
- BSL licensing is acceptable for your use case.
- You want in-broker WASM transforms.
Choose Pulsar when
- You need native multi-tenancy (tenants/namespaces with quotas).
- You need built-in geo-replication without external tooling.
- Independent compute/storage scaling is a hard requirement.
- You want multiple wire protocols (Kafka, MQTT, AMQP) on one cluster.
- You have the ops capacity for a three-tier deployment.
Benchmark caveat
All published benchmarks are vendor-influenced. Run OpenMessaging Benchmark on your own hardware before making sizing decisions.
| Metric |
Kafka |
Redpanda |
Pulsar |
| Throughput (single broker, NVMe) |
500–800 MB/s |
800–1200 MB/s |
150–300 MB/s |
| p99 produce latency |
5–15 ms |
1–5 ms |
5–20 ms |
| Cold-read from tiered storage |
Seconds (KIP-405) |
Seconds (S3 fetch) |
Seconds (offloader) |
| Cluster-wide aggregate |
Linear with brokers |
Linear with brokers |
Linear with brokers + bookies |
Cost Comparison
| Factor |
Kafka |
Redpanda |
Pulsar |
| Compute |
JVM heap + OS page cache |
Per-core Seastar shards |
Broker heap + bookie heap |
| Storage |
Local disk (hot) + S3 (tiered) |
Local NVMe (hot) + S3 (tiered) |
Bookie disks (hot) + S3 (tiered) |
| Ops headcount |
Medium |
Low |
High |
| Managed offering |
Confluent Cloud, AWS MSK |
Redpanda Cloud |
StreamNative Cloud |
Migration Paths
| From → To |
Difficulty |
Notes |
| Kafka → Redpanda |
Low |
Same wire protocol; swap broker binary. |
| Kafka → Pulsar |
Medium |
Use KoP plugin or rewrite clients to Pulsar SDK. |
| Redpanda → Kafka |
Low |
Same wire protocol; swap broker binary. |
| Pulsar → Kafka |
Medium–High |
Rewrite clients; no cursor/subscription equivalent. |
Sources