Redpanda

Kafka-API compatible streaming platform written in C++ on Seastar — single binary, no JVM, no ZooKeeper, with native tiered storage and WASM data transforms.

Overview

Redpanda is a drop-in Kafka replacement that re-implements the Kafka wire protocol in C++ on the Seastar reactor framework. Each topic partition is its own Raft group; the controller is also Raft-based, so no separate ZooKeeper or KRaft quorum is needed. Operationally, it is one static binary plus a single config — markedly simpler than running a multi-node Kafka cluster with Strimzi or Confluent.

Key Facts

Attribute	Detail
Website	redpanda.com
GitHub	redpanda-data/redpanda
Stars	~10k+ ⭐
Latest Version	25.x line (CY 2026)
Language	C++ (Seastar)
License	BSL 1.1 (community), with Apache-2.0 conversion after 4 years per release
Company	Redpanda Data
Commercial	Redpanda Enterprise, Redpanda Cloud (BYOC, Dedicated, Serverless)
Compatibility	Kafka API (producer, consumer, AdminClient), Schema Registry, Kafka Connect

Evaluation

Pros	Cons
Single static binary; no JVM or ZooKeeper	BSL license (Apache after 4 years) — review for SaaS embedding
Thread-per-core (Seastar) gives low tail latency	Smaller community than Kafka
Per-partition Raft simplifies architecture	Some Kafka ecosystem tooling expects JMX (mostly non-blocking but sometimes annoying)
Native tiered storage to S3/GCS/Azure	Tiered Storage Read Replicas require Enterprise license
Iceberg integration for analytics tables	Vendor-controlled benchmarks; verify on your own hardware
WASM transforms in-broker (data transforms)	Different ops mental model — debugging Seastar requires familiarity
Drop-in for Kafka clients including Connect	Confluent's KIP-based feature flow doesn't always land same-day

Architecture

flowchart TB
    Client["Kafka client / kafkactl / kcat"]
    subgraph RedpandaNode["Redpanda Node — single C++ process"]
        rpk["rpk CLI"]
        Reactor["Seastar reactor\n(thread-per-core)"]
        Controller["Controller\n(Raft group)"]
        PartitionRaft1["Partition Raft 1"]
        PartitionRaft2["Partition Raft 2"]
        PartitionRaft3["Partition Raft 3"]
        ArchivalUploader["Archival Uploader"]
        SchemaRegistry["Schema Registry"]
        Pandaproxy["Pandaproxy (REST)"]
        WasmTransform["WASM Transform Runtime"]
    end
    subgraph Cloud["Object Storage"]
        S3["S3 / GCS / Azure ADLS"]
    end
    Client --> Reactor
    Reactor --> Controller
    Reactor --> PartitionRaft1
    Reactor --> PartitionRaft2
    Reactor --> PartitionRaft3
    PartitionRaft1 --> ArchivalUploader
    PartitionRaft2 --> ArchivalUploader
    PartitionRaft3 --> ArchivalUploader
    ArchivalUploader --> S3
    Reactor --> SchemaRegistry
    Reactor --> Pandaproxy
    PartitionRaft1 --> WasmTransform

See messaging/redpanda/architecture for component details.

Use Cases

• Kafka workload modernization — drop into existing Kafka pipelines for lower latency / TCO.
• Low-latency fintech, gaming, ad tech — tail-latency-sensitive workloads.
• Edge / single-node — Redpanda runs comfortably on small ARM nodes; no JVM tax.
• Iceberg-native streaming — Redpanda can write topic data directly into Iceberg tables.
• In-broker stream processing (WASM) — small per-message transforms without a separate Flink/Stream cluster.

Licensing & Pricing

• Redpanda Community Edition (BSL 1.1):
• Free for production use, including commercial.
• Source-available; production restrictions: cannot offer the software as a service to third parties.
• Per-release Apache-2.0 conversion 4 years after release — older versions are eventually fully Apache-licensed.
• Redpanda Enterprise — commercial license; required for Tiered Storage Read Replicas, Continuous Data Balancing, Audit Logging.
• Redpanda Cloud:
• BYOC — runs in your VPC.
• Dedicated — single-tenant Redpanda-operated cluster.
• Serverless — multi-tenant pay-per-data.

Ecosystem

• rpk — single CLI for cluster admin, topic ops, debugging, profiling.
• Redpanda Console — web UI for topic browsing, consumer groups, schema, ACLs, security (redpanda-data/console).
• Connect — based on the Benthos project (acquired by Redpanda); declarative stream processing.
• WASM transforms — small per-message transforms compiled to WASM and executed in-broker.
• Iceberg integration — topic-to-Iceberg-table writer.
• MCP server — mcp-server-redpanda for AI-agent integration.

Compatibility & Requirements

Requirement	Detail
Kafka API	Compatible across producer, consumer, AdminClient, transactions, ACLs
Kafka Connect	Compatible; runs as a separate process
Schema Registry	Built-in HTTP API compatible with Confluent Schema Registry wire format
Object storage	S3, GCS, Azure ADLS Gen2; MinIO supported for dev only
OS	Linux (Ubuntu, RHEL, Debian)
CPU	x86_64, ARM64
Storage	Local NVMe required for low latency; S3 for tiered tier
Memory	2 GB / core minimum recommended
Network	TCP 9092 (Kafka), 9644 (Admin), 8081 (Schema Registry), 8082 (Pandaproxy), 33145 (RPC)

Latest Versions

• 25.x — current stable line (CY 2026): Iceberg native writer GA, RBAC enhancements, MCP server.
• 24.x — added native data transforms (WASM), Continuous Data Balancing.
• 23.x — Tiered Storage GA, Connect (Benthos) acquisition.
• 22.x — initial multi-tenant Cloud Dedicated.

Track at redpanda.com/blog and github.com/redpanda-data/redpanda/releases.

Alternatives

• Apache Kafka — the reference implementation; broader vendor and tooling ecosystem.
• WarpStream — S3-native Kafka API, no broker disks; sacrifices p99 latency.
• Bufstream — newer S3-native Kafka API.
• Apache Pulsar — segregated compute/storage; multi-tenant from day one.
• NATS — different protocol entirely; not Kafka-compatible.

Migration & Lock-in

• API-level lock-in: low. Kafka clients work unchanged; switching back to Apache Kafka is a config change.
• Tooling lock-in: medium. rpk, Redpanda Console, and WASM data transforms have no equivalent in Apache Kafka.
• Tiered storage format is Kafka-compatible — kafka-export.sh style tools can read offloaded segments.
• Iceberg outputs are entirely portable.

Community Health

• Active vendor (~150-person Redpanda Data); rapid release cadence.
• Open-source contributions accepted; CLA required.
• Discord community and active GitHub issue tracker.
• Annual user conference (Streaming Summit).

Sources

• Redpanda Documentation (queried via Context7 /redpanda-data/docs).
• Redpanda GitHub.
• redpanda-data/console.
• BSL License terms.
• Redpanda blog — performance benchmarks.
• Confluent counter-position on Redpanda benchmarks (read both sides).

Open Questions

• For sustained 1M msg/sec sustained workloads, how does Redpanda's tail latency compare to Apache Kafka with KRaft mode + tiered storage on identical NVMe?
• What is the operational overhead of running WASM transforms at scale (CPU per partition)?
• For BSL-licensed deployments, what business uses (e.g. embedding in a SaaS product) require an Enterprise license vs Community?
• How do Continuous Data Balancing decisions interact with sustained writes — does it cause periodic latency spikes?
• For Iceberg integration, is there schema evolution parity with Schema Registry?