Skip to content

Messaging Patterns: Log-stream vs Queue vs Pub/Sub

How the five messaging systems in this vault map to the three fundamental messaging patterns.

Pattern Definitions

Pattern Semantics Replay? Consumer model
Log-stream Append-only ordered log; consumers track offsets Yes Pull (consumer controls position)
Queue FIFO with competing consumers; message removed on ack No Push or pull; message consumed once
Pub/Sub Fan-out to all subscribers; no persistence by default Depends Push (broker delivers to all)

System × Pattern Matrix

System Log-stream Queue Pub/Sub Notes
Kafka ✅ Primary ✅ via consumer groups ✅ via consumer groups (each gets all) Everything is a log; queue and pub/sub are consumer-group patterns over the log.
NATS ✅ JetStream ✅ Queue groups (Core) + JetStream WorkQueue ✅ Core NATS (at-most-once) Core NATS is pure pub/sub; JetStream adds log-stream and queue semantics.
RabbitMQ ✅ Streams ✅ Primary (quorum queues) ✅ Fanout exchange Queues are the native primitive; streams add log-replay.
Redpanda ✅ Primary ✅ via consumer groups ✅ via consumer groups Same as Kafka (Kafka-API compatible).
Pulsar ✅ Primary ✅ Shared subscription ✅ Exclusive/Failover subscription Subscription types map to all three patterns on the same topic.

Choosing a Pattern

flowchart TD
    Start["What do you need?"]
    Start -->|"Replay old events"| LogStream["Log-stream"]
    Start -->|"Distribute work across workers"| Queue["Queue"]
    Start -->|"Notify all subscribers"| PubSub["Pub/Sub"]
    LogStream --> KafkaRedpanda["Kafka / Redpanda / Pulsar"]
    LogStream --> NatsJS["NATS JetStream"]
    LogStream --> RabbitStream["RabbitMQ Streams"]
    Queue --> RabbitQQ["RabbitMQ quorum queues"]
    Queue --> NatsQG["NATS queue groups / JetStream WorkQueue"]
    Queue --> KafkaCG["Kafka consumer groups"]
    Queue --> PulsarShared["Pulsar Shared subscription"]
    PubSub --> NatsCore["NATS Core (at-most-once)"]
    PubSub --> RabbitFanout["RabbitMQ fanout exchange"]
    PubSub --> KafkaCGAll["Kafka (each consumer group gets all)"]
    PubSub --> PulsarExcl["Pulsar Exclusive subscription"]

Trade-off Summary

Dimension Log-stream Queue Pub/Sub
Ordering Per-partition / per-subject FIFO per queue No ordering guarantee
Replay Yes (by offset / time) No (consumed = gone) No (fire-and-forget)
Throughput High (batch, sequential I/O) Medium (per-message ack overhead) Highest (no persistence)
Latency Medium (batch + fsync) Low–medium Lowest (no persistence)
Durability Strong (replicated log) Strong (replicated queue) None (unless layered)
Use case Event sourcing, CDC, analytics Work distribution, RPC Notifications, real-time updates

Sources