Open-source, cloud-native observability platform written in Rust with Apache Parquet columnar storage and S3-native architecture — positioned as a 140x cheaper alternative to Elasticsearch.
OpenObserve (O2) is a high-performance observability platform built in Rust that provides unified logs, metrics, traces, and Real User Monitoring (RUM). Its architecture is built around Apache Parquet columnar storage on object storage (S3/GCS/Azure Blob), eliminating the need for local SSD clusters. It uses the Apache Arrow DataFusion query engine and supports SQL for logs/traces and PromQL for metrics.
Why it's better: Written in Rust (no GC, no JVM), Parquet columnar storage on S3 delivers 140x lower storage costs vs Elasticsearch, single-binary deployment, SQL as the primary query language (lower learning curve), stateless node architecture for effortless horizontal scaling, and built-in RBAC/SSO for enterprise readiness.
What is the maximum tested ingestion rate in a real production environment? — Official docs state OpenObserve handles hundreds of thousands of events per second per node, with 7-30 MB/sec ingestion per vCPU core. Very-high-speed ingestion (using ZO_FEATURE_PER_THREAD_LOCK=true and OTLP gRPC) can boost throughput 60-100% each, for a combined ~4x improvement. No single published maximum, but multi-node clusters target millions of events/sec. See observability/openobserve/operations.
How does DataFusion performance compare to ClickHouse for the same analytical queries? — No direct published benchmark exists. OpenObserve uses Apache DataFusion (Arrow-based) for SQL query execution with SIMD acceleration, broadcast join optimization (up to 99.9% data transfer reduction), and query partitioning for progressive results. ClickHouse has its own vectorized engine optimized for aggregations on columnar storage. Comparative benchmarks vs Elasticsearch pending — OpenObserve claims 140x lower storage cost; independent verification against ClickHouse-based alternatives (e.g., SigNoz, Qryn) not yet published as of April 2026.
What is the AGPL-3.0 impact for SaaS providers embedding OpenObserve? — AGPL-3.0 requires any modified version used to provide a network service to have its full source code made available to users. SaaS providers embedding OpenObserve as a component must either (a) release their modifications under AGPL-3.0, (b) use OpenObserve as a separate service behind an API boundary (arguably not a "derivative work"), or (c) contact OpenObserve Inc. for a commercial license exception. Legal review is strongly recommended before embedding in proprietary SaaS products.
How does compaction performance scale with 10,000+ streams? — Each stream is compacted independently; at 10,000+ streams the compactor must manage many small Parquet files concurrently. OpenObserve merges files up to 256 MB and checks every 10 seconds. High stream counts increase compaction CPU and memory overhead and can exhaust file descriptors. No published benchmark at this scale; monitor ZO_COMPACT settings and file handle limits. Community reports on GitHub issues suggest this is an active area of optimization.
Does OpenObserve support federated querying across multiple S3 buckets? — Yes, via Super Cluster mode. Multiple OpenObserve clusters (each backed by its own S3 bucket) are joined into a super cluster. A leader cluster dispatches queries to worker clusters via gRPC, each processes locally, and results are merged. Metadata (schemas, dashboards, alerts) is synchronized across clusters. This enables cross-region and cross-bucket federated search. See observability/openobserve/architecture.
What is the WAL recovery behavior when an ingester crashes mid-flush? — Incoming data is written to per-org/stream-type WAL files (paired with Memtables). When a Memtable reaches 256 MB or WAL reaches 128 MB, it becomes immutable and is flushed to local Parquet, then uploaded to object storage every 10 seconds. If an ingester crashes mid-flush, the WAL is replayed on restart to reconstruct unflushed Memtables. If the WAL disk itself is lost, data is unrecoverable unless replication factor > 1 (which writes to multiple ingesters). Partially uploaded Parquet files are handled by the immutable-to-object-storage pipeline. See observability/openobserve/architecture.