How It Works

Core Mechanisms

Storage Engine

All Victoria databases share foundational design principles:

1. LSM-Tree Storage: Uses a custom implementation of the Log-Structured Merge-Tree optimized exclusively for telemetry appending. Incoming data is written to in-memory buffers, then flushed to immutable on-disk files which are periodically merged (compacted) in the background.
2. ZSTD Compression: All data is compressed using ZSTD with delta-encoding tuned specifically for floats and timestamps, achieving ~50% less disk usage than Prometheus and 10–20x less than Elasticsearch.
3. Deterministic Sharding: In cluster mode, vminsert hashes incoming metrics by their labels to decide which vmstorage nodes should own them. This negates the need for a complex internal distributed consensus algorithm (like Raft/Paxos).
4. Data Localization: Blocks of time-series data are grouped tightly by time buckets, compressed, and written to disk asynchronously.
5. Native Translationless Ingestion: VictoriaTraces opens HTTP/gRPC ports for OTLP data, while VictoriaLogs directly accepts Loki API, Elasticsearch Bulk, and Fluentbit JSON. This bypasses the heavy CPU overhead usually required to translate signals into internal formats.

VictoriaMetrics (Metrics)

flowchart LR
    subgraph Ingestion["Ingestion APIs"]
        PR["Prometheus<br/>remote_write"]
        IL["InfluxDB<br/>line protocol"]
        DD["Datadog<br/>API"]
        OT["OTLP"]
        GR["Graphite"]
    end

    subgraph VM["VictoriaMetrics"]
        WB["Write Buffer<br/>(in-memory)"]
        TSDB["LSM-Tree TSDB<br/>(on-disk, ZSTD)"]
        IDX["Inverted Index<br/>(label → series ID)"]
    end

    subgraph Query["Query"]
        MQL["MetricsQL / PromQL<br/>Engine"]
    end

    Ingestion --> WB --> TSDB
    WB --> IDX
    MQL --> IDX --> TSDB
    MQL --> Grafana["Grafana"]

    style VM fill:#2a2d3e,color:#fff
    style Ingestion fill:#0d7377,color:#fff
    style Query fill:#ff6600,color:#fff

Key insight: VictoriaMetrics stores time-series data using a custom columnar format where timestamps and values are stored in separate columns, enabling efficient batch reads and high compression ratios.

VictoriaLogs (Logs)

flowchart LR
    subgraph Ingestion["Ingestion APIs"]
        LK["Loki Push API"]
        ES["Elasticsearch<br/>Bulk API"]
        SL["Syslog"]
        OT["OTLP"]
        FB["Fluentbit<br/>JSON Lines"]
    end

    subgraph VL["VictoriaLogs"]
        direction TB
        Parse["Parser<br/>(structured + unstructured)"]
        BF["Bloom Filters<br/>(instead of inverted index)"]
        Col["Columnar Storage<br/>(daily partitions)"]
    end

    Ingestion --> Parse --> BF
    Parse --> Col

    LogsQL["LogsQL Engine"] --> BF --> Col
    LogsQL --> Grafana["Grafana"]

    style VL fill:#2a7de1,color:#fff
    style Ingestion fill:#0d7377,color:#fff

Key insight: VictoriaLogs uses Bloom filters instead of traditional inverted indexes. This dramatically reduces RAM and CPU usage — but means full-text search relies on sequential scanning through bloom-filtered partitions rather than instant index lookups.

Storage: Data is organized into daily partitions (e.g., 20260410/), enabling efficient retention management by simply deleting old partition directories.

VictoriaTraces (Traces)

flowchart LR
    subgraph Ingestion["Ingestion"]
        OTLP_H["OTLP HTTP<br/>:10428"]
        OTLP_G["OTLP gRPC<br/>:4317"]
        JG["Jaeger"]
        ZP["Zipkin"]
    end

    subgraph VT["VictoriaTraces"]
        direction TB
        TP["Trace Parser"]
        VLS["VictoriaLogs<br/>Storage Engine"]
    end

    Ingestion --> TP --> VLS

    JQ["Jaeger Query API"] --> VLS
    TQ["Tempo DS API<br/>(experimental v0.8+)<br/>/tags, /search, /v2/traces"] --> VLS
    JQ --> Grafana["Grafana<br/>(Jaeger or Tempo DS)"]
    TQ --> Grafana

    style VT fill:#e65100,color:#fff
    style Ingestion fill:#0d7377,color:#fff

Key insight: VictoriaTraces is built on top of the VictoriaLogs storage engine, inheriting its columnar storage, bloom filters, and compression. It does NOT require external object storage — everything runs on local disk.

Tempo DS compatibility (v0.8+): As of v0.8.0 (March 2026), VictoriaTraces exposes experimental Grafana Tempo datasource APIs (/tags, /search, /v2/traces/*), enabling use with Grafana's native Tempo datasource. TraceQL metrics and pipelines are not yet supported — but basic trace search and lookup work, making VT a partial drop-in for Tempo for simple use cases.

Query Languages

MetricsQL (Metrics)

MetricsQL is a backward-compatible superset of PromQL with extensions that fix common PromQL pain points:

Feature	PromQL	MetricsQL
Auto lookbehind window	❌ Required	✅ Auto-calculated from step
rate() extrapolation	Yes (causes fractional integers)	No (returns accurate integers)
keep_metric_names	❌	✅ Preserves metric names
Numeric suffixes (Ki, Mi, Gi)	❌	✅ 8Ki = 8 * 1024
NaN handling	Returns NaN	Automatically removes NaN
aggr_over_time()	❌	✅ Multiple aggregates in one pass
Graphite filter syntax	❌	✅ {__graphite__="foo.*.bar"}

# MetricsQL example: rate without explicit window (auto-calculated)
rate(http_requests_total{job="api"})

# Keep metric names when applying functions
rate({__name__=~"foo|bar"}) keep_metric_names

# Use numeric suffixes
process_memory_bytes > 2Gi

LogsQL (Logs)

LogsQL is purpose-built for VictoriaLogs with a pipe-based syntax:

# Filter error logs from last 5 minutes
_time:5m AND level:="error"

# Full-text search with pipe transformations
error connection refused | stats count() by (host)

# Extract fields at query time
_time:1h | extract "status=<status_code>" | stats count() by (status_code)

# JSON log parsing
_time:5m | unpack_json | level:="error" | fields _time, msg, trace_id

Key difference from LogQL (Loki): LogsQL does NOT require a mandatory label selector. You can search across all log streams with free-text queries, while Loki requires {label="value"} first.

Data Flow

sequenceDiagram
    participant App as Applications
    participant Agent as vmagent / OTel Collector
    participant Auth as vmauth (Proxy)
    participant MI as vminsert / vlinsert / vtinsert
    participant MS as vmstorage / vlstorage / vtstorage
    participant MQ as vmselect / vlselect / vtselect
    participant G as Grafana

    App->>Agent: Emit metrics / logs / traces
    Agent->>Auth: Push telemetry (HTTP/gRPC)
    Auth->>Auth: Route by URL path
    Auth->>MI: Forward to correct insert node
    MI->>MS: Hash & distribute to storage
    Note over MS: LSM-Tree write + ZSTD compress
    Note over MS: Background merge & compaction

    G->>Auth: Query (PromQL / LogsQL / Jaeger API)
    Auth->>MQ: Forward to select node
    MQ->>MS: Fetch data chunks
    MS-->>MQ: Return compressed data
    MQ-->>Auth: Aggregate, sort, deduplicate
    Auth-->>G: Return results

Pull Sequence

1. vmagent scrapes Prometheus targets and pushes data to vmauth
2. vmauth reads the HTTP path (e.g., /api/v1/write vs /insert/jsonline) and routes the payload to the correct backend

Write Sequence

1. vminsert (or vlinsert/vtinsert) hashes the payload and distributes it to backend storage nodes
2. Storage nodes write to in-memory buffer + WAL, then async-flush to disk

Merge Sequence

1. In the background, storage nodes continually merge small data files into larger chunks (LSM compaction) for faster sequential reads

Read Sequence

1. Grafana sends a query to vmselect via vmauth
2. vmselect asks all relevant vmstorage nodes for data chunks
3. vmselect sorts, deduplicates, and runs aggregation functions natively
4. Results returned to Grafana

Related Notes

• Architecture
• Operations
• Commands & Recipes