Skip to content

bpftrace

High-level tracing language for Linux eBPF — awk-like scripts that compile through LLVM to eBPF bytecode and attach to kernel probes in seconds, answering live system questions without writing a loader program.

Summary

bpftrace (~10.3k stars, Apache-2.0) is the de-facto interactive tracing instrument of the modern eBPF stack. Probes are addressed by name (tracepoint:syscalls:sys_enter_openat, kprobe:vfs_read) with wildcard discovery built in; state lives in maps, aggregation primitives produce ASCII histograms, and everything runs from a single command line or short .bt script. It complements rather than replaces application-level stacks — see the canonical comparison.

Evaluation

  • Why it's better: Minutes-to-answer latency for kernel questions that would otherwise need a C project; ships in distro repositories (apt/dnf), so no bespoke toolchain on target hosts for packaged use; PERCPU-backed aggregation makes hot-path counting cheap while keeping reads correct.
  • When it fits: On-call investigation, ad-hoc syscall/I/O latency analysis, scheduled sampling (intervals), verifying hypotheses before committing to a compiled tool, teaching probe concepts to teams.
  • When it doesn't: Long-lived daemons shipping to fleets, complex user-space interaction, custom network data paths, or anything needing a maintained binary artifact — graduate those to libbpf projects (see eBPF Developer Tutorial for the learning path, libbpf-bootstrap for scaffolding).

Pros and Cons

Pros Cons
One-liners replace hour-long investigations DSL ceiling — complex logic gets awkward fast
Packaged in major distros, Apache-2.0 license Linux-only; needs privileged capability grants
Built-in histogram/aggregation stdlib with PERCPU safety Runtime probing captures sensitive plaintext (args, buffers)
Wildcard probe discovery (-l) makes the kernel explorable Kernel config surface determines which probe types work at all
An ahead-of-time mode builds redistributable bundles upstream Sync map reads documented as expensive CPU iteration

Common Use Cases

  • Counting open/openat syscalls per process via tracepoint blocks (BEGIN init, interval:s:10 async print(@) loops from the upstream stdlib docs).
  • Latency histograms straight off return values — kretprobe:vfs_read { @bytes = hist(retval); }.
  • Exploring available probes before anything else: bpftrace -l 'tracepoint:syscalls:sys_enter_*', wildcards included.
  • Periodic sampling pipelines mixing interval:ms producers with consumer intervals doing sync threshold checks then clear(@).

Licensing & Commercial Use

Apache-2.0. Unrestricted commercial and internal use; contribution model is DCO-based upstream.

Ecosystem & Connections

  • Compiler front end uses Clang; LLVM lowers scripts to BPF bytecode; runtime rides on libbpf and bcc (upstream language docs, verified via Context7).
  • Same kernel surface as every CO-RE lesson in the tutorial — probe types learned here transfer directly.
  • Text output composes naturally with shell pipelines, alerting wrappers, or backends like Grafana agents choosing to sample via bpftrace.
  • Nix-build supported for reproducible dev environments (nix build .?submodules=1 for libbpm submodules per upstream nix.md).

Compatibility & Requirements

Upstream dependency_support.md enumerates required kernel configs — verified set includes CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_HAVE_EBPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_FTRACE_SYSCALLS=y, CONFIG_FUNCTION_TRACER=y, CONFIG_KPROBE_EVENTS=y, CONFIG_UPROBES=y, CONFIG_UPROBE_EVENTS=y, CONFIG_DEBUG_FS=y; a bundled check_kernel_features script validates a target. Effective privilege requirements mirror all eBPF loading (root or CAP_BPF+CAP_PERFMON class capabilities). BTF availability improves typed access; classic symbolic probes function through debugfs paths on older kernels.

Latest Versions

v0.26.1 released 2026-06-02 (GitHub Releases API, checked 2026-08-27). Active C++/C codebase (~3M lines C++ core), pushed 2026-08-24.

Alternatives

  • Hand-written libbpf tools (tutorial path) — full control, permanent artifacts.
  • BCC Python tools — richer ready-made toolkit but heavier runtime deps; bpftrace's CLI UX supersedes them for ad-hoc use.
  • Raw ftrace/debugfs — zero-eBPF fallback when kernels can't load programs at all.
  • Productized auto-instrumentation — Coroot, Beyla-style agents when you want answers, not queries.

Migration & Lock-in Risks

Scripts are small text artifacts pinned to probe naming conventions that are stable kernel ABI surfaces; migration cost concentrates in rare native-type helpers if kernels lose matched tracepoints. No vendor lock-in anywhere in the chain.

Community Health

~10.3k stars, ~1.6k forks class activity, steady releases twice yearly cadence recent years, broad distro packaging adoption (verified GitHub API 2026-08-27). Contribution docs, developer guide, and Nix environment maintained alongside.

Notes In This Folder

  • Architecture — compile pipeline, probe providers, map/aggregation engine, sync-vs-async semantics.
  • Operations — installs, one-liner cookbook, building from source, kernel-feature checks, troubleshooting.
  • Security — capability model, sensitive-data exposure while tracing, third-party script risk, lockdown modes.

Sources

  • GitHub repository — stars/license/push verified via API 2026-08-27
  • Upstream docs via Context7 (language.md, stdlib.md, README, developers.md, dependency_support.md, nix.md), retrieved 2026-08-27
  • Releases — v0.26.1 via GitHub API 2026-08-27
  • Vendored one-liner tutorial port referenced from eBPF Developer Tutorial sources

Questions

  • Does the AOT bundle format pin probe-type availability such that bundles break across minor kernel bumps?
  • What telemetry intermediaries (OpenTelemetry bridges) currently convert bpftrace output streams for long-term storage?