Skip to content

Security

Context

DFlash 2 is a decoding-time optimization, not a network service — its security surface is different from an inference engine's. The angles that matter: losslessness as an output-integrity property, the drafter-checkpoint supply chain, privacy characteristics of the draft/verify data flow, and the (small) set of ways the optimization itself can go wrong operationally.

Output Integrity: Losslessness as a Guarantee

The core security-relevant property of DFlash 2 is that it is provably output-preserving:

  • Greedy decoding produces token-for-token the same output as the target model alone.
  • Sampled decoding draws from the target model's exact distribution via rejection sampling over draft proposals.
  • The selector and convolutions only change which candidates get proposed — verification and rejection sampling are untouched, so the committed distribution is unchanged regardless of drafter quality.

This matters for regulated or audit-sensitive deployments: adding DFlash 2 does not change model behavior, alignment characteristics, or output policy — an output audit run on autoregressive decoding remains valid under DFlash 2.

What can break the guarantee in practice is runtime correctness, not the algorithm:

  • z-lab/dflash Issue #146 reports CUDA-graph crashes under load — a correctness-class bug. A crash is visible, but any engine bug that silently mis-schedules speculation (e.g., wrong accepted-prefix accounting) would corrupt output.
  • The TPU port's "sequence length inflation" bug (draft state drifting from the target's accepted-token count) is the canonical example of this bug class: fixed by synchronizing the proposer strictly with the true accepted token count.
  • Operational rule: diff-test after every engine or drafter upgrade (see Monitoring and Audit Hooks below) — losslessness makes the check exact.

Supply Chain: Drafter Checkpoints and Build Provenance

DFlash 2 adds a second artifact to the model supply chain — the drafter — loaded alongside the target model into the serving process with code-execution potential (custom architecture code, trust-remote-code in SGLang configs).

Artifact Source Risk Posture Mitigation
Official drafters incoai/* and z-lab/* on Hugging Face (mirrored pairs, e.g. Qwen3.8-27B-DFlash2) Moderate — new org, fast-moving project Pin exact revisions; prefer the z-lab/ mirror when paring incoai vs z-lab; scan safetensors before load
Community drafters e.g. RadixArk/Qwen3.8-27B-DSpark, DaoCloud/Muse-Glimmer-30B-DSpark, vendor drafter orgs (nvidia, RedHatAI, modal-labs, XiaomiMiMo, poolside, meta-models) Higher — third-party training provenance unknown Treat as untrusted model code; review model cards for training data claims; quarantine in a staging server first
GGUF drafts incoai/Qwen3.8-27B-DFlash2-GGUF (llama.cpp path) Moderate — conversion adds a transform step Verify conversion provenance; compare greedy output against the safetensors path once
Engine builds vLLM from PR head ref (#52816), llama.cpp PR #27342, ollama branch PR #17865, oMLX signed dmg High while pinned to PR refs — code not yet through mainline review Pin exact commit SHAs; move to merged/mainline releases as soon as available; verify the oMLX signature
dflash pip package PyPI (dflash, MIT) Low — client/benchmark harness only Standard pip hash-pinning practice

Additional notes:

  • Drafter-target mismatch is a quality/availability issue, not an integrity one — a wrong drafter degrades speed and can crash the server, but lossless verification still filters any bad tokens. Do not rely on that as a safety net for code-level compromise, which verification cannot catch.
  • No known CVEs or published compromises of DFlash artifacts as of 2026-08-28 — this reflects the project's age (v1 Feb 2026), not audit depth. TBD — re-check the issue tracker and advisories before production use.

Privacy and Data Flow

  • All computation stays in the serving process. Drafting, selection, and verification operate on the same prompts and KV state as ordinary decoding; no DFlash-specific data leaves the server. The dflash CLI documents no telemetry (its only outbound channel is a feedback form) — verify against the installed version if this matters to you.
  • No persistent cross-request state. Draft-side KV state is transient per cycle (Architecture — Internals); nothing drafter-side persists beyond the request. Standard engine KV-cache isolation and prefix-sharing rules therefore still govern cross-request leakage — DFlash does not weaken them by design, but engine-level cache-isolation bugs would now span both models.
  • The API perimeter is unchanged. Clients talk to the standard OpenAI-compatible endpoint; apply the usual controls: API-key auth, TLS, network segmentation of the GPU serving VLAN, per-tenant rate limits.
  • Prompt content shapes acceptance, not exposure. Higher task predictability (math/code) raises acceptance; nothing about the drafter makes prompt data more observable to other tenants.

Access Control and Hardening

  • Model-loading is the privileged operation. Whoever can point a SGLang/vLLM server at a drafter repo executes new model code with trust-remote-code. Restrict server launch and model-manager access (relevant for GUI servers like oMLX on shared machines — its admin dashboard binds to 127.0.0.1:8891 by default; keep it that way on multi-user hosts).
  • Resource controls bound speculation overhead. mem-fraction-static, max-running-requests, and cuda-graph-max-bs-decode cap the drafter's memory and scheduling footprint; the drafter adds ~2B params (BF16) of resident memory.
  • Denial of budget, not denial of service. At high concurrency, speculation overhead compresses gains toward 1x — a misconfigured rollout can silently waste drafter memory and cycle time. Watch acceptance length (see Monitoring and Audit Hooks below).

Threat Model Summary

Threat Vector Impact Likelihood Control
Malicious drafter checkpoint HF repo / mirror swap Code execution in server process Low today, grows with ecosystem Pin revisions, scan artifacts, staging server, prefer official mirrors
Tampered engine build PR-ref installs Arbitrary code execution Low Pin SHAs, move to mainline releases, verify signatures (oMLX dmg)
Silent output corruption Engine speculation bug (e.g. #146-class, TPU seq-inflation-class) Integrity loss on committed tokens Low Greedy diff-testing vs autoregressive mode; track issue tracker; acceptance-length monitoring
Cross-request data leakage Engine KV/radix-cache bug spanning target and draft KV Confidentiality breach Low (design preserves isolation) Standard engine isolation updates; keep engine patched
Budget waste (spec overhead) High-concurrency rollout without benchmarking Cost/throughput regression Medium Concurrency-tier benchmarking before rollout; disable speculation for throughput-shaped traffic
Model unavailability Checkpoint-locked ecosystem; no training code for private fine-tunes Cannot accelerate custom models Certain today (Issue #1) NeMo training recipe; Z Lab/Modal engagement for custom drafters
Mask-token aliasing (training) Reusing pad/eos as mask_token_id in a self-trained drafter Quiet acceptance-length erosion (quality, not integrity) Medium Reserve a dedicated rarely-used token; match the id at inference; monitor train/accept_len

Monitoring and Audit Hooks

  • Track acceptance length per request — the natural health metric (mean committed tokens per verification step). Sudden drops indicate a wrong block size, a mismatched drafter, quantization drift, or a runtime bug; the dflash benchmark harness reports it directly.
  • Keep a lossless diff in the release checklist — run a fixed prompt set greedily through autoregressive and DFlash modes after every engine/drafter change; outputs must be byte-identical, making output-integrity regression testing exact rather than statistical.
  • Record artifact provenance at deploy time — log the exact HF revision SHAs of target and drafter, the engine build (PR ref or release), and the oMLX/pkg versions alongside service version so any supply-chain incident maps to a precise artifact set.
  • Watch the upstream issue tracker — correctness-class reports (e.g. CUDA-graph crashes) land as GitHub issues first; there is no separate security-advisory channel for the drafter ecosystem as of 2026-08-28.

Sources