Skip to content

Jev vs AnyJev vs LLM Structured Outputs

Summary

Agent loops make many small judgments: route this ticket, is this tool call risky, how relevant is this chunk. There are three ways to get a typed answer to such a question from text. Jev (TypeSafe AI) is a hosted decision model that returns calibrated probabilities over options you declare. AnyJev (Nokia Applied Research) reproduces that interface over open LLMs you host, reading decisions off next-token log-probabilities. LLM structured outputs (JSON-schema modes and constrained decoding) make a generative model emit schema-valid output. All three can guarantee the shape of the answer. They differ on calibration, hosting, cost model, and whether they can generate at all.

Compared topics: Jev (which also covers AnyJev) · LLM Fundamentals (structured output and constrained decoding). Facts come from those topic pages, checked 2026-09-25.

High-Level Comparison

Dimension Jev AnyJev LLM structured outputs
What it is Hosted "System One" decision model Open-source library: Jev-style decisions over open LLMs A generation mode: API JSON-schema modes, or self-hosted constrained decoding
Vendor / license TypeSafe AI; closed weights, hosted API only; SDKs MIT Nokia Applied Research (with Tencent Hunyuan researchers); Apache-2.0 Provider APIs (proprietary) or open libraries: XGrammar, llguidance, Outlines
Latest version jev-1.13.0 (2026-09-15) anyjev 0.0.2 on PyPI (2026-09-21); 0.1.0 with L2 unreleased Library-specific, e.g. XGrammar 0.2.8 (2026-09-24), llguidance 1.8.0 (2026-08-11), Outlines 1.3.3 (2026-08-06)
Output choice (up to 255 options), score (up to 10 levels), noul (probability 0-1), each with a distribution and confidence Same primitives; letter readout caps choice at 26 options Any JSON, regex, or grammar, including free text fields
Shape guarantee Structural: output cannot violate the declared types Structural: options are read off label-token probabilities Schema-conformant (API strict modes) or 100% grammar-conformant (constrained decoding); JSON mode alone guarantees only valid JSON
Calibration RLCD training aims for calibrated confidence; self-reported, re-verify on your traffic L0 debiases (no labels, not calibrated); L1 calibrates with 100-500 labels per question; L2 heads need 100-300 labels None guaranteed; the model's stated confidence is text, not a probability
Can it generate? No: no prose, code, summaries, or extracted values No Yes
Where data goes TypeSafe or a gateway (Vercel, OpenRouter, Cloudflare); retention terms not published (TBD) Stays in your infrastructure Your model host, or the API provider under its retention terms
Latency 70-500 ms end to end (vendor); ~150-500 ms observed; roughly flat in question count K prefills per choice at L0 (cyclic shifts); depends on your model and GPU Autoregressive: grows with output length; masking overhead near zero for XGrammar (vendor-reported)
Price $0.042/MTok input, output free (TypeSafe discloses it may be subsidized) Your GPU cost Provider token prices, or your GPU cost
Access (2026-09-25) Direct signups paused since 2026-09-22; existing accounts and the three gateways work pip install anyjev Generally available

How Each Produces a Decision

The diagram shows the three request paths for the same question ("which queue gets this ticket?") and where the typed result comes from in each.

flowchart LR
    APP["Agent loop<br/>(state + question)"]

    subgraph J["Jev"]
        J_API["POST api.typesafe.ai/v1/systemone<br/>or Vercel / OpenRouter / Cloudflare"] --> J_M["jev-1.13.0<br/>one pass, questions in parallel"]
        J_M --> J_OUT["choice + probabilities<br/>+ confidence"]
    end

    subgraph A["AnyJev"]
        A_LLM["Open LLM you host<br/>(measured on Qwen3)"] --> A_LP["Label-token log-probs<br/>K cyclic shifts, prior divided out"]
        A_LP --> A_CAL["L1 temperature scaling<br/>or L2 head (labelled)"]
        A_CAL --> A_OUT["choice + probabilities<br/>+ level"]
    end

    subgraph S["Structured outputs"]
        S_LLM["Generative LLM<br/>(API or vLLM / SGLang)"] --> S_MASK["Grammar mask per token<br/>XGrammar / llguidance / Outlines"]
        S_MASK --> S_OUT["Schema-valid JSON<br/>(no calibrated probability)"]
    end

    APP --> J_API
    APP --> A_LLM
    APP --> S_LLM
    J_OUT --> BR{"Your code: thresholds,<br/>act / escalate / abstain"}
    A_OUT --> BR
    S_OUT --> BR

Evidence

Measure Jev AnyJev Structured outputs
LocalLLaMA/typed-decisions accuracy (20 questions, 2,000 held-out decisions) 0.727, as published by Jev's authors; not rerun Qwen3-32B L1: 0.699; L2 heads on Qwen3-4B to 32B: 0.771-0.799, pooled ECE 0.03-0.05 Not measured on this benchmark
Agent reliability SREGym-Lite: pass rate 40% to 48% with Jev as plan ranker and submission gate (independent) Not reported Not applicable
Auto-decidable traffic at <=5% error Not reported Qwen3-8B, BANKING77 20-way: 7.7% (raw readout) to 52.0% (L1) Not applicable

First-party benchmark

The LocalLLaMA/typed-decisions tables are AnyJev's own benchmark, regenerated 2026-09-22. AnyJev's L2 result needs labels per question and per model; Jev needs none. Only Qwen models have been measured. Treat the head-to-head as indicative, not settled.

Shared Caveats

  • Schema safety is not judgment safety. All three can return a valid but wrong answer. Jev and AnyJev expose confidence so code can abstain; structured outputs need a separate check.
  • Prompt injection moves answers. TypeSafe's limitations page and arXiv 2609.28613 show adversarial text can shift Jev's probabilities. The same risk applies to any LLM-based judge. Use scores as a pre-filter in front of hard enforcement (permissions, allowlists, sandboxes), never as the enforcement itself.
  • Calibration drifts. Re-measure after changing the model version, the questions or criteria, or the input distribution. AnyJev's L1/L2 must be refit on distribution shift.
  • Retention. Structured outputs on the Anthropic API cache the JSON schema for up to 24 hours, even under ZDR. TypeSafe's and the gateways' retention terms are unpublished. See Zero Data Retention.

Which One Should I Pick?

The flowchart gives a starting point for one decision point in an agent loop. Many systems mix all three: structured outputs for generation, Jev or AnyJev for the gates around it.

flowchart TD
    START{"Does the answer need<br/>generated content?"} -->|"Yes: text, code, extracted values"| SO["LLM structured outputs"]
    START -->|"No: pick, score, or yes/no<br/>over options you supply"| DATA{"May the state leave<br/>your infrastructure?"}
    DATA -->|"No"| ANY["AnyJev on your own LLM"]
    DATA -->|"Yes"| ACC{"Can you get a TypeSafe<br/>account or use a gateway?"}
    ACC -->|"Yes"| JEV["Jev"]
    ACC -->|"No"| ANY
    ANY --> LAB{"Can you label 100-500<br/>examples per question?"}
    LAB -->|"Yes"| ANY_L["AnyJev L1 or L2<br/>(calibrated)"]
    LAB -->|"No"| ANY_L0["AnyJev L0 (debiased, not calibrated)<br/>or structured outputs"]
    JEV --> STAKES{"High-stakes or<br/>irreversible action?"}
    ANY_L --> STAKES
    STAKES -->|"Yes"| HARD["Add hard enforcement<br/>and a human path"]
    STAKES -->|"No"| ACT["Act above your<br/>confidence threshold"]

Choose Jev When

  • The judgment is closed-outcome, high-volume, and latency-sensitive (routing, guardrails, tool gating, relevance)
  • You want calibrated probabilities without collecting labels
  • Sending the state to TypeSafe or a gateway is acceptable, and you can pin jev-1.13.0 for replayable behavior

Choose AnyJev When

  • State must stay in your VPC, or you need an open-weights, auditable stack
  • You can label 100-500 examples per question to reach L1/L2
  • Your options fit in 26 labels per choice

Choose LLM Structured Outputs When

  • The output includes generated or extracted content, not just a choice among supplied options
  • You already run a serving engine (vLLM enables structured outputs by default) or use an API strict-schema mode
  • You do not need calibrated confidence, or you add your own evaluation

Sources