Skip to content

Reference

Look-up facts for running an LLM Wiki

File roles, operation contracts, scale thresholds, the qmd search layer, the link and citation contract this vault's agents follow, and a security hardening checklist. For the reasoning behind these rules see Explanation. For step-by-step tasks see How-to Guides.

Wiki Layout and File Roles

The three layers come from Karpathy's gist (ref note). The folder names are common conventions, not requirements of the pattern.

Layer Typical path Owner Agent access Purpose
Raw sources raw/ Human (curator) Read-only Immutable articles, papers, transcripts, clippings
Raw assets raw/assets/ Human Read-only Images downloaded locally so the LLM can view them and URLs cannot rot
Wiki pages wiki/ or knowledge/<domain>/<topic>/ LLM Read/write Entity, concept, topic, and comparison pages
Content index index.md LLM Read/write Catalog of every page with a one-line summary, updated on every ingest
Activity log log.md LLM Append-only Chronological record of ingests, queries, and lint passes
Schema CLAUDE.md / AGENTS.md Human + LLM Read (edits reviewed) Folder shape, page conventions, workflows, link rules

log.md Entry Format

The gist recommends a consistent, parseable prefix per entry, so plain Unix tools can read recent history. Its example heading:

## [2026-04-02] ingest | Article Title
Field Values Notes
Date YYYY-MM-DD in square brackets Makes grep "^## \[" list entries in order
Operation ingest, query, lint Matches the three core operations
Subject Source title, question, or lint scope Free text after the pipe

Operations at a Glance

Operation Trigger Reads Writes Output
Ingest Human adds a source to raw/ New source, index.md, related pages Source summary/reference note, updated entity and concept pages (the gist cites 10-15 pages per source), index.md, log.md Short summary of changes
Query Human asks a question index.md (or qmd results), then the relevant pages Optional new page if the answer is worth keeping, log.md Cited answer (page, table, chart, or Marp deck)
Lint Schedule or human request Whole wiki Fixes, merge proposals, log.md Contradictions, stale claims, orphans, missing pages, data gaps

Scale Thresholds

Wiki size Navigation mechanism Basis
Up to ~100 sources, "hundreds of pages" Read index.md, then open the relevant pages Stated in the gist: "works surprisingly well at moderate scale"
Hundreds to ~2,000 pages Hierarchical indexes (root index, then domain index, then page) Vault heuristic, not measured
Thousands of pages Hybrid search (qmd) through its CLI or MCP server Vault heuristic; the gist recommends a search engine "as the wiki grows"

Thresholds are heuristics

No published benchmark measures where index traversal breaks down. Some community write-ups (unverified) report trouble around 100-150 articles when index summaries stop discriminating between pages. Treat all numbers here as starting points (as of 2026-09).

qmd Quick Reference

qmd (Query Markup Documents) is the on-device search engine the gist recommends. Facts are from the qmd README and the npm registry, checked 2026-09-25.

Fact Value
Author Tobi Lütke (tobi/qmd)
Package @tobilu/qmd (npm), binary qmd
Latest version 2.8.3 (2026-08-16); 2.0.0 shipped 2026-03-10
License MIT
Runtime Node.js >= 22 or Bun >= 1.0; macOS needs Homebrew SQLite for extension support
Storage SQLite with FTS5 (BM25) plus a vector index; models cached in ~/.cache/qmd/models/
Retrieval Query expansion, parallel BM25 and vector search, Reciprocal Rank Fusion (k=60), LLM re-ranking, position-aware blending

Default local GGUF models (auto-downloaded from Hugging Face on first use):

Model Role Approx. size
embeddinggemma-300M-Q8_0 Embeddings (override with QMD_EMBED_MODEL) ~300 MB
qwen3-reranker-0.6b-q8_0 Re-ranking ~640 MB
qmd-query-expansion-1.7B-q4_k_m Query expansion (fine-tuned) ~1.1 GB

CLI commands used in this topic:

Command Purpose
qmd collection add <path> --name <name> Register a folder (optional --mask "**/*.md")
qmd context add qmd://<collection> "<text>" Attach descriptive context returned with matching documents
qmd embed Build embeddings (qmd embed -f re-embeds after a model change)
qmd search "<text>" Fast BM25 keyword search
qmd vsearch "<text>" Vector (semantic) search
qmd query "<text>" Hybrid search with re-ranking (best quality)
qmd get <path-or-#docid> Fetch a document
qmd mcp / qmd mcp --http MCP server over stdio, or HTTP on localhost:8181
qmd status Index health and MCP daemon state

MCP tools exposed: query (typed lex/vec/hyde sub-queries), get, multi_get, status. Unknown MCP parameters are silently ignored, for example a singular collection instead of collections.

These rules are for agents that write into and answer from this vault. agents.md (section "Static Site Generation (SSG) Compatibility") is normative. This section adds worked syntax and verified build behavior. On conflict, agents.md wins. The reasons behind the contract are in Explanation: Why Wikilinks Are the Citation Unit.

Citation rules for answers and new wiki prose:

  1. Cite the narrowest note that supports the claim. A section anchor beats a whole note, and a whole note beats a domain index.
  2. Cite the note, never the URL. URLs live in a note's Sources section, nowhere else.
  3. If no note covers the claim, create the note first (or a reference note), then cite it.
  4. When two notes disagree, cite both and flag the conflict in the newer note. Do not silently pick a winner.

Scoping Rules

The public build runs mkdocs-roamlinks-plugin with docs_dir: knowledge, so every scoped wikilink resolves from the knowledge/ root. Obsidian resolves by path-suffix matching and is more forgiving, so a link that works locally can still break on the site.

Form Verdict Example
Folder-scoped from knowledge/ required [[ai-agents/llm-fundamentals/how-to-guides#speculative-decoding]]
Folder-scoped with alias allowed outside tables [[ai-agents/llm-wiki/index|LLM Wiki]]
Dotted path, scoped required .md suffix [[observability/observability-2.0/index.md|Observability 2.0]]
Bare note name forbidden (ambiguous) [[index]], [[explanation]]
Block reference forbidden [[note^abc123]], breaks MkDocs compilation
Relative forbidden [[../note]], use the scoped form
Unscoped title forbidden [[My Note]], works in plain Obsidian, ambiguous here
Same-folder Markdown link required ./ [Explanation](./explanation.md#threat-model)
Parent or subfolder Markdown link allowed [Domain](../index.md), [Comparison](comparisons/x.md)

Root pages have no wikilink exception. Link the knowledge-base home and the disclaimer with Markdown links ([Home](../../index.md), [Disclaimer](../../disclaimer.md)).

Behavior of mkdocs-roamlinks-plugin 0.3.2 (the latest release on PyPI, 2023-04-19; requirements.txt allows >=0.1.0,<1.0.0). Verified against the plugin source on 2026-09-25.

Input What the plugin does Result
Wikilink containing / Joins the path to docs_dir, appends .md only if the path has no . at all Correct link, no existence check (a missing target surfaces only in MkDocs or kb_audit.py)
Wikilink without / Walks every file under docs_dir, compares names case- and -_-insensitively, keeps the last match in filesystem walk order Often a different topic's file
Markdown link whose target has no / and ends in .md, .png, or .jpg Same filename walk, last match wins Usually a different topic's file
Markdown link starting ./ or containing / Not touched Resolved normally by MkDocs
Wikilink #Heading part Slugged by the plugin: lowercase, drop characters other than word characters, spaces, and -, then turn each run of spaces into one - Matches MkDocs slugs for plain headings
[[#Heading]] Same-page anchor Works
Numeric alias such as 300 or 300x200 Treated as image width or height Adds { width="300" }

Syntax Gotchas

Each one below either broke a real link in this vault or was reproduced against the plugin source.

  • Bare sibling Markdown links resolve to the wrong topic. Always prefix same-folder links with ./. A 2026-09-25 audit found 300+ links that the published site sent to the wrong page.
  • Dotted folder names need an explicit .md. Write [[observability/observability-2.0/index.md]], not [[observability/observability-2.0/index]].
  • The plugin rewrites links inside code unless the build guards it. It runs regexes over raw Markdown, including inline code and fenced blocks. Since 2026-09-26 the vault build runs a hook (meta/mkdocs_hooks/protect_code_wikilinks.py) that masks [ inside code before roamlinks and restores it in the HTML, so literal examples in code render unchanged. Without that hook, show literal syntax with HTML entities: <code>&#91;&#91;path]]</code>. In Obsidian, a link inside code is dead text either way.
  • An alias must start with a non-digit and be at least two characters long. [[a/b|2026 plan]] and [[a/b|X]] compile to a broken target (b|2026 plan.md). kb_audit.py does not catch this.
  • Anchor slugs differ for some headings. For a heading with a spaced hyphen (Kafka - Log Compaction) the plugin produces kafka---log-compaction while MkDocs produces kafka-log-compaction. For non-ASCII headings (Café Setup) the plugin keeps café-setup while MkDocs produces cafe-setup. Link such headings with a Markdown link and the MkDocs slug.
  • No escaped-pipe aliases in tables. [[a/b\|Label]] compiles to b/.md. Use a Markdown link in tables: [Label](../topic/index.md).
  • Case matters at build time. macOS (APFS) matches Note.md to note.md. Linux CI and MkDocs do not. Write targets in their exact on-disk case.
  • Spaces in Markdown-link targets need %20. A target such as (ref - name.md) does not parse as a link. (./ref%20-%20name.md) does. Wikilinks handle spaces natively.
  • No trailing-slash directory links. MkDocs cannot resolve (topic/). Always link the explicit index.md.
  • The public build publishes only knowledge/. A wikilink into projects/ or meta/ publishes as a dead link.
  • Relative Markdown links use explicit file paths, never directory paths.
  • Every new folder needs a .pages file. A topic .pages without a ... token must list every page, or the unlisted pages are missing from the sidebar.
  • A renamed or removed page needs a redirect_maps entry in mkdocs-public.yml.

Lint Hooks

The lint pass (see How-to Guides: Linting) checks these invariants. meta/scripts/kb_audit.py automates the ones marked "yes". It skips fenced and inline code.

Invariant kb_audit.py check Severity
No same-folder Markdown link without ./ yes (bare-sibling-link) ERROR
Every Markdown link and scoped wikilink target exists yes (broken-link) ERROR
Every wikilink is folder-scoped yes (bare-wikilink) ERROR
Dotted wikilink paths end in .md yes (dotted-wikilink) ERROR
title, type, status present and within enums yes (frontmatter) ERROR
Pages listed in a .pages nav without ... yes (nav) WARN
last_checked newer than about 6 months yes (stale) WARN
Only quadrant and ref files in a topic yes (shape) WARN
No orphan notes, every cited note exists no, agent or Obsidian backlinks pane n/a
Alias starts with a non-digit no n/a

Security Hardening Checklist

Moved from the security discussion in Explanation.

  • API keys live in environment variables or a secret manager, never in the wiki.
  • The agent runs only inside the scoped knowledge directory, never from / or ~.
  • The "read-only raw sources" rule is written into the AGENTS.md / CLAUDE.md schema.
  • Full disk encryption is enabled on the host (FileVault, BitLocker, or LUKS).
  • The agent requires human approval for destructive terminal commands.
  • qmd and agent CLI versions are pinned, and GGUF model sources are verified.
  • The wiki is under git with a private remote for rollback and audit.

Sources