QMD: On-Device Hybrid Search for Markdown and Agents
What QMD is
QMD ("Query Markup Documents") is a command-line search engine for Markdown that runs entirely on the local machine. Its README describes it as "an on-device search engine for everything you need to remember": markdown notes, meeting transcripts, documentation and knowledge bases, searchable "with keywords or natural language" and "ideal for your agentic flows". The repository is tobi/qmd, the npm package is @tobilu/qmd, and package.json lists Tobi Lutke as author. It is written in TypeScript, runs on Node.js 22 or later or on Bun, and is MIT-licensed.
The pitch is a single tool that combines three retrieval techniques that usually live in separate systems: BM25 full-text search through SQLite FTS5, vector search through sqlite-vec, and LLM re-ranking. The embedding, expansion and reranking models are small GGUF files run through node-llama-cpp, with no hosted service involved by default. Everything sits in one SQLite file under ~/.cache/qmd/. The same index is reachable from the CLI, from a TypeScript SDK, from an MCP server (stdio or HTTP) and from a plain HTTP /query endpoint, so a coding agent can search a notes folder the same way a person does in a terminal.
The audience is developers with lots of Markdown who want an AI assistant to retrieve from it without uploading it anywhere. This page covers version 2.9.0, released on 17 November 2026 according to the CHANGELOG, and includes a hands-on run on a CPU-only Linux VM where only the embedding model was available.
How it works
Indexing and storage
A collection is a directory plus a glob mask (default **/*.md). qmd collection add scans the files, extracts a title (first heading or file name), hashes the content to derive a six-character docid such as #abc123, and writes the text into SQLite with an FTS5 index. At that point keyword search already works. Embedding is a separate step, qmd embed. It splits each document into chunks of about 900 tokens with 15% overlap and embeds each chunk with the configured model. The README documents a scoring scheme that prefers headings and code-fence boundaries as cut points within a 200-token window, and avoids splitting code blocks. For TypeScript, JavaScript, Python, Go and Rust files, --chunk-strategy auto adds tree-sitter break points at classes, functions, type aliases and imports.
Configuration lives in ~/.config/qmd/index.yml: collections, path-level "context" strings, optional pre-update shell hooks and model overrides. A context string attached to qmd://notes or qmd://docs/api is returned with every matching result to help an LLM judge relevance; the README calls it the key feature. A project-local index can also be created with qmd init, which keeps config and database in .qmd/ inside a repository.
Three search modes
qmd search: BM25 only. No model is loaded.qmd vsearch: vector search. Despite the name, the README notes that free-textvsearch"still use[s] local query expansion", so it needs the generation model as well as the embedding model.qmd query: the full hybrid pipeline. A fine-tuned 1.7B model writes typed sub-queries (lexfor BM25,vecandhydefor vector search). The original query (weighted twice) and the expansions are fused with Reciprocal Rank Fusion (k = 60, plus small top-rank bonuses). The top 30 candidates go to a Qwen3 0.6B reranker, and the final order blends retrieval and reranker scores by position: 75/25 for ranks 1 to 3, 60/40 for ranks 4 to 10, and 40/60 beyond that.
Typed queries skip expansion. qmd query 'vec: …' or a multi-line lex:/vec: query runs only the sub-queries given, and --no-rerank skips the reranker. That combination is the documented route for "embedding-only retrieval: no local expansion or reranking model needed", and it is what made the hands-on test below possible.
Models
Three GGUF models are downloaded from Hugging Face on first use and cached in ~/.cache/qmd/models/: embeddinggemma-300M-Q8_0 for embeddings (about 300 MB), qwen3-reranker-0.6b-q8_0 for reranking (about 640 MB) and qmd-query-expansion-1.7B-q4_k_m for query expansion (about 1.1 GB). Each role can be overridden in the models: block of index.yml with another hf: URI or a local GGUF path. The embedding model can also be set through QMD_EMBED_MODEL, which the README suggests using with Qwen3-Embedding for Chinese, Japanese and Korean text. Version 2.9.0 added an OpenAI-compatible remote embeddings endpoint and a hosted reranker (TypeSafe System One); local GGUF stays the default. The inference layer is node-llama-cpp 3.22.3, a Node binding for llama.cpp, which picks Metal, CUDA or Vulkan when available and falls back to CPU otherwise.
Key features in 2.9.0
- Hybrid retrieval in one SQLite file: FTS5 BM25,
sqlite-vecvectors and an LLM cache in a singleindex.sqlite, with documented tables for collections, contexts, documents, chunks and vectors. - Agent-friendly output:
--formatacceptscli,json,csv,md,xmlorfiles;--all --min-scoreexports every match above a threshold;qmd getretrieves a document by path or docid with an optional:from:countline range, andmulti-getworks by glob or list. - MCP server with five tools (
query,get,multi_get,status,metadata) over stdio or Streamable HTTP, plus a Claude Code plugin. The HTTP mode keeps models loaded between requests and can run as a daemon. - DNS-rebinding protection on the HTTP server: requests with a non-loopback
Originor an unexpectedHostheader get a 403, adjustable throughQMD_ALLOWED_ORIGINSandQMD_ALLOWED_HOSTS. - Metadata filtering and discovery (new in 2.9.0): a
qmd.metadatafrontmatter block with typed values, a recursive JSON filter grammar (and/or/not, comparisons,in/nin/all, text operators,type,exists) on every search surface, andqmd collection metadatato list the keys and values that are actually indexed. - Collection processors (new in 2.9.0): SDK callbacks or trusted commands that turn one source file into several Markdown index documents.
- Trust gating for checked-in config: a
.qmd/index.ymlthat arrives with agit clonecannot run update hooks, index paths outside the project or load non-default models untilqmd trustapproves them. Without a terminal (agents, CI, MCP) those fields are skipped. - Maintenance commands:
qmd doctorchecks the runtime,sqlite-vec, models, embedding fingerprints and GPU;qmd cleanupdrops orphans and repacks the vector table;qmd benchscores a fixture of queries across thebm25,vector,hybridandfullbackends.
Getting started
Install globally with npm or Bun, per the README:
npm install -g @tobilu/qmd
bun install -g @tobilu/qmd
Create a collection, describe it, and embed it:
qmd collection add ~/notes --name notes
qmd context add qmd://notes "Personal notes and ideas"
qmd embed
The three search modes, from fastest to most thorough:
qmd search "project timeline"
qmd vsearch "how to deploy"
qmd query "quarterly planning process"
When only an embedding model is available (remote or local), the README's embedding-only form skips expansion and reranking:
qmd query 'vec: your semantic query' --no-rerank
MCP clients that launch servers as subprocesses use this configuration (shown in the README for Claude Desktop and Claude Code):
{
"mcpServers": {
"qmd": {
"command": "qmd",
"args": ["mcp"]
}
}
}
Claude Code can also install it as a plugin:
claude plugin marketplace add tobi/qmd
claude plugin install qmd@qmd
For one long-lived server that keeps models in memory, use the HTTP transport (port 8181 by default; the daemon writes its PID under ~/.cache/qmd/):
qmd mcp --http
qmd mcp --http --daemon
qmd mcp stop
On macOS the README asks for Homebrew SQLite for extension support.
Hands-on: QMD 2.9.0 on a CPU-only VM
Everything in this section was run on 22 November 2026 on a cloud VM: Linux 6.18 x86_64, 2 vCPUs, about 7.8 GiB RAM, no GPU, no swap, Node.js 22.23.5 (the launcher prefers Node when both runtimes are present). @tobilu/[email protected] was installed from the npm registry into a project directory instead of globally. XDG_CACHE_HOME and XDG_CONFIG_HOME, both documented, pointed the index, config and model cache at a scratch directory. Hugging Face was unreachable from this VM, so the reranker and query-expansion models could not be downloaded. The default embedding model, embeddinggemma-300M-Q8_0.gguf (333.6 MB, SHA-256 b5ce9d77…0d63), had been fetched from its Hugging Face URL beforehand and was used as a local file. Wall time and peak RSS come from a Python wrapper around resource.getrusage. These are single-machine numbers, useful for relative comparison only.
The test corpus was 265 real Markdown files (3.3 MB, about 334,000 words): the 111 chapters of The Rust Programming Language listed in the book's SUMMARY.md, and the 154 FastAPI documentation pages listed in its mkdocs.yml, both fetched from raw.githubusercontent.com. Each set became a collection (rustbook, fastapi) with a one-line context.
Install, indexing and embedding
| Step | Result |
|---|---|
npm install @tobilu/[email protected] | 20 s, 159 packages, node_modules 936 MB. 735 MB of that is prebuilt @node-llama-cpp binaries, including 597 MB of CUDA builds and 73 MB of Vulkan on a machine with no GPU. better-sqlite3 and sqlite-vec loaded from prebuilds with no compiler |
qmd --version | "qmd 2.9.0 (aa247d0)", 0.16 s, 60 MiB peak RSS |
collection add (BM25 index) | 111 files in 0.38 s, 154 files in 0.44 s, about 85 MiB RSS; index 6.9 MB |
qmd embed, CPU only | 1,590 chunk vectors for 265 documents. The completed run reported "Embedded 1195 chunks from 173 documents in 16m 21s" (981 s wall, 1,671 s CPU, 962 MiB peak RSS). The other 395 chunks had been written by an earlier run that was stopped after 10 minutes, and they were kept and skipped on restart. That is roughly 1.2 chunks per second on 2 cores, or about 22 minutes for the whole corpus |
| Index after embedding | 13.8 MB (3.3 MB of source text) |
qmd update, nothing changed | 0.29 s, "0 new, 0 updated, 154 unchanged" |
One file edited, then update + embed | 0.25 s ("Removed 6 stale vector row(s)"), then 6 chunks in 7.9 s including model load; a BM25 search for the new sentence found it at once |
qmd doctor | All storage checks passed, including "3 sampled chunks reproduce stored vectors"; flagged "model cache: missing 2/3" and "running on CPU (2 math cores)" |
One message was misleading. After every qmd update, including the no-change run, the CLI printed "Run 'qmd embed' to update embeddings (265 unique hashes need vectors)", while qmd status and qmd doctor both reported every document embedded.
Query latency
| Mode | Cold CLI process | Warm HTTP server (POST /query) |
|---|---|---|
BM25 (qmd search / lex) | 0.13 to 0.23 s, 66 to 98 MiB | 4 to 9 ms |
Vector (query 'vec: …' --no-rerank) | 3.4 to 4.0 s, about 959 MiB | 47 to 56 ms (the first request took 3.1 s) |
Hybrid (lex: + vec:, --no-rerank) | 3.2 to 3.9 s | not measured separately |
rg -l -i -F on the same files | 7 to 24 ms | n/a |
Almost all of the 3.5 s for a CLI vector query is loading the embedding model, so the README's advice to run qmd mcp --http for repeated use holds up. The HTTP server held about 926 MiB resident with the model loaded. MCP initialize, tools/list (which returned the five documented tools) and a get call all worked over /mcp. A request carrying Origin: https://evil.example got a 403, as documented.
Keyword versus vector: result quality
Twelve queries were written against known target pages. Four use exact terms that appear in the text. Eight are paraphrases that avoid the page's own vocabulary. Each ran as BM25 (qmd search), vector-only (vec:) and hybrid (lex: plus vec: with the same text, RRF fusion, no reranker), returning 10 results. The table shows the rank of the target page ("–" means not in the top 10). The last column counts files that contain the query as a literal, case-insensitive phrase according to ripgrep.
| Query | Target | BM25 | Vector | Hybrid | rg files |
|---|---|---|---|---|---|
| "lifetime elision" | ch10-03 lifetimes | 1 | 1 | 1 | 1 |
| "Arc Mutex" | ch16-03 shared state | 1 | – | 2 | 0 |
| "BackgroundTasks" | tutorial/background-tasks | 2 | 2 | 2 | 4 |
| "root_path" | advanced/behind-a-proxy | 1 | 1 | 1 | 4 |
| "send an email after the response has already been returned" | tutorial/background-tasks | – | 1 | 2 | 0 |
| "let several threads change the same counter without data races" | ch16-03 shared state | – (0 hits) | 1 | 1 | 0 |
| "app is served under a sub-path by nginx and the docs page breaks" | advanced/behind-a-proxy | – (0 hits) | 2 | 2 | 0 |
| "a value that has more than one owner" | ch15-04 Rc<T> | 4 | 9 | 5 | 0 |
| "how long a reference stays valid" | ch10-03 or ch04-02 | – | 1 | 2 | 0 |
| "stop other websites from calling my API from the browser" | tutorial/cors | – | 9 | 10 | 0 |
| "store user passwords securely" | security/oauth2-jwt | 2 | 2 | 2 | 0 |
| "program stops immediately because of a bug it cannot recover from" | ch09-01 panic! | – | 6 | 6 | 0 |
On the four exact-term queries, BM25 put the target first three times. The fourth miss was a fair one: reference/background.md, the API page for the same class, ranked above the tutorial. On the eight paraphrased queries, BM25 found the target in the top 10 only twice (mean reciprocal rank 0.09). Vector search found all eight in the top 10, three of them at rank 1 (MRR 0.55). The hybrid run also found all eight, but only one at rank 1 (MRR 0.43). Long sentences sent to the lex leg matched noisy pages such as FastAPI's release notes, and RRF promoted them. With no expansion model to rewrite the query into keywords, sending the same sentence to both backends made results worse than vector alone. Literal grep found nothing for any paraphrase, as expected.
The vector results were good but not sharp. CORS came in at 9th and Rc<T> at 9th, behind loosely related ownership chapters. The full pipeline might close that gap: the README reports, for its own example fixture, "bm25 ~0.50, vector ~0.70, and hybrid/full ~1.00", but the reranker could not be tested here. There is also a possible source-side explanation. The CHANGELOG's [Unreleased] section, written after 2.9.0, says local embedding inputs are now kept "in a single evaluation batch so pooled embeddings cover the full input (#1121)" and tells users to run qmd embed --force. If that bug affects 2.9.0 as published, the vector numbers above may understate what the next release delivers. That could not be verified here.
A scoring detail worth knowing: with --no-rerank, the score field is a position score (1, 0.5, 0.33, 0.25, 0.2 for ranks 1 to 5) rather than the cosine-derived 0 to 1 value in the README's score table. --explain showed the underlying vector score (0.53 for the top hit of one query). A --min-score 0.3 cut-off therefore keeps the top three results no matter how weak they are.
Behaviour with models missing
| Action | What happened |
|---|---|
qmd embed with the default hf: model and Hugging Face unreachable | Printed "Model: embeddinggemma-300M-Q8_0.gguf" and then hung with no error until killed after 5 minutes. curl to the same URL failed in 0.25 s with a proxy 403, so the hang was not a slow network. An empty models/ directory and a lock file were left behind |
QMD_EMBED_MODEL=/path/model.gguf qmd embed | Same hang. The earlier collection add and context add commands had already written a models: block with the default hf: URIs into index.yml, and the 2.9.0 code gives models.embed priority over the environment variable, so the variable was ignored |
models.embed in index.yml set to the local file path | Worked; doctor labelled it "non-default model configuration … might be ok" |
Free-text qmd vsearch / qmd query | Hung at "Expanding query..." (stopped after 60 s and 30 s) |
POST /query with the default rerank: true | No response within 40 s. The server stayed healthy, and the next rerank: false request answered in 47 ms |
qmd search, get, status, doctor | Unaffected; the README is right that "'qmd search' works without any model downloads" |
The practical lesson for offline machines or locked-down networks: put local GGUF paths for every role in index.yml (or pre-fill the model cache), and have agents send typed lex/vec sub-queries with reranking off unless the reranker is present. A missing model shows up as a silent stall, not an error. Any wrapper should set a timeout.
Scaling the keyword index
The corpus was copied 20 times into a third collection: 5,300 files, 66 MB. collection add took 10.35 s at 443 MiB peak. Because QMD keys vectors by content hash, the identical copies needed no new embeddings: "Copied 1590 vector(s) into collections that gained already-embedded documents". The index file grew to 100.6 MB (qmd status reports it as 95.9 MB). SQLite's dbstat attributed 80.8 MB to the FTS5 tables and 12.6 MB to the vector chunks. BM25 queries over all 5,565 documents still took 0.15 to 0.29 s per CLI call. For comparison, rg -l -i mutex over the same 66 MB took 36 ms and GNU grep -rli 57 ms, both finding 80 files. QMD does not collapse duplicates in results: the top 10 for "Mutex" was the same docid in ten collections and paths. The embedding cost was not tested at this scale, because duplicated content would understate it.
Use cases
- Retrieval for coding agents. Point an MCP client at
qmd mcpand the agent can search engineering notes, ADRs or a docs repository, then pull exact line ranges withget. The tool descriptions the server sends encourage typedlex/vec/hydesub-queries and anintentfield. - A personal knowledge base without a server. A notes folder, meeting transcripts and saved articles become one searchable SQLite file, with contexts that tell the model what each folder is.
- Docs search inside a repository.
qmd initkeeps a project-local index in.qmd/, and the trust gate stops a cloned repository's config from running hooks or loading models without approval. - Feeding RAG prompts.
--format md --fullor--all --files --min-scoreproduce context bundles or file lists for another tool. - Filtered search over structured notes. Frontmatter metadata such as status, topics or priority can restrict any search, which suits curated documentation sets.
How QMD compares
Facts come from each project's README and licence file as read on 22 November 2026. Only ripgrep and GNU grep were run here.
| Tool | What it is, per its README | Semantic search | Runs as | Licence |
|---|---|---|---|---|
| QMD 2.9.0 | On-device search over Markdown collections with BM25, vectors and LLM reranking | Built in, local GGUF models (optional remote embeddings) | CLI, SDK, MCP stdio/HTTP; one SQLite file | MIT |
| ripgrep | "A line-oriented search tool that recursively searches the current directory for a regex pattern", respecting gitignore rules | None | Single binary, no index | MIT or Unlicense |
| Nogon (formerly Hister) | "A private search engine for the pages you visit and the files you keep"; browser extension, crawler, history import | Optional, "through an embeddings endpoint you configure" | Go server with web UI, TUI, CLI and MCP; multi-user | AGPL-3.0 |
| Typesense | "A fast, typo-tolerant search engine"; "an Open Source Algolia Alternative" | Vector, semantic and hybrid search, with built-in models "like S-BERT, E-5" or external APIs | Server (Docker image), HTTP API with API key | GPL-3.0 |
| Qdrant | "A vector similarity search engine and vector database" written in Rust | Dense, sparse and multivector search plus hybrid; embeddings supplied by the application | Server (Docker), REST/gRPC; managed cloud offered | Apache-2.0 |
ripgrep is the baseline that every Markdown folder already has. It answered roughly ten times faster than a cold qmd search call in this test (7 to 24 ms against 130 to 230 ms), needs no index and never goes stale. It cannot rank results or find paraphrases, but for exact identifiers like root_path it is hard to beat. QMD earns its place when the question is in natural language or when an agent needs ranked, chunked, line-addressable results.
Nogon, renamed from Hister according to its README, is the closest in spirit: personal, local, MCP-capable. Its centre of gravity is web pages captured by a browser extension, with a server and a web UI. QMD is file-first and serverless by default, and does its own local embedding and reranking instead of calling an embeddings endpoint. Another page in the same niche, XERJ, is described as a local-first search engine and MCP server for agents. It was not evaluated for this page.
Typesense and Qdrant are servers built for applications. They bring typo tolerance, faceting, payload filtering, replication and scale-out that QMD does not try to offer. They also mean running a service and an ingestion pipeline. QMD's vector store is sqlite-vec, which its CHANGELOG describes in terms of "brute-force" scans of chunk tables, not an approximate-nearest-neighbour index of the kind libraries such as Faiss provide. That is fine at notes-folder scale. For millions of vectors or a multi-user product, a dedicated engine is the more natural choice.
Agent memory tools such as Claude-Mem solve a neighbouring problem: storing what an agent did, rather than searching documents a person already has.
Things to know before adopting
- The best mode needs all three models. Plain
qmd queryand even free-textqmd vsearchneed the 1.1 GB expansion model. The default reranker adds 640 MB. Without them, useqmd searchor typedvec:/lex:queries with--no-rerank, as tested above. - Missing models stall rather than fail. In this VM, every command that needed an unavailable model hung silently instead of reporting a download error. Wrap agent and CI calls in timeouts, and run
qmd doctororqmd pullwhen setting up a machine. - Config beats environment. Once
index.ymlhas amodels.embedentry, whichcollection addwrote in this run,QMD_EMBED_MODELis ignored. Edit the YAML instead. The README's statement that the variable "overrides the default" is accurate only while no explicit config exists. - CPU embedding is slow. About 1.2 chunks per second on 2 vCPUs meant roughly 22 minutes for 3.3 MB of Markdown. A large vault on a laptop without Metal, CUDA or Vulkan needs patience, or a remote embeddings endpoint. Updates afterwards are incremental and cheap.
- Footprint. Model-backed commands peaked near 960 MiB RSS; the install was 936 MB, mostly unused GPU builds on this box.
- Hybrid without expansion can hurt. Sending one long sentence to both
lexandvecranked worse thanvecalone in this test. Give thelexleg real keywords. - Scores are not always comparable. With reranking off, displayed scores are rank positions, so
--min-scoredoes not filter by relevance. - Unauthenticated HTTP. The README says plainly that the endpoints are unauthenticated. Binding to
0.0.0.0skips the Host check, so put authentication in front of any off-host deployment. - Remote options send data out. Remote embeddings send document text to the configured endpoint. TypeSafe reranking sends query and chunk text to a fixed hosted API, and the README warns its limits "are resource bounds, not a spending cap". Both are opt-in.
- Language coverage. The default embedding model is "English-optimized"; for CJK text the README suggests Qwen3-Embedding.
- Fast-moving project. The CHANGELOG runs from 0.1.0 (dated 7 December 2025, a day before the GitHub repository's creation date) to 2.9.0 within a year, with index format migrations along the way. The 2.9.0 vector partitioning, for example, means a downgrade needs
qmd embed -f. Pin versions in automation. - Markdown first. The default mask is
**/*.md. Other formats need masks or the new processors, and files over 10 MB are skipped.
Project activity
Checked 22 November 2026: latest release v2.9.0, dated 17 November 2026 in the CHANGELOG and published to npm as @tobilu/qmd 2.9.0 at 03:51 UTC on 18 November. Licence: MIT (copyright 2024–2026 Tobi Lutke). The GitHub release v2.9.0 was published at 03:48 UTC on 18 November 2026, per the GitHub releases API. The tobi/qmd repository had about 31,000 stars (31,332 at the time of writing) and 1,977 forks. It is written in TypeScript, was created on 8 December 2025, develops on main and was last pushed on 20 November 2026. The [Unreleased] section already holds an embedding fix that asks for a re-embed.
Enjoying this project?
Discover more amazing open-source projects on TechLogHub. We curate the best developer tools and projects.
Repository:https://github.com/tobi/qmd
GitHub - tobi/qmd: QMD: On-Device Hybrid Search for Markdown and Agents
QMD is an MIT-licensed local search engine for Markdown with BM25, sqlite-vec vectors, LLM reranking and an MCP server. Version 2.9.0 tested on CPU: indexing, e...
github - tobi/qmd
Related Projects
Maintain QMD: On-Device Hybrid Search for Markdown and Agents? Add the badge to your README
Show that the project is featured on TechLogHub and send readers to this write-up.
[](https://techloghub.com/open-source/qmd-local-search?utm_source=techloghub_badge&utm_medium=referral&utm_campaign=readme)