← Journey

V1 Runs

The first version that actually works end to end: local inference, retrieval, verification, citation.

The first thing that worked was the knowledge-from-data path: a question about a document the model had never seen, answered correctly, with a citation. That’s the whole thesis in one query. Everything before it was theory; this was the thing.

Building in Rust

The port to Rust took longer than expected. The original prototype was Go, and Go has goroutines and a GC that make certain things easy. Rust makes you explicit about ownership and lifetimes in ways that are painful when you’re moving fast. But the result is a single static binary you can copy to a server and run with no runtime dependencies. That matters for something you want to deploy cheaply.

Knowledge from Data

The first test was a fake document, sample-docs/zephyr.md, with facts the model could never have seen during training. Completely invented stuff: a protocol with specific memory constraints, a mascot named Pippa, an inauguration year. Then indexed it and asked questions.

$ metis ask "What does the Zephyrian Protocol mandate about memory, and what's its mascot?"
The Zephyrian Protocol caps resident memory at 1.84 GB [1]. Its mascot is a blue heron named Pippa.
sources: [1] zephyr.md (0.32)

The model didn’t know any of this. It retrieved it. Citation included. If you give it a different document, it gives different answers about the same questions.

Exact Arithmetic

The tool use case works too. Exact arithmetic through a subprocess rather than through model weights that approximate:

you> What is 84937 x 2261, divided by 7?
  [tool] calc(84937*2261) = 192042557
metis> 192042557 / 7 = 27434651.

The bare model gets that multiplication wrong. It gives you 191,737,397, which is plausible-looking and completely incorrect. The calculator doesn’t approximate.

The GVS Loop

The GVS loop (Generate, Verify, Search) is running. It works roughly like this: generate a candidate answer grounded in the retrieved evidence, then run the same model again in judge mode to check whether the candidate is actually supported by that evidence.

The judge gets a different prompt. Something like “does this CLAIM follow from this EVIDENCE?” rather than anything that implies the model should evaluate its own reasoning. That framing matters. Research by Huang and others showed that asking a model “are you sure?” nets you nothing. Asking it “is this claim supported by this specific text?” gives you something closer to 90-100% accuracy even at 1.7B.

If the first candidate fails verification, the conductor searches: generates a few more candidates at higher temperature, verifies each, keeps the first one the evidence supports. If none of them pass, it abstains.

Abstention was the part that took most convincing to commit to. It feels like a failure mode. The system should answer. But “I don’t know” is the right answer to questions without a retrievable answer, and a model that says “I don’t know” is more trustworthy than one that invents a plausible-sounding response.

Memory Footprint and Setup

Memory footprint: Qwen3 1.7B at Q4 takes about 1.1 GB. The all-MiniLM embedder is 80 MB. The full system runs comfortably under 1.5 GB RAM, with room for the KV cache and the index navigation structures.

The web search path is live too. Point it at a self-hosted SearXNG instance and it treats the web as a library that’s too big to store. The same verify-and-cite loop runs over web results. The binary stays TLS-free by design; SearXNG handles the HTTPS to upstream search engines.

Total setup time on a clean machine, assuming ollama is installed:

ollama pull qwen3:1.7b
ollama pull all-minilm
cargo build --release
./target/release/metis index ./docs
./target/release/metis chat

That’s it. Fully local, no API key, no cloud dependency.

Key takeaways:

Next: the deployment bug that turned a 3-second answer into a 7-minute one.

← All entries