Our Own Weights

Eleven entries into this journey, there has been one uncomfortable constant: the brain was always someone else’s. We built the Library, the Hands, the Conductor, the verifier — but the Cortex was a Qwen we downloaded. Every claim about “small models punching up” rested on weights we didn’t train and couldn’t change.

Today that ended. A 14.8-million-parameter model, trained from scratch on this MacBook, on our own code, served from this MacBook, answered a prompt inside OpenCode — the coding agent we actually use. The whole loop ran on one laptop. No GPU cluster, no API key, no cloud. Total cost: zero dollars and about an hour of the machine’s time.

The output was mostly gibberish. We’ll get to that — and to why we shipped it anyway.

Night 0, Measured

The rule of this house is that no number ships unless it was measured on real hardware. So before any architecture debate, we ran the calibration the plan (design doc 13) demanded: train a real model with MLX on the M3 Pro and let the clock speak.

what	measured
model	14.8M params, 8 layers, byte-level
corpus	36.7 MB — 8,000 files of our own TypeScript, Rust, Python
throughput	13,677 tokens/s, sustained, fp32, unoptimized
hardware efficiency (MFU)	0.24 of the GPU’s assumed peak
validation loss after 16 minutes	3.78 bits/byte, still falling
extrapolation	~394M tokens per 8-hour night

Two things surprised us. First, the efficiency number landed almost exactly on the plan’s assumption (0.24 measured vs 0.25 assumed) — before any optimization, in full precision. The budgets we wrote down survive contact with reality. Second, nothing broke. Training a transformer from scratch on a laptop in 2026 is not heroic. It’s sixteen minutes.

The Pipeline Is the Product (Today)

The MVP is three small files and a config entry:

1. train.py — a from-scratch GPT trained with MLX on code harvested from our own projects.
2. serve.py — an OpenAI-compatible server (streaming included) that loads those weights.
3. One provider block in OpenCode’s config, pointing at http://127.0.0.1:8484/v1.

Then:

opencode run -m metis/metis-1-mvp "function add("

And OpenCode streams back whatever our model thinks code looks like. Which, right now, is this:

e'm    aena
  e  eot,yevr  ad;) -e t}t  n i   {up v /{n  p  u...

That was the model after thirty seconds of training — a throwaway checkpoint we used to test the plumbing. Letter soup, but look closer: braces, semicolons, colons after identifiers. The shape of code forming already.

Then we let the real run finish: 2,500 steps, 41 million tokens, 51 minutes of wall-clock on the laptop. Validation loss fell from 3.78 to 1.58 bits per byte. Same prompt, same pipeline, inside OpenCode:

// Solll soll soll and workers server other hore AI proda to and bedgs read...
    inferProd: ({
      creater_id: "Select factt f-selected",
      more: "workers",
      params: { rows: rowstrols, infer: rowstrols, int, interal: "Works",
    },

Comments. Object literals with keys and string values. Nested structure that opens and closes. Fifty-one minutes of one laptop’s time, and the model went from noise to the grammar of TypeScript. The words are still invented — meaning is what the nights are for — but the progression is measured, visible, and it cost nothing.

We could have waited weeks and shown something impressive. We shipped this on purpose: the MVP proves the pipe — train → serve → agent, sovereign end to end. Quality is now an optimization problem inside a working loop, not a leap of faith before one.

Why This Is a Different Architecture, Not a Smaller Copy

Everyone else’s recipe: train the biggest model you can afford, on everything, once, in a datacenter — then rent it back to the world through an API. metis-1 breaks that recipe in five places, and each break is a decision we’ve already tested somewhere in this journey.

1. Param-poor, token-rich — and aimed at one target. The lesson of today’s capable small models is not that small is magic; it’s that parameters shrink when you flood them with tokens. We take that recipe and narrow the distribution to a single job: code, shaped as edits, for an agent. Every capability we refuse to train — chat, trivia, a hundred languages — is budget returned to the one capability we keep.

2. Knowledge lives outside the weights. We proved in entry 3 that retrieval collapses a 25× parameter gap, and our RNT experiments showed why: facts in weights cost O(facts) parameters; facts in context cost O(1). metis-1 is trained retrieval-native from step 0 — the facts it needs are always in its context window, so it is never once rewarded for memorizing. The encyclopedia stays on disk. The weights buy reasoning only.

3. The verifier is a compiler, not an opinion. Entries 6 and 11 taught us that checking is cheaper than generating — but only if the checker can be trusted. For code, the checker is compile ∧ typecheck ∧ tests: a deterministic oracle that cannot be sweet-talked. A tiny model wrapped in an incorruptible verifier can afford to be wrong cheaply and often; a frontier model without one has to be right on the first try, at frontier prices.

4. Capacity comes from memory, not FLOPs. A laptop GPU is FLOP-poor but RAM-rich, so the architecture spends the cheap resource: a small frozen trunk, plus per-skill specialist modules, plus sparse key-value memory layers where a forward pass touches 32 slots out of a million instead of multiplying every parameter. New skills are new modules on a frozen trunk — which also means old skills physically cannot be forgotten.

5. Training is a metabolism, not an event. A datacenter model is born finished and ages from day one. metis-1 trains every night on the machine that used it all day: the verifier labels the day’s failures for free, and only the failures become the night’s curriculum. Mastery-based, forever. The measured budget says ~394M tokens fit in every night of sleep — the question is no longer “can we afford to train” but “what should tonight’s lesson be.”

Put together: a model that is never trained on what it can look up, never trusted on what a compiler can check, never frozen the way a shipped API is frozen — and never leaves the hardware we own. That is not a smaller copy of a frontier model. It is a different animal.

Honest Limits

The MVP model is not useful yet, and no amount of framing changes that. It has seen one pass over one laptop’s code at byte level. It cannot complete a function, follow an instruction, or call a tool. The gap between “the pipe works” and “the model is good” is exactly the work the plans (design docs 12 and 13) lay out: bf16 and a compiled training step to roughly double throughput, the GitHub mine — issues, merged PRs, CI verdicts, millions of verified edit-shaped examples — and then the trunk’s first real nights.

But the asymmetry has flipped, and that’s the point of this entry. Yesterday, training our own Cortex was a plan in a document. Today it’s a running loop with measured throughput, a served endpoint, and a seat inside our agent. Everything from here is iteration on a machine that trains while we sleep.

The soup is already learning to talk.