From token prediction to self-verifying loops
The engineering around the model made the difference. That's this whole talk.
Each upgrade fixes the last one's limit. One thread runs through all seven — the context window — and ● CONTEXT flags it. New to all this? Start with the beginner deck.
Where it started: a text box and a next-token predictor. Astonishing — and all it can do is guess.
A model is a probability distribution over the next token, sampled one token at a time.
Your keyboard sees ~one word; the old n-gram models saw two or three. An LLM with no useful context drifts exactly like this — the upgrade that sees everything is next.
● CONTEXT This is the thread: attention runs over the context window, so what you load into it is the program.
● CONTEXT Two of these let you watch attention actually run over the context — the thread, lit up.
Problem: hard answers don’t fit in one forward pass. Fix: let the model write itself a scratchpad — and dial how much.
Same model, same question. The only difference: it thought on paper before it answered.
● CONTEXT Reasoning is the model extending its own context — the scratchpad is program it writes for itself.
| Effort | Reach for it when… |
|---|---|
| low | cheapest & fastest — bulk, delegated grunt work. |
| medium | the cost-saving step-down for routine agentic work. |
| high | the API default — start here for everything that isn't code. |
| xhigh | start here for coding & agentic runs — Anthropic's recommended default for both, and Claude Code's. |
| max | frontier problems only — past xhigh it mostly adds cost. |
Claude Code's "ultracode" = xhigh + standing permission to run multi-agent workflows (chapter 06) — a harness mode, not an API level.
Effort moves you along one model's curve; the model picks which curve — and a newer curve sits above and to the left. On Frontier‑Bench, Anthropic reports Opus 5 more than doubling Opus 4.8 at a lower cost per task; past xhigh, max buys ~nothing for roughly 25% more tokens. Its low/medium are unusually strong — re-sweep, don't inherit.
Problem: it can only talk — it can’t read your repo or run a test. Fix: tool calls, and a harness to execute them.
The model only emits tokens — the harness is the program that gives them hands. Rent the model, run the harness, pick the client; most models work in most harnesses.
● CONTEXT A tool result is just more context. "Reading a file" = loading its tokens into the window.
Every arrow to the server carries the whole conversation again — the API keeps nothing between calls. Remember the growing bars; the why is next, the price is chapter 05.
Which tokens fill that context window is what you're really controlling.
Problem: one step per turn — you are the relay. Fix: the harness re-runs until a verifier says done.
Levels 1–3 are the deck so far: knowledge (ch 01), thinking (ch 02), tools (ch 03) — you mostly get them for free. Level 4 — closing the loop against a goal — is this chapter, and where the failure modes live.
That's the entire trick: a while-loop around a token predictor. Which raises the question the next slide answers — who decides when it's done?
"exit code 0" is. A verifier is an objective check the model can't argue with.
Never recognises completion — each lap piles new code on broken code. Burns tokens, gets worse.
Reward hacking: test.skip, hardcoded expected values, deleting tests, appending || true. The verifier goes green; the work is fake.
Both are spec bugs, not model bugs. The loop optimises exactly what you wrote down; if "pass" is gameable, it gets gamed.
Name the verifier command, forbid the shortcuts by name, and demand real evidence — test output, an exit code, a screenshot diff.
Problem: every lap re-sends the whole history — cost grows quadratically and quality rots. First, the diagnosis.
"fix it" is 3 tokens; the request that carries it is ~13,000. The other 12,997 are the memory — every reply, tool call, and file read gets re-sent, verbatim, forever.
Turn 20 drags all 19 previous turns with it — the longer the session, the heavier and slower every turn.
| Input tokens | Output tokens | |
|---|---|---|
| What | everything re-sent — history, files, tool results | generated text, incl. thinking tokens |
| Volume | huge, grows every turn | usually small |
| Price / token | baseline (cheap) | ~5× input |
| Cacheable? | yes — coming up | no |
| Model · July 2026 | Input / MTok | Output / MTok |
|---|---|---|
| Haiku 4.5 | $1 | $5 |
| Sonnet 5 | $3 · intro $2 | $15 · intro $10 |
| Opus 5 | $5 | $25 |
| Fable 5 | $10 | $50 |
Agentic sessions are input-dominated: the model reads far more than it writes.
Three patterns: the ~5× output premium holds across the lineup; each tier jump roughly doubles both prices; and Opus 5 landed at Opus 4.8's price — a generation of capability for free. That last one is the tell: this is a price per token, not a price per task. Next slide.
| Measured | Result |
|---|---|
| Opus 4.5 vs Sonnet 4.5 SWE-bench, medium effort | matched Sonnet's best score using 76% fewer output tokens |
| Opus 5 vs Opus 4.8 internal trading eval | best score on ~1⁄7 the reasoning tokens, under half the latency |
| Sonnet 5 vs Opus 4.8 BrowseComp | same accuracy at ~⅓ the per-task token cost |
| Opus 5 xhigh vs max GDPval-AA v2 | higher score on 25% fewer output tokens |
A weaker model flails: it re-reads files, makes more tool calls, guesses wrong and retries — and every lap re-sends the whole history. You save 5× per token and spend 3× the tokens.
Why it happens: capability shows up as fewer laps, not just better answers. Fewer laps means less thinking, fewer tool calls, and — because every lap re-sends everything before it, so the session total grows quadratically — a much smaller bill.
The caveat that keeps this honest: it only holds where the task is hard enough for the shortcut to exist. Classification doesn't get cheaper on Opus. The rule stays cheapest model that clears the bar — measured on your work, not assumed.
The bill (Opus 5, $5 / $25 per MTok): input would be $0.11, but ~19k of it is cache-read at 0.1× → ~$0.03. Output: 1,600 × $25/M → ~$0.04.
The model reads 14× more than it writes — yet the tiny output side costs more: output is 5× the price and never cached. That's why thinking tokens are the expensive dial — billed once, here, then stripped from context: they never come back on the input side.
● CONTEXT Caching = paying less to re-send the same context.
● CONTEXT Cache is per-model — switch models mid-session and you pay to rebuild the cached context.
● CONTEXT Where agents go to get confused is the last stretch of the window — hygiene is just keeping it lean.
Problem: one window, one brain, carrying everything. Fix: route the cheapest adequate brain; delegate to fresh windows.
● CONTEXT Delegation is context engineering — spend the main window on decisions, not exhaust.
| Maker | Small | Medium | Large | Above (frontier) |
|---|---|---|---|---|
| Anthropic | Haiku | Sonnet | Opus | Fable |
| OpenAI · GPT‑5.6 | Luna | Terra | Sol | — |
| Google · Gemini 3.5/3.6 | Flash‑Lite | Flash | Pro | Deep Think |
S / M / L repeats at every lab; Anthropic adds Fable above the large model — priced above Opus (2× per token). Whether it still scores above it is the next slide. Snapshot, July 2026 — this reshuffles constantly.
| Benchmark · July 2026 | Sonnet 5 | Opus 4.8 | Opus 5 | Fable 5 |
|---|---|---|---|---|
| SWE-bench Pro · real repo fixes | 63.2 | 69.2 | 79.2 | 80.0 |
| OSWorld 2.0 · computer use | — | 55.7 | 70.6 | — |
| Frontier-Bench v0.1 · hard agentic | — | ~18 | ~43 | 33.7 |
| ARC-AGI-3 · novel problems | — | — | 30.2 | — |
Two shapes here. On ordinary coding the tiers are 10–15 points apart — real, not dramatic. On benchmarks built from problems the last generation couldn't touch, it's a 2×+ jump (Frontier-Bench) or 3× the whole field (ARC-AGI-3, where the next-best model scores 7.8). SWE-bench Verified is saturating — Opus 5 sits at 96%. Caveat: Anthropic publishes most of these as claims rather than tables — exact figures come from third-party trackers and move a point or two.
Why it matters here: jumps this size make an old routing table wrong, not just suboptimal. Dashes are honest — not every model is run on every benchmark — and none of these is your workload. Re-measure.
● CONTEXT Thinking tokens are output that lands right back in the context window.
Fable is for judgement work — two slides on. And re-run the sweep after a model launch: Opus 5's low/medium are strong enough that inherited settings are usually wrong.
You hired a staff engineer. Use it for the hard call and the review — and only once Opus 5 at max effort has actually failed. Then hand the typing to something cheaper.
The API also has a built-in advisor tool (beta), consulted mid-generation. One rule: the advisor must be at least as capable as the executor — so sonnet·executor → opus/fable·advisor is fine, opus·executor → sonnet·advisor is a 400. Claude Code has no --advisor flag, so in the harness it's the standing rule above.
Fable plans, delegates, and reviews what comes back; haiku / sonnet / opus do the typing.
The UI reports the wrapper's Claude model — the label is the only cue the real worker was GPT-5.6 Sol. Parallel cross-vendor workers need worktree isolation so edits don't collide.
| Tier · the lineup from before | Suggested usage |
|---|---|
| Small — Haiku · Luna · Flash-Lite | classification, log triage, autocomplete, high-volume pipelines — anything you'd regret paying Opus prices for. |
| Medium — Sonnet · Terra · Flash | the workhorse: implementation from a clear spec, tests, refactors, bulk subagents, data wrangling. |
| Large — Opus · Sol · Pro | the agentic default: long-horizon coding, debugging, multi-step tool work, user-facing writing & design. |
| Frontier — Fable · Deep Think | plan reviews, the hard architectural call, verify stages, final polish — never bulk implementation, mechanical edits, or first drafts. |
Same split every time: the top tier decides and checks; cheaper models produce the volume. Escalate on failure, not vibes — a cheaper model at higher effort often beats a pricier one at low. But price the task, not the token: on genuinely hard work the bigger model gets there in fewer laps, and fewer laps is where the money actually is.
● CONTEXT Each stage gets a fresh window — one stage's mess never carries into the next.
You never hand-write the script — you state the shape (stages, fan-out, the verify bar) and the model writes & runs the orchestration. Knowing the shape tells you what to ask for.
Reach for the simplest rung that does the job. A workflow for a one-line question is as wrong as eyeballing a thousand-file migration by hand.
Problem: everything you fixed dies at /clear — every session starts blank. Fix: durable context — CLAUDE.md · skills · plugins.
● CONTEXT CLAUDE.md is context you pay for every turn — the whole reason skills exist is to keep it thin.
● CONTEXT Skills answer one question: how do I teach the agent my job without paying for the lesson on every turn?
● CONTEXT Skills exist to be lazy-loaded into context only when it makes sense.
No registration, no restart — the harness matches your sentence to the description and loads the body. A skill is just context, injected at the right moment.
Matt Pocock's approach: dozens of tiny skills — conventions, testing style, PR etiquette — each triggering on its own cue.
A whole method as a skill system: brainstorm → plan → work → review → compound. Every solved problem becomes a new doc or skill.
A skill system is an engineering culture the agent can read — two worked examples on the next slides.
github.com/mattpocock/skills — alignment before code, always; read the files, copy the patterns, adapt the conventions.
The unit of progress isn't the merged PR — it's the knowledge that compounds. (Kieran Klaassen / Every.)
● CONTEXT A plugin is your team's context, made distributable.
Skills = your habits. Plugins = your team's habits — versioned, installable in one command.
A token predictor → that thinks → with hands → in a loop → whose context you engineer: route it, delegate it, cache it, write it down.
● CONTEXT Every marker in this deck pointed back here.
Thanks! Questions?
David Budáč · figures a July 2026 snapshot — models & prices move fast, so re-check the primary sources.