From token prediction to bounded, verified work
“Fix the invoice bug.” The agent has to infer both the rule and what counts as done.
“Apply the VAT rate from the order. Preserve the tests, run the invoice suite, and show the diff.”
Illustrative comparison, not two measured runs. A clearer request makes the result easier to assess; it does not guarantee success.
Model → harness → tool result.
Acceptance cases, limits and evidence.
Useful context, measured cost and reusable checks.
One task throughout: correct the invoice total, then verify its CSV export.
The animation is a simplified decoding example. It does not describe everything learned during training.
Conceptual diagram: multiple heads and layers contribute. Context is an input you curate, not a literal program.
Reasoning effort is a dial: more steps can help, and they cost time and tokens. Compare the supported settings on the same acceptance cases.
Proposes text or a structured tool call.
Builds context, applies permissions and coordinates tool execution.
The terminal, editor or web interface where you inspect and steer the work.
For the invoice task: the model requests a test run; the harness arranges it; the client displays the result.
A tool request is a proposal to execute. Permissions and the tool environment determine what actually happens.
Illustrative API-backed loop. History may be sent again, referenced through a stateful API, or compacted; it is not permanent model memory.
For the invoice fix: at most three repair attempts, then report evidence and unresolved failures.
VAT comes from the order; zero VAT and rounding cases pass.
Existing tests remain meaningful. Inspect changed assertions and test discovery.
Review the diff and exported values against the task, including gaps in coverage.
Exit code 0 is useful evidence. It is not proof that the full task is correct.
| Failure | Concrete control |
|---|---|
| Repeats the same failed repair | Limit attempts; require a new hypothesis before retrying. |
| Weakens tests to make them pass | Review the test diff; run independent acceptance checks. |
| Changes unrelated behavior | Scope allowed paths; inspect the complete diff. |
Prompts help communicate these rules. Enforce the important boundaries in tools, permissions and review.
Pair this request with an enforced runtime budget and a review of the resulting artifact.
Illustrative input volumes, not a bill. Repeated history counts as input; cache hits may have a different rate.
| Category | What to record |
|---|---|
| Uncached input | New input processed at its applicable input rate. |
| Cache writes and reads | Separate token counts and the configured cache rates. |
| Generated output | Output usage, including billable reasoning where applicable. |
Bill = sum of each category’s tokens × its rate, plus any separate tool or service fees.
| Illustrative usage and rates | Cost |
|---|---|
| 19,000 cached input × $0.20 / million | $0.0038 |
| 3,700 new input × $4 / million | $0.0148 |
| 1,600 output × $20 / million | $0.0320 |
| 22,700 input + 1,600 output in total | $0.0506 |
Assumed rates, not a measured bill. Without the cache hit: $0.0908 input + $0.0320 output = $0.1228.
Animation shows prefix reuse. Its price label is illustrative; rates vary. Cached content still occupies context.
The invoice rule, failing case, relevant code, decisions and latest test result.
Duplicate file dumps, stale logs and abandoned approaches, once their useful findings are captured.
Watch for missed requirements and repeated work. There is no universal “dumb zone” or safe fill percentage.
Example return: VAT calculation locations, the failing path, supporting lines and unresolved questions.
Use representative invoice cases and the same permissions and tools.
Record quality, elapsed time, all token categories, retries and review effort.
Repeat representative cases before changing the default configuration.
A higher-priced model can finish in fewer attempts. Whether it costs less is a measurement, not a rule.
There is no universal “effort first” ladder. Re-evaluate defaults when the task or model changes.
Implement the bounded invoice change. Return the diff, test output and known gaps.
Check the tax rule, changed tests and edge cases. Return findings with evidence and severity.
Choose configurations from measured performance in each role. A second model’s opinion is not a verifier.
Pass evidence between stages. Parallelize independent investigation; coordinate edits and integration.
| Task shape | Starting point |
|---|---|
| Explain the VAT rule | One question with the relevant source. |
| Fix the invoice calculation | One agent with tools and acceptance checks. |
| Repeat a repair cycle | A bounded loop with a meaningful stop condition. |
| Investigate independent areas | Helpers with explicit inputs, returns and an integration owner. |
Add orchestration when it resolves a real bottleneck. Measure its coordination cost too.
Use the instruction-file convention your harness actually loads. Keep standing rules short and specific.
Procedure: inspect the diff → run the cases → report evidence and gaps.
Input: invoice defect.
Output: rule, scope and acceptance cases.
Input: accepted plan.
Output: patch, tests and unresolved issues.
Input: patch and evidence.
Output: reviewed result and a reusable lesson.
Teach the workflow through its outputs. Add a new skill only when a repeated need justifies it.
The repeatable invoice-check procedure and supporting files.
A versioned way to distribute the procedure and supported integrations.
Test discovery and behavior in a clean session before sharing. Installation does not replace review or permission boundaries.
Optional reference: arithmetic, workflow sketch and skill publication
Primary sources checked 3 October 2026. Worked examples are illustrative. All decks.