Section 12Context, tokens & determinism

Everything the agent knows lives in its context window. Treat it as a scarce budget you curate — not a bucket you fill. This is context engineering, and it's the modern replacement for prompt engineering.

Section 11 left you running several agents at once — which means several context windows filling at once, and one of them (the orchestrator's) filling faster than all the rest. Before you can manage that well, it helps to be precise about what the window actually is: not the agent's memory, but its entire working world. The ticket, your AGENTS.md, every file it opened, every test run it watched fail, every dead end it backed out of — all of it sits in that one window, and on every turn the model re-reads all of it before producing the next word.

Two consequences follow, and this section is built around them. First: the window is smaller than the brochure says — not in how many tokens it accepts, but in how many it stays sharp across, and the gap between those two numbers is measured, not folklore. Second: because you can never make the model itself reliable by talking to it, reliability has to come from the machinery around it — hooks, tests, and gates that don't care how full the window is. Budget the context; make the scaffolding deterministic. Everything else here is detail on those two moves.

Avoid the dumb zone

As a conversation grows, it fills with "sediment" — old context, dead ends, wrong turns — and quality drops well before the advertised limit. Pocock puts this dumb zone around 100k tokens. The cure is not a bigger window; it's a fresh one. Treat each task as a clean session: spin up, do the ticket, commit, kill it. To carry only what matters across the gap, run /handoff — it compresses the state into a small file a fresh agent reads cleanly. Compaction (auto-summarising) buys time, but a fresh session beats a compacted one.

The advertised window is not the usable window

The dumb zone isn't superstition; it's one practitioner's name for something researchers have measured from several directions. Frontier models advertise context windows of a million tokens — Claude Opus 4.8 and Claude Fable 5 both do — yet quality starts degrading from roughly 100k tokens regardless of the number on the box. Chroma's "Context Rot" study (July 2025) ran 18 frontier models — GPT-4.1, Claude Opus 4 and Gemini 2.5 among them — through the same tasks at growing input lengths, and every single one got worse as the input grew, long before any advertised limit. An earlier finding explains part of why: the "lost in the middle" effect (Liu et al.) showed that models retrieve information placed at the start or end of a long context far more reliably than the same information placed in the middle — which is exactly where the important decisions of a long working session end up.

The practical translation: don't do percentage arithmetic on the brochure figure — plan for degradation from roughly 100k tokens regardless of the number on the box. On the ~200k-token windows most coding agents actually run with day to day that's half the window; on a million-token window it's a tenth, and the extra room is headroom for raw material, not extra sharpness. Either way, the threshold lands precisely where Pocock put the dumb zone. The research and the field practice agree; the brochure is the outlier.

Numbers to steer by

Research only helps once it becomes a habit, and the habit is: act at half-full, not at the limit. Waiting for automatic compaction to save you means the session has already spent a long stretch in its worst operating range — and a compacted session is a summary of the sediment, not a fresh start.

The orchestrator fills up first

Here's the asymmetry that makes Section 11's parallel setup work: subagents get a fresh context window by design. Every worker you spin up starts empty — its ticket and handoff notes, nothing else. The session that fills up is the orchestrator, because it receives every summary, every status report, every result. So point the hygiene where the pressure is. Fan the heavy reading out — "find every place the booking flow touches the calendar" is a subagent job: thirty files read in a disposable window, ten lines returned. Keep the orchestrator's window for decisions. And when the orchestrator itself nears the thresholds above, do for it what it does for its workers: write the handoff, start a fresh one. A fresh session with a written handoff beats automatic compaction every time — the handoff carries what you chose to keep; compaction carries what an algorithm guessed you'd need.

Spend fewer tokens, keep more quality

Session hygiene controls when you leave; this list controls how fast you get there. Every item is the same idea from a different angle: don't pay tokens for anything the agent doesn't need in front of it right now.

• Keep AGENTS.md/CLAUDE.md lean — don't pay to re-read facts the agent can look up.
• Use your ubiquitous language so the agent thinks in short, shared terms.
• Lean on prompt caching: put the stable stuff (system prompt, AGENTS.md, tool defs) first so it's charged once, then cheaply reused.
• Load MCP tools on demand rather than all upfront; fan out heavy reading to subagents that return a short summary and keep the main context clean.
• Match the model to the task, not the other way round. A strong model (e.g. Opus) for planning and hard reasoning, a mid-tier model (e.g. Sonnet) for most routine coding, a small fast model (e.g. Haiku, an order of magnitude cheaper) for high-volume simple work — Claude Code's hybrid planning mode switches automatically between two tiers for exactly this reason.

The minimal-CLAUDE.md doctrine

"Keep it lean" deserves teeth. A 2026 ETH Zurich study (arXiv 2602.11988) measured what repository context files like AGENTS.md actually do to coding agents: on average they raised inference cost by more than 20% while not generally improving task success — auto-generated files even slightly reduced it; only carefully hand-written ones helped. The conclusion isn't "delete the file" — it's that every line must earn its place in the instruction budget. Three rules keep it honest:

• Target under ~200 lines. If it's longer, it's holding things that belong in linked docs the agent can fetch when relevant, not re-read every session.
• Write the line, build the hook, delete the line. A prose rule is scaffolding — it's how every rule starts life, and it works most of the time. But once a hook enforces the rule mechanically, the sentence describing it is pure token cost, and the two will eventually drift apart. Deleting the prose line is the last step of building the hook, not an optional cleanup.
• Never enumerate directories. A folder listing is stale the day after it's written. Give the agent an architecture diagram (the same diagram-as-code file the docs site renders, Section 13) and let it navigate from structure, not from a list.

Time for an honest audit, because this guide is its own worst offender: Sections 5, 7, 8, 9 and 10 each told you to put a rule in AGENTS.md, and followed literally, the file busts the budget this section just set. So here is what the salon app's file actually looks like after the cut — abridged, but the real shape:

# AGENTS.md — salon-booking (final, abridged)

## Commands
- Test: `pnpm test` — suite must be green before any commit.
- Type-check: `pnpm tsc --noEmit` before claiming a task done.

## House rules
- Surgical changes: fix what the ticket names; file a ticket
  for anything else you notice (S5).
- Conventional Commits on every commit (S8).
- Modules talk only through the interfaces in
  `docs/architecture.md` (S10).
- A new field that could hold personal data: stop and
  flag it, never silently ship it (S9).
- Prefer a native platform feature over a library,
  where one now exists (S12).

## Read when relevant
- `docs/architecture.md` — module map (diagram-as-code)
- `docs/done-checklist.md` — judgment items, checked at review (S7)

Notice what's missing. The very first line you ever wrote — "Use pnpm, never npm," from Section 2 — is gone, because the hook from that same section now enforces it; the prose line was temporary scaffolding until the hook existed, then deleted. That's rule 2 in action, and it's the pattern for every enforceable rule that follows: the personal-data flag will go the same way the day it becomes a CI gate. Reference material didn't make the cut either — the judgment checklist from Section 7 lives in its own linked doc, loaded when a pull request is reviewed rather than re-read every session. And the per-feature edge-case questions (network drops, double-taps, race conditions) moved into the ticket template from Section 4, where they surface exactly when a feature is being specified. What survives in the file itself is only what applies to every session and can't yet be enforced mechanically — and the file stays a comfortable distance under 200 lines.

Determinism — reliable outcomes from a random model

The model is stochastic; you will never make it repeatable. So don't try. Instead wrap it in deterministic machinery that only accepts correct output: hooks that block wrong actions every time, tests that must pass, CI that must be green, sandboxes that bound what's possible. You don't get a deterministic model; you get deterministic outcomes. That distinction is the quiet engine under this whole guide.

Evals — testing the agent, not just the code

Tests (Section 5) check whether one feature is correct. Evals check whether your agent, prompt, or skill is still good — the regression suite for the workflow itself, not the product. Start small: 20–50 real tasks pulled from actual failures you've hit, not hundreds written speculatively up front. Grade the outcome — did it solve the task? — not the exact path taken; an agent that reaches the right answer a different way than you expected isn't wrong.

Split them the way you'd split any test suite: a capability eval is a hard task you don't reliably pass yet — a target to improve toward, low pass rate expected. A regression eval must stay near 100% — it exists purely to catch backsliding when you change a prompt, a skill, or a model version. A capability eval graduates into a regression eval once it's reliably passing. After changing your own workflow, sample transcripts by hand occasionally — a good score alone can hide a broken grader or an eval that quietly stopped measuring anything real.

Concretely, for the salon app: a capability eval might be "implement a full two-way calendar sync from a one-line ticket" — hard today, a target for tomorrow. A regression eval might be "given a bug ticket, the /tdd flow still writes a failing test before any implementation" — a promise about your workflow that must never quietly break, exactly the way Section 5's tests are promises about the product.

And an eval needs no framework — a folder of task files plus a loop over them is the whole apparatus. Here is one complete regression eval for the salon app, a single small file:

# evals/regression/cancel-booking.yaml
prompt: >
  Fix: cancelling a booking under 24h before the slot
  should charge the late-cancellation fee (ticket #142).
repo_state: eval-cancel-base        # a tagged, known-good commit
pass:
  - the booking-cancellation test suite passes
  - no file outside /booking changed

The runner is a dozen lines of shell: for each task file, check out the tagged repo state, run the agent in headless mode with the prompt (claude -p, or your tool's equivalent), then check the pass criteria mechanically — run the named suite, git diff --stat against the allowed paths — and append pass/fail to a results log in the same folder. Re-run the folder after every prompt, skill, or model-version change; a regression eval that dips from 100% is the workflow telling you precisely what broke.

Compensate for the model's knowledge cutoff

A model's training has a cutoff date; the web platform doesn't stop moving after it. Left to its defaults, an agent reaches for the pattern it was trained on — a carousel library, a hand-rolled modal wired up with a stack of ARIA attributes, a resize-observer for a responsive component — even when a newer, native platform feature now does the same job with less code. As one concrete example: by 2026, Chrome and Safari can build an accessible carousel with the CSS ::scroll-marker and ::scroll-button pseudo-elements and no JavaScript at all — a feature that shipped after many models' training data, and still isn't supported everywhere, so an agent needs to be told to check for it (and wrap it in @supports with a fallback) rather than silently reinventing it in JavaScript. If those feature names mean nothing to you, that's fine — you don't need to understand them; the pattern is the point: hand the agent current docs instead of trusting its memory.

The fix isn't hoping the model happens to know — it's the same context-engineering principle as everywhere else in this section: hand it current information instead of relying on memory. Google's own answer to exactly this problem is Modern Web Guidance — a free, Apache-2.0 skill pack (npx modern-web-guidance@latest install, or a Claude Code plugin from the GoogleChrome/modern-web-guidance marketplace) that injects current Chrome best practice and Baseline data — the shared signal for which web features are safely usable across major browsers — straight into the agent's context. Google's own measurement: it lifted best-practice adherence 37 percentage points in testing, with one demo (a dark/light-mode toggle) shrinking from 172 kB of hand-rolled JavaScript to under 3 kB of modern CSS. Install something like it, or at minimum point the agent at up-to-date documentation (an MCP documentation tool, or the relevant MDN/browser-vendor page) before it builds a common UI pattern, and put "prefer a native platform feature over a library, where one now exists" in AGENTS.md as an explicit instruction, not an assumption. Less code shipped isn't just tidier — it's fewer dependencies to secure (Section 9), fewer places to hide a bug, and less for the next person, human or agent, to understand cold.

npx modern-web-guidance@latest install   # Modern Web Guidance skill pack

Budgeted context and deterministic gates are, so far, habits you run by hand. Section 13 makes them permanent — baked into a global config and a golden-path template, so every future project starts with the meter visible and the table rules already in place.