Start With Risk, Not Benchmarks
Long autonomous agents do not fail because every step is too hard. They fail because a few high-leverage decisions are made cheaply, then thousands of cheap steps faithfully amplify the mistake. The routing policy should start from blast radius: architecture, ambiguous requirements, migration strategy, security-sensitive changes, and final critique deserve frontier reasoning. Routine inspection, file-by-file edits, test triage, formatting, copy swaps, and bounded generation should default to workhorse models.
This is infrastructure, not preference. Encode task classes in the harness: `architect`, `critic`, `implementer`, `reviewer`, `triage`, `research`, `fixup`. Give each class a default model, effort level, token ceiling, tool permissions, and escalation rule. The expensive model should see dense, curated state and make irreversible decisions. The cheaper model should run the volume: search the repo, summarize findings, patch narrow files, rerun tests, and produce artifacts for the critic.
The failure mode is routing by vibes: using the flagship model for every terminal turn because it feels safer, or using the cheap model everywhere because the budget dashboard is scary. Both are sloppy. Route by expected cost of error. A bad CSS spacing pass is cheap to revert. A bad persistence abstraction can poison the overnight run.
Make The Harness Decide
Once routing is risk-based, the harness has to enforce it mechanically. Do not rely on the agent to remember budget discipline after six hours of context churn. A practical loop stores state outside the conversation: run id, goal, current phase, branch or worktree, model used, token spend, test results, open defects, escalations, and artifacts. Postgres, Neon, SQLite, or even JSONL is fine if it is durable and queryable. The point is that resumption and routing come from state, not chat memory.
A typical policy is `research -> plan -> implement -> verify -> critique -> fix`. Research can be parallelized on cheaper models with read-only tools. Planning may start cheap, but the final plan goes to the frontier model if it changes architecture, APIs, data shape, auth, payments, deployment, or user-visible interaction contracts. Implementation is workhorse by default, split into small patches. Verification is mostly deterministic: tests, linters, typechecks, benchmarks, browser checks, screenshots, log scans. Critique escalates when failures are ambiguous, when the loop repeats the same fix twice, or when a diff touches sensitive paths.
This also solves context bloat. Each phase gets a fresh context containing only the state record, relevant files, acceptance criteria, and artifacts from prior phases. Sub-agents return structured findings, not sprawling transcripts. Old project rules and `agents.md` files should be pruned after model upgrades because stale steering wastes context and can actively degrade newer models.
Exploit Parallelism Without Losing Control
Routing becomes more valuable when the loop fans out. Ten cheap agents in isolated worktrees can inspect performance, security, API compatibility, UX regressions, test failures, and dependency risk in parallel. They should not all be allowed to mutate the same repo. Most sub-agents should be read-only analysts that produce ranked findings with file references, reproduction steps, confidence, and suggested fix shape. The manager then chooses which changes to implement.
For write-heavy parallelism, isolate everything: git worktree per agent, separate database branch or fixture namespace, separate ports, separate temp dirs, and separate logs. Merge through a deterministic queue. If two agents touch the same files, force a critic pass before combining. Massive file-by-file migrations can work well with cheap or mid-tier coding models, but only when the transformation is narrow and verification is strong. The frontier model should define the migration contract and audit sampled diffs, not grind through every file.
The trap is confusing more agents with more autonomy. Parallel loops multiply hidden mistakes unless the harness constrains the surface area. Give each worker a bounded prompt, a timebox, a token cap, and an output schema. If it stalls, do not let it ramble. The manager should terminate, summarize the partial state, and either reroute to a stronger model or shrink the task.
Spend Frontier Tokens At The Edges
A good model router spends expensive reasoning at compression points. Use it to turn messy requirements into a precise goal, to choose architecture, to critique a large diff, to diagnose repeated failure, to decide whether a test failure indicates bad code or bad tests, and to simplify overcomplicated output. Then downshift. UI iteration, CSS adjustments, fixture generation, mechanical refactors, documentation sweeps, benchmark iteration, and code-review deduplication rarely need the top model after the target is clear.
For design-heavy or interaction-heavy work, spend high effort on the first scaffold and behavioral contract, then medium or cheaper models for iteration. For codebase-wide review, run parallel cheap passes first, dedupe findings, then ask the stronger model to rank severity and choose the fix order. For research-heavy work, let cheaper agents gather and normalize evidence, but route the final judgment to the model with the best long-horizon reasoning. For high-latency loops, fast models can be excellent workers even if they are poor one-shot creators.
Track this by completed task, not tokens alone. Log model, phase, wall time, input tokens, output tokens, cache reads, retries, test pass rate, and whether a human intervened. A cheap model that loops six times may be more expensive than one frontier pass. A frontier model that writes dense but unreadable code may create review debt. Your router should learn from local outcomes, not vendor leaderboard claims.
Put Safety In The Policy
Model routing is incomplete without permission routing. The same task class that chooses the model should choose tools. Research agents get read-only filesystem and search. Implementers can patch the worktree but not deploy. Deployment agents require approval or a narrow command allowlist. Browser automation should be available for real workflows like authenticated admin changes, DNS, hosted dashboards, and app testing, but it needs the same guardrails as shell access.
Add pre-tool-use hooks for catastrophic commands and sensitive paths. Block root, home, broad recursive deletes, credential files, production databases, and unscoped cloud operations. Pair this with approval modes that match the loop. Overnight work can be autonomous inside a sandbox, but production mutation should be a separate routed phase with stricter policy. The best router does not merely ask “which model?” It asks “which model, with which context, which tools, which budget, and which exit criteria?”
That is the central shift: frontier models become scarce control-plane components, not the default executor. Let them make the decisions that shape the run. Let cheaper workers do the volume under deterministic verification. Long autonomous loops become reliable when the expensive reasoning is concentrated where error compounds, and everything else is bounded, measured, and replaceable.
Key takeaways
- Route by blast radius: spend frontier reasoning on architecture, critique, ambiguous failures, and sensitive changes.
- Store loop state outside chat so model choice, resumption, and escalation are driven by durable facts.
- Run cheap sub-agents in parallel for bounded analysis, then merge through a supervised manager and critic pass.
- Measure cost per completed task, including retries and human intervention, not just price per token.
- Bind model routing to permission routing: task class should determine tools, approvals, budgets, and exit criteria.