Jev gotta be kidding me! You used Fable tokens for this?

At AmpUp, a course is not built by one agent. It is built by a team of them, with a human at every handoff. It feels less like prompting a chatbot and more like running a studio. Here is the crew, and the model each one runs on.

Agent	Model	Why this model
Outlining	Sonnet	Structures the course from a goal and a syllabus. Bounded and well specified.
Subject matter	Opus	Synthesizes source material into accurate content. The correctness bar is highest here.
Storyboarding	Opus	Turns raw content into a teachable arc, scene by scene. Pedagogical judgment, not a template.
Designer	Opus	Lays out and styles each slide from a brief. Open-ended visual reasoning over a lot of context.
Marketing / brand	Sonnet	Applies brand guidelines to finished slides. Rule-following, not open-ended.
Instructor	Sonnet	Writes the talking-head script to a set delivery format.

The split is deliberate. The three agents that synthesize, sequence, and design earn Opus, because getting those wrong is expensive and hard to reason back from. The three that apply a structure or a rule run on Sonnet. So no agent is on a bigger model than its job needs, and that is already a saving over running the whole studio on one frontier tier.

The gap this post is about opens up within each agent. An Opus agent and a Sonnet agent both spend their day fielding a mix of real authoring and small edits. Once a course exists, most requests are small: delete this slide, remove the narration, move the summary to the front, make this slide more redder. Those still run on whatever model the agent is set to, so a three-word nudge lands on Opus. What we always wanted was a layer that looked at each request and predictably said which model that turn deserved.

Two reflexes on a reasoning model

A frontier chat model reasons. It is flexible, and it is slow and expensive per call. Call that System 2. A purpose-built classifier recognizes. It is narrow, and it is fast and cheap. Call that System 1. Agents run almost everything through System 2, including two jobs that are pure reflex and fire on every turn.

Should this action run? A permission check ahead of each tool call. And which model should handle this turn? A routing decision ahead of each request. Both are classification, not generation. We measured both, using Jev, a typed-decision classifier reached through OpenRouter.

Decision one: the permission gate

Auto mode classifies every action before it runs, into allow, block, or ask. On our course agent a typical build makes about 34 tool calls, and heavy iteration sessions reach into the hundreds. Each one is a classification if the gate is on. Run that gate on a chat model and you have put a reasoning engine in a control loop where it waits.

At roughly $0.0000227 a decision, the gate itself is a rounding error. A fast, calibrated gate can sit in front of every action without anyone waiting on it. Low-confidence answers and API failures fall back to human review, so a broken classifier slows the agent down rather than opening a hole.

Decision two: model routing

This is the one with the money in it. A permission gate saves you the gate’s own cost. A router saves you the cost of the thing it gates, the frontier model call itself.

How we measured it
Sample	150 real course-building conversations, 174 priced user turns.
Cost	Actual per-turn spend from each turn’s total_cost_usd. Total $661.39, mean $3.80 per turn.
Classification	Each request bucketed locally by complexity. No customer text left the process.
Prices	Live OpenRouter rates. Haiku is exactly one third of Sonnet and one fifth of Opus, on input and output.

Sort the turns by what the request actually is, and the share of turns stops matching the share of spend.

Open builds are 46% of turns but 64% of spend, and they should stay on the frontier model. The routing opportunity is the mechanical turns, and those are not cheap. A two-word command re-runs the full course context through the frontier model. In our sample, 22 mechanical turns cost more than $2 each, and together they are 15% of all spend. “Update the side nav” cost $4.49.

The routing target is not “route a third of turns.” It is the turns where a two-word ask triggers a frontier rebuild. Those few turns carry most of the recoverable cost.

Pricing the risk

A cheaper model is only a saving if it gets the edit right. A wrong edit that has to be redone on the frontier model costs the cheap attempt plus the retry, so it is worse than never routing. Per dollar of a downgraded turn, at price ratio r and failure rate p:

saving per $  =  (1 - p)(1 - r) - p*r  =  (1 - r) - p

Savings vanish only at p = 1 - r: a 67% failure rate against a Sonnet base, 80% against Opus. That break-even is the useful output. It turns an unknown (“what is the failure rate?”) into an easy question (“is it plausibly under two thirds?”). For commands like “delete” and “add talk track”, obviously yes.

Is the failure rate a guess? We checked.

We could not replay real customer turns on the cheap model, because sending real course content to an external model is exfiltration and a full-agent replay would mutate live data. So we ran the same mechanical commands on Haiku and on Sonnet over an identical synthetic course, scored against a known-correct answer.

Command	Haiku vs Sonnet	Correct
Delete a slide	byte-identical	both
Remove narration	Haiku dropped the field, Sonnet emptied it	both
Reorder to front	byte-identical	both
Rename course	byte-identical	both
Generate quiz talk track	comparable narration	both

Haiku matched the frontier model on all five, with zero wrong edits. The one divergence is the instructive part: on “remove narration”, Haiku deleted the field while Sonnet set it empty. Both are correct, but if the schema requires the field, Haiku’s shape would be rejected and retried. That is a real failure vector, and it is a schema-shape mismatch you fix by constraining the output, not a reasoning failure. This is a small, synthetic, single-shot run, so read it as indicative. It still puts the true failure rate near the left edge of the chart above.

What it means for AmpUp

The measured build averages $8.59 in model spend across its turns. At an assumed 100 courses a day, that one agent is about $859 a day, roughly $215,000 a year. Routing the mechanical turns to Haiku, at the conservative 10% failure rate:

That is 11% to 18% of one agent’s spend, from routing alone, with authoring untouched. It is a lower bound, since the same pattern applies to every other agent in the pipeline.

How the router works

The router is a PreToolUse-style hook, the same slot the permission gate uses, plus a pre-model decision on each user turn. It sends only the request and the working context to Jev, which returns a typed choice with calibrated confidence. Low confidence routes up to the safer model rather than guessing. The Claude Agent SDK, agno, and deepagents all expose these hooks, so this is not specific to our stack. Every team on those frameworks faces the same fork: skip the checks and run fast but blind, or put a chat model in the loop and pay for it on every turn. A System 1 classifier is the third option.

What Jev is worth

Two numbers, one for cost and one for time.

On cost, routing the small turns to a smaller model saves 11 to 18 percent of the design and edit agent’s spend, about $23,000 to $40,000 a year at 100 courses a day. The routing decisions that earn that saving cost about half a cent a day, so the layer costs nothing worth counting against what it returns.

On time, the same classifier used as the permission gate answers in about 264 milliseconds where a chat-model gate takes around four seconds. Over the roughly 34 checks in a build that is close to two and a half minutes of waiting removed, down to under ten seconds. A simple edit routed to a faster model also comes back sooner, so the person at the handoff is not watching a spinner while Opus thinks about a rename.

Both numbers come from one fact about Jev: a decision that costs $0.0000227 and lands in a quarter of a second can sit in front of every turn, sorting what deserves the expensive model from what does not.

What we are not claiming. The permission-gate latency uses a published LLM-judge baseline, not a live measurement of auto mode’s own classifier. The routing study covers one agent’s real traffic, and 100 courses a day is an assumed scale. The routing tiers came from a heuristic, not a trained router, and the failure rate is validated only on a small synthetic run. We assume a downgraded turn runs the same token flow at the cheaper price; a weaker model that takes more rounds would erode that.

Where this stands

We are still watching this for quality gaps. A router can be wrong in ways a fixed setup never is, and we are mapping exactly where those failures show up before we lean on it any harder. So far it holds. The edits come back right, the authoring turns are untouched, and most days I go home having saved a Claude Max membership’s worth of spend on this one agent alone. That is a good trade, and we will keep checking the parts that could bite.

See how AmpUp turns real work into practice that adapts to the learner, without running every keystroke through the biggest model in the room.