Multi-model agents and routers
Most production agents aren’t “one model for every task.” A cheap model triages or retrieves, a stronger one reasons over the hard cases, or a router picks per request — mixing models is usually more cost-efficient than picking one model for everything, not less. So where does that fit?
Today: the whole flow is one configuration
Section titled “Today: the whole flow is one configuration”ParetoOps doesn’t need to know how a result was produced — only what it cost in total and whether
the task passed. That means a multi-model flow slots in exactly like a single-model one: give it
one configId (e.g. router-v2), and record, per task:
costUsd: the sum of every model call the flow made for that task (triage + reasoning + anything else).passed: whether the flow’s final answer was correct.taskId: so it can be paired against a baseline, same as any other config.
{ "configId": "router-v2", "configName": "Haiku-triage + Sonnet-reasoning", "taskId": "ticket-042", "model": "router-v2", "provider": "custom", "costUsd": 0.0043, "passed": true }Every feature already works on this: analyze places router-v2 on the frontier against
single-model configs on cost-per-successful-task; gate --baseline catches a regression if a
router change makes it worse or pricier; watch can early-stop a losing router variant the same
way. The comparison is honest precisely because it doesn’t care what’s inside the box — a
3-call router and a 1-call model are judged on the same two numbers everyone actually pays for.
One thing that doesn’t apply automatically: --reprice
Section titled “One thing that doesn’t apply automatically: --reprice”gate --reprice recomputes cost from token counts against the
pricing catalog, matched by model + provider. A composite flow’s model field (router-v2
above) has no catalog entry, so --reprice leaves its costUsd as originally recorded rather than
guessing at a blended rate. Two ways to still get that benefit for a router config:
- Keep recording the actual total spend per task (what the example above already does) — this is already correct and doesn’t need repricing.
- Or register your own blended rate for the virtual model name in
pareto.pricing.json(pareto-ops pricing:set --model router-v2 --provider custom --prompt <rate> --completion <rate>) if you want--repriceto recompute it from token totals instead.
Optimizing within a flow: step-level decomposition (Pro)
Section titled “Optimizing within a flow: step-level decomposition (Pro)”Comparing whole flows tells you that router-v2 beats always-sonnet. In Pro, ParetoOps can also decompose agentic runs and multi-step traces into their constituent LLM steps via analyze --by-step:
- Step-level cost attribution: pinpoint exactly how much each step (e.g.
triage,planner,tool_call,synthesizer) contributes to overall task spend. - Per-step Pareto frontiers: evaluate alternative models at each individual step to identify drop-in upgrades or cost reductions (e.g., swapping a $3/M model for a $0.20/M model for preliminary triage without sacrificing end-to-end task accuracy).
- Native trace ingestion: automatically extracts step-level spans from LangSmith child runs, OpenTelemetry GenAI spans, and Braintrust traces.
In Python, this is available via the Pro API analyze_steps(trials).