# LangChain benchmarked NVIDIA's NeMo Switchyard router across multi-step agentic tasks, and the spend came back wildly lopsided

Only 7% of model calls needed the frontier model.

Those calls took 68.4% of the budget. https://www.langchain.com/blog/switchyard-agent-routing-benchmark

The cheaper model, Nemotron 3.5 Lightning, handled 93% of calls for 10.4% of the spend.

Routing the easy work away cut total cost by 74% and gave up roughly six points of accuracy.

Most teams cannot evaluate that trade, because most teams have never measured which of their calls are the expensive 7%.

They pick one model for the whole agent and treat the bill as a fixed cost of doing business.

It is not fixed. It is a routing decision nobody made.

The useful part of the benchmark is not the router. It is the arithmetic underneath it.

LangChain published the break-even directly: the minimum share of traffic you have to offload equals the cost of the judge divided by the gap between the expensive model and the cheap one.

That decides whether routing is worth building before you build it: • Log which calls actually escalate, before choosing a router • Price the judge, because the classifier is not free • Compare cost per completed task, not cost per call • Put the acceptable accuracy floor in writing, or the trade has no shape

NVIDIA shipped the router and the small model together for exactly this reason. https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/

The frontier model stops being the answer once you can see how rarely it is the one doing the work.

One model per agent was never an architecture. It was a default nobody revisited.

#ai #llm #ai-agents #agentic-ai