NVIDIA NeMo Switchyard Routes AI Across LLMs

NVIDIA released NeMo Switchyard, a library for routing AI agent workloads across multiple language models instead of locking an application into one. The problem it targets is common in production agents: a single task like computer-use automation might need classification for step one, heavy reasoning for step two, and a cheap model for routine cleanup afterward. Pick one model for the whole job and you either overpay for capacity you don't need or shortchange the steps that need it most.

NeMo Switchyard handles this by evaluating each request at runtime, checking available context, then sending the work to whichever model fits the task's requirements, cost constraints, and policy rules. Developers don't have to rebuild their applications around a specific provider or rewrite logic every time they swap in a new model. The library supports several routing approaches, letting teams tune the balance between accuracy and cost for their own workloads rather than defaulting to the largest, most expensive model for every call.

NVIDIA tested this system-of-models approach on a computer-use task scored against the Terminal-Bench Hard benchmark, comparing routed performance against using a single top-tier model like DeepSeek V4 for everything.