Editorial illustration for Switchyard: A Rust Library That Routes LLM Traffic Between OpenAI and Anthropic
Switchyard: Route LLM Traffic Between OpenAI, Anthropic
Claude Code talks to the Anthropic Messages API. Codex CLI talks to OpenAI's format. The model a team wants to actually run often sits behind vLLM, NVIDIA NIM, or Ollama, speaking neither dialect natively. Nobody wants to fork and rewrite a coding agent just to swap out a backend, so that translation work has to happen somewhere between the client and the model.
NVIDIA's answer is Switchyard, a Rust proxy and library released under Apache 2.0, with docs posted at docs.nvidia.com/nemo/switchyard. It routes LLM requests across providers, converts between OpenAI and Anthropic wire formats on the fly, and logs operational metrics while doing it. The routing logic itself is typed and composable, meant to be assembled rather than hardcoded. The binary ships on crates.io, the launcher is on PyPI, and it can be self-hosted anywhere a team wants to run it.
NVIDIA is direct about where the project stands: pre-alpha, experimental, explicitly not for production, with the API and routing algorithms expected to change substantially before a v1.0 lands. That framing matters for anyone deciding whether to plug it into a live agent stack today.
Why this matters
Switchyard is NVIDIA solving a plumbing problem that's been slowing down anyone building agent infrastructure: the tools your team wants to use (Claude Code, Codex CLI) speak different dialects than the model backends you actually run (vLLM, NIM, Ollama). Instead of forking agents to match APIs, NVIDIA is betting the fix belongs in a thin, embeddable routing layer. The library path is the interesting part for builders.
By keeping switchyard-libsy free of an HTTP stack and handing every model call back to the caller, NVIDIA is drawing a hard line between "decide where a request goes" and "own the network request." That's a more composable design than a typical proxy, and it means teams can bolt routing logic into existing Rust services without inheriting a new runtime dependency. For founders wiring together multi-model products, this lowers the cost of staying provider-agnostic. For researchers, it's worth watching whether the routing algorithms stay simple dispatch or grow into something closer to real load-balancing intelligence.
Open source, so scrutiny of that roadmap starts now.
Related Reading
Browse all OpenAI news →
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic OpenAI's GPT Transcribe Cuts Error Rate to 3.31%, Improving on GPT-4o
Open Source
Related Topic OpenAI Calls Apple's Lawsuit 'Rotten to Its Core' in Rebuttal
Policy & Regulation
From Archives Google's Gemini 3.8 Flash Priced at USD 0.58 Per Task
From Archives OpenAI's Astra Model Scores Perfect on Security Exploit Test
From Archives Trump Administration Backs OpenAI in Copyright Case
Policy & Regulation
From Archives OpenAI Limits Astra's AI Reasoning Technique for Safety Monitoring
Open Source
Common Questions Answered
What problem does Switchyard solve for teams building LLM agent infrastructure?
Switchyard solves the API dialect mismatch problem where tools like Claude Code and Codex CLI speak different formats than backend models running on vLLM, NVIDIA NIM, or Ollama. Instead of forking and rewriting coding agents every time you swap backends, Switchyard provides a translation layer that handles the format conversion automatically between the client and model.
How does Switchyard translate between OpenAI and Anthropic API formats?
Switchyard is a Rust proxy and library that acts as an intermediary between clients and LLM backends, automatically translating requests and responses between OpenAI's and Anthropic's API formats. This allows teams to use the same agent code regardless of whether they're routing to OpenAI, Anthropic, or self-hosted model backends.
What additional capabilities does Switchyard provide beyond API translation?
Beyond translating between OpenAI and Anthropic formats, Switchyard records operational metrics, exposes typed and composable routing algorithms, and functions as both a proxy service and an embeddable library. This flexibility allows teams to integrate Switchyard either as a standalone routing service or directly into their applications without requiring an HTTP stack.
Why is Switchyard's library path particularly valuable for builders?
Switchyard's library path is valuable because it allows builders to embed the routing logic directly into their applications without needing a separate HTTP service, keeping the solution lightweight and flexible. This approach enables teams to maintain a single codebase while seamlessly switching between different LLM providers and backends.
Further Reading
- NVIDIA-NeMo/Switchyard: Switchyard lets LLM traffic route between OpenAI and Anthropic APIs - GitHub
- Switchyard: LLM proxy with API translation and multi-backend routing - Open Source Alternatives
- NVIDIA-NeMo/Switchyard - GitHub mirror coverage - daily.dev
- Managing Local LLM Orchestration - DGX Spark / GB10 Projects - NVIDIA Developer Forums
- Anthropic API Compatibility - LLMGateway Docs
