Editorial illustration for Five Small, Open-Weight Models Built for Agentic Tool Calling
Small Open-Weight Models Excel at Tool Calling
Five Small, Open-Weight Models Built for Agentic Tool Calling
Five small, open-weight models built specifically for tool calling landed in agent pipelines this year, and none of them come from the usual frontier labs chasing benchmark headlines. NVIDIA's research team spent 2025 making a case that ran against the industry's default setting: bigger context, more parameters, sharper reasoning, all in service of agents that mostly do the same handful of narrow jobs over and over. Call a function, parse a return value, format a response, repeat. That's not a job description that needs a generalist model with billions of parameters sitting behind an API call and a per-token invoice.
The shift shows up in what teams are actually shipping in early 2026. Instead of routing every agent action through a frontier model, engineers are slotting in small language models trained specifically for tool calling, function selection, and structured output, tasks that are repetitive and bounded rather than open-ended. The five models below represent that pattern in practice: open weights, small enough to run cheaply, and built around the narrow work agents actually spend most of their time doing.
If you want a closer look at which specific models are leading on this right now, KDnuggets recently rounded up five small, open-weight models built specifically for agentic tool calling, spanning a few billion parameters each and built to run without a data center behind them. Powering Heterogeneous Systems Where Big and Small Models Split the Work The most architecturally interesting use of SLMs isn't replacing large models outright; it's pairing them. The pattern that's become standard in 2026 puts a high-reasoning frontier model in the role of planner, handling strategy and ambiguity resolution, while domain-specific small models act as the workers, each fine-tuned for one atomic task like parsing, classification, or summarization.
Why this matters
The five models KDnuggets flagged aren't a curiosity, they're a signal about where budget and engineering time should go next. If a model with a few billion parameters can handle tool calling reliably on a laptop or a single GPU, that changes the math for anyone building agents that need to run cheaply, locally, or at scale without a frontier API bill attached. For founders, this is a cost and latency argument as much as a technical one.
For developers, it means the interesting work is shifting toward fine-tuning and orchestration rather than just prompting the biggest model available. For researchers, the open-weight part matters most: these models can actually be inspected, retrained, and benchmarked instead of treated as a black box behind an API. We'd push back on any claim that small models now match frontier ones across the board, that's not what's being shown here.
What's being shown is narrower and more useful: for the specific job of calling tools correctly, size may not be the constraint people assumed it was.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Guide Shows How Python Connects to Existing AI Models via Custom Requests
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives Meta Tests GPT-5.5 With 'Watermelon' Model
From Archives GPT-4o, Gemini, and Claude Vision Can Reason Over Visual Details
Common Questions Answered
Why did NVIDIA's research team focus on small, open-weight models for tool calling instead of pursuing larger models with more parameters?
NVIDIA's research team challenged the industry's default approach by demonstrating that agents performing narrow, repetitive tasks don't require massive context windows and parameters. Their strategy prioritized efficiency for specific functions like calling functions, parsing return values, and formatting responses, which are the core operations most agents perform repeatedly.
What are the key advantages of using small, open-weight models with a few billion parameters for agentic tool calling?
Small open-weight models can run reliably on a laptop or single GPU without requiring a data center, which significantly reduces costs and latency for developers building agents. This approach eliminates expensive frontier API bills while maintaining the ability to deploy agents at scale locally or on modest hardware infrastructure.
How do small and large models work together in heterogeneous systems for agentic applications?
Rather than replacing large models entirely, the most architecturally interesting pattern pairs small and large models together, with each handling different aspects of the task. This hybrid approach allows small models to handle tool calling efficiently while leveraging larger models where their capabilities are truly needed.
What does the emergence of these five small, open-weight models signal about future investment in AI development?
These models indicate that budget and engineering time should shift toward building efficient, specialized tools rather than pursuing ever-larger general-purpose models. The signal suggests that for cost-conscious developers and founders, the practical advantages of local, cheap, and scalable agent deployment outweigh the appeal of frontier model capabilities.
