Editorial illustration for LLMs need fine-grained catalog context: large model routes data to SLMs
LLM Routing: How AI Connects Catalog Details Precisely
LLMs need fine-grained catalog context: large model routes data to SLMs
Most AI is terrible at shopping. It can list ingredients but fails at the actual task: getting what you need when the store is out of stock.
Your brownie recipe calls for cocoa powder. The shelf is bare. An AI might suggest carob powder.
That is a bad, dusty suggestion. A useful system would know you need unsweetened baking chocolate, or maybe a specific type of dark cocoa, because it understands the recipe as a chemical formula, not just a list of words. This is the brownie recipe problem.
It breaks chatbots and recommendation engines every day.
First, data is fed into a large foundational model that can understand intent and categorize products. That processed data is then routed to small language models (SLMs) designed for catalog context (the types of food or other items that work together) and semantic understanding. In the case of catalog context, the SLM must be able to process multiple levels of details around the order itself as well as the different products. For instance, what products go together and what are their relevant replacements if the first choice isn't in stock?
The fix is a handoff. A big, general model interprets your request. It figures out you are baking, not making a mole sauce.
It then passes that intent to a small, specialized model that lives inside a product catalog. This small model knows every nut, bolt, and baking soda brand. It knows which olive oil is for dressing and which is for high-heat frying.
It knows walnuts can be replaced with pecans, but hazelnuts would change the dish entirely.
This architecture accepts a simple truth. One model cannot know everything. The large model sees the forest.
The small models know every tree, the soil around it, and what grows in its shade. For commerce to feel less like talking to a database and more like talking to a competent human, this division of labor is not clever. It is necessary.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives GitHub adds Claude and Codex agents to Copilot’s Agent HQ platform
From Archives Franken‑stack tax drives costly AI pitfalls; platform‑native architecture needed
Common Questions Answered
How do Instacart's Large Language Models (LLMs) handle complex product search queries?
Instacart's approach involves using LLMs to understand user intent for challenging 'long-tail searches' where traditional machine learning models fell short. The strategy moves from context-engineering with guardrails to ultimately fine-tuning models that can transform generalist models into domain-specific experts for product catalog understanding.
What challenges do traditional Query Understanding (QU) systems face when processing user searches?
Traditional Query Understanding systems struggle with broad queries like 'healthy food' that span multiple categories and lack specificity. Additionally, these systems often suffer from a lack of labeled data, making it difficult to precisely interpret user intent and deliver accurate search results.
Why are Small Language Models (SLMs) becoming important for catalog attribute extraction?
Small Language Models offer a more efficient and targeted approach to processing product catalog data, allowing for specialized attribute extraction across millions of SKUs. They can handle diverse attribute requirements, such as numeric reasoning for sheet count or extracting evolving flavor information, while being more cost-effective and scalable than large language models.
Further Reading
- Exploring the Potential of Large Language Models in Fine-Grained Code Review Classification — ArXiv
- Large Language Models Achieve Fine-Grained Opinion Annotation Through Declarative Pipelines — Quantum Zeitgeist
- LLM-Based Fine-Grained ABAC Policy Generation for IT Networks — SCITEPRESS
- LLMs Meet VLMs: Boosting Open Vocabulary Object Detection with Fine-Grained Descriptors — ICLR
