Editorial illustration for PolyAI's Dialog-RSN-1 Fuses Speech Recognition and Response
PolyAI's Dialog-RSN-1 Unifies Speech Recognition
PolyAI released Dialog-RSN-1 on its platform this week, a dialog model that listens to the caller's raw audio instead of working off a text transcript. The system folds turn-taking, speech recognition, function calling and response generation into a single audio-native model, and PolyAI says it's already fielding live production calls for paying customers. The company reports sub-300ms response times, an 11% relative gain in containment at a restaurant group, and a 37% cut in latency at an insurer.
The model only handles audio on the input side; text-to-speech stays a separate step, so the output voice remains something engineers can still control and swap. It runs as a request-based LLM, probed on demand rather than kept as an always-on stream that ties up a GPU. English only for now, and access runs strictly through PolyAI's platform: no open weights, no public API. Existing customers can turn it on today, new ones can request early access.
That deployment model matters because of who PolyAI is selling to. The company reported over 100 enterprise customers and more than 2,000 live deployments at its $86 million Series D in December 2025, spanning restaurants, insurance, healthcare and telecom.
PolyAI has introduced Dialog-RSN-1, a dialog model that perceives the caller’s audio directly instead of reading a transcript. It fuses turn-taking, speech recognition, function calling and response generation into one audio-native model, and is already handling live production calls.
Why this matters
For teams building voice agents, Dialog-RSN-1 is a bet that the transcript layer was always the wrong abstraction. By reading raw audio instead of ASR text, PolyAI claims to preserve tone, hesitation and interruption cues that get flattened the moment speech becomes a string. That's a real architectural choice, not a demo trick, backed by specifics: cache prefilling while the caller talks, append-only prompts, sticky GPU routing, a speculative drafter hitting 3.9 mean accepted tokens. Those are the kinds of numbers you publish when you've actually shipped to production, not when you're teasing a paper.
Keeping TTS separate is the pragmatic part. It means enterprises still get to pick and control the output voice, which matters for brand and compliance reasons that audio-native purists often ignore. The request-based design also avoids the always-on streaming tax that kills unit economics at scale.
Worth watching: whether "audio-native input, separate TTS output" becomes the standard split for production voice AI, or whether someone ships a fully audio-to-audio model with comparable latency numbers.
Related Reading
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
Open Source OpenAI's GPT Transcribe Cuts Error Rate to 3.31%, Improving on GPT-4o
Open Source Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
From Archives Deep Agents v0.7 Updates Base Input Tokens, Performance Validated
From Archives Nscale Acquires Anyscale in Vertical AI Compute Push
Common Questions Answered
How does PolyAI's Dialog-RSN-1 differ from traditional dialog models that use transcripts?
Dialog-RSN-1 processes raw audio directly from callers instead of relying on transcribed text, preserving important vocal cues like tone, hesitation, and interruptions that are lost when speech is converted to text. By fusing turn-taking, speech recognition, function calling, and response generation into a single audio-native model, it eliminates the transcript layer as an intermediary abstraction.
What performance metrics has PolyAI reported for Dialog-RSN-1 in production?
PolyAI reports sub-300ms response times, an 11% relative gain in containment at a restaurant group customer, and a 37% cut in latency at an insurance customer. These metrics demonstrate the model's effectiveness in real-world production environments where it is already handling live calls for paying customers.
What technical optimizations does Dialog-RSN-1 use to achieve fast response times?
Dialog-RSN-1 employs several architectural optimizations including cache prefilling while the caller is speaking, append-only prompts, sticky GPU routing, and a speculative drafter that achieves a 3.9 mean accepted token rate. These techniques work together to minimize latency and enable the model's sub-300ms response performance.
Why is processing raw audio instead of ASR text considered a significant architectural choice for voice agents?
Processing raw audio preserves paralinguistic information such as tone, hesitation, and interruption cues that are inherently flattened when speech is converted to text transcripts. This architectural approach allows Dialog-RSN-1 to better understand caller intent and context, leading to improved performance metrics like higher containment rates and reduced latency in production deployments.
