Editorial illustration for Kyutai's Speech Model Solves Spoken Math Problems Using New Audio-Token Method
Kyutai's Speech Model Solves Math Without Text
Kyutai released Voice of Reason on Hugging Face this week, two open-weight speech-to-speech models built to solve math problems without ever converting audio to text. Both start from GLM-4-Voice-9B, a model that normally scores 27.3% on spoken GSM8K, and push accuracy to 77.1% through a combination of supervised fine-tuning and reinforcement learning. No transcription step. No separate text-based LLM sitting in the pipeline doing the actual reasoning.
That distinction matters because most speech assistants still rely on cascaded systems: convert speech to text, run a text model, convert the answer back to speech. It works, but every stage adds delay, and the process throws away tone, pacing, and other audio cues that carry meaning. Speech-native models skip that overhead, but they've historically struggled with reasoning tasks like math because they have to keep producing audio at a steady clip to sound conversational, which leaves little room for the kind of extended "thinking" text models get to do silently.
Kyutai's team says this is the first time reinforcement learning has been applied to math reasoning inside a speech-native model, and the training setup behind that jump from 27.3% to 77.1% is where the real engineering happened.
Base GLM-4-Voice scores 27.3% on GSM8K. The earlier STITCH method raised that to 58.7% by adding reasoning chunks. The research team call their work the first application of RL to math reasoning in speech-native models.
Why this matters
The jump from 27.3% to 77.1% on spoken GSM8K is the headline number, but the audio-token merging trick is the part worth watching. Collapsing the audio vocabulary into a single abstract token at each position sidesteps a problem that's dogged speech-to-speech systems: you don't need to know exactly which sound comes next, just that some valid audio does. That's a cheaper, lower-variance signal for RL, and if the value-invariance proof holds up under scrutiny, it's a technique other speech model builders will want to borrow rather than reinvent.
For developers, the practical draw is that this runs on one H100 in BF16 and skips the transcribe-then-reason-then-synthesize pipeline most voice assistants still lean on. Fewer moving parts means fewer places for latency and error to creep in. Founders building voice agents that need to handle arithmetic, scheduling, or any task with a checkable answer should take note. Researchers, meanwhile, get a genuinely testable claim: does this merging approach generalize past math to other structured spoken tasks, or is GSM8K a special case dressed up as a breakthrough.
Related Reading
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
Open Source Box CEO Aaron Levie Says AI Will Drive "Biggest Cybersecurity Upgrade
Open Source Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
From Archives ROBOTIS Integrates NVIDIA Isaac ROS Into AI Worker for Vision-Guided Tasks
From Archives Xiaomi Credits AI Gains to Expanded Reinforcement Learning
Common Questions Answered
How does Kyutai's Voice of Reason model solve spoken math problems without converting audio to text?
Voice of Reason uses a speech-native architecture that processes audio directly without a transcription step or separate text-based LLM in the pipeline. Instead, it employs an audio-token merging method that collapses the audio vocabulary into single abstract tokens at each position, allowing the model to reason directly on audio representations.
What accuracy improvement did Kyutai achieve on the spoken GSM8K benchmark compared to the base GLM-4-Voice model?
Kyutai improved accuracy on spoken GSM8K from the base GLM-4-Voice score of 27.3% to 77.1%, representing a significant 49.8 percentage point increase. This improvement was achieved through a combination of supervised fine-tuning and reinforcement learning applied to the audio-token method.
What is the audio-token merging technique and why does it matter for speech-to-speech systems?
Audio-token merging collapses the audio vocabulary into a single abstract token at each position, which sidesteps the traditional problem of predicting exactly which sound comes next. This approach provides a cheaper, lower-variance signal for reinforcement learning since the model only needs to know that some valid audio should follow, rather than predicting specific audio outputs.
How does the STITCH method compare to Kyutai's Voice of Reason in solving spoken math problems?
The earlier STITCH method raised the base GLM-4-Voice accuracy from 27.3% to 58.7% by adding reasoning chunks, while Kyutai's Voice of Reason achieves 77.1% accuracy. Kyutai's research team claims their work is the first application of reinforcement learning to math reasoning in speech-native models, surpassing STITCH's performance.
What models does Kyutai's Voice of Reason build upon and where is it available?
Voice of Reason consists of two open-weight speech-to-speech models built on top of GLM-4-Voice-9B, which originally scored 27.3% on spoken GSM8K. The models have been released on Hugging Face as open-weight models, making them publicly available for use and further research.
Further Reading
- Kyutai releases speech-native math models that skip the text LLM - RuntimeWire
- kyutai/glm-4-voice-of-reason-9b - Hugging Face
- kyutai/glm-4-voice-of-reason-stitch-9b - Hugging Face
- Voice of Reason: Reinforcement Learning for Spoken Math - arXiv
- GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbots - arXiv
