Editorial illustration for Google’s AMIE AI conducts first real-time video medical consultations
Google's AMIE AI Conducts Real-Time Video Medical Consults
Google Research and Google DeepMind put their experimental medical AI system, AMIE, in front of a camera for the first time. Until now, AMIE's clinical reasoning had been tested mostly through text-based exchanges, typed symptoms in, diagnostic questions out. That format misses most of what actually happens in an exam room. A doctor clocks a limp before a patient mentions hip pain, hears a wheeze that never gets described out loud, watches someone wince when they shift in their chair.
To close that gap, researchers built AMIE on top of Gemini and Project Astra, using a multi-agent architecture that lets the system process video and audio alongside text. The goal was a version of AMIE that could watch a patient, guide them through a physical exam over video, and reason through a diagnosis in real time, the way a clinician would on a video call. Google researchers then ran a randomized study with simulated consultations, pairing the system against primary care physicians using trained patient actors, to see whether it held up on the basics: taking a history, reaching an accurate diagnosis, communicating clearly.
In a randomized study using simulated consultations with patient actors and a group of primary care physicians, clinical evaluators assessed AMIE favorably across core clinical competencies, including history-taking thoroughness, diagnostic accuracy, management appropriateness and communication quality.
Why this matters
Google hasn't published error rates, patient outcomes, or details on how AMIE handles ambiguous visual cues like a subtle limp versus a deliberate one. The demo shows the system responding to video in real time, which is a real engineering feat, but a study is not a deployment. We've seen this pattern before with medical AI announcements: an impressive controlled demonstration followed by years of regulatory and liability questions before anything touches an actual clinic.
For developers and founders building in health tech, the signal here isn't "video diagnosis is solved." It's that multimodal, real-time reasoning over live video is now a component you can plausibly build toward, which matters far beyond medicine. For researchers, the open questions are the interesting part: how does AMIE weigh visual evidence against verbal history, and what happens when the two conflict? Google calling this "first-of-its-kind" is marketing language worth noting but not taking at face value.
Watch for peer-reviewed clinical trial data, not blog posts, before treating this as validated medicine rather than an impressive tech preview.
Related Reading
Browse all Google news →
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
Related Topic NVIDIA and Google Cloud let developers scale AI from prototype to production
Market Trends
Related Topic Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI
From Archives Mathematician Questions AI Breakthroughs' Real-World Impact
From Archives New Method Could Let Anyone Distill AI Models' Secrets
From Archives Deepmind WeatherNext Cuts Cyclone Forecast Error to 230-Kilometer Average
From Archives Google's DiffusionGemma Solves 85% of Puzzles After Minimal Fine-Tuning
LLMs & Generative AI
Common Questions Answered
What is the key difference between AMIE's previous text-based testing and its new real-time video consultations?
Previously, AMIE's clinical reasoning was tested through text-based exchanges where typed symptoms were input and diagnostic questions were output. The new real-time video format allows AMIE to observe physical cues that are missed in text interactions, such as a patient's limp, wheezing sounds, or wincing movements during examination, which are critical components of actual medical consultations.
What clinical competencies did evaluators assess AMIE on in the randomized study?
Clinical evaluators assessed AMIE across four core clinical competencies: history-taking thoroughness, diagnostic accuracy, management appropriateness, and communication quality. The study used simulated consultations with patient actors and a group of primary care physicians to conduct these evaluations.
Why does Google's AMIE announcement not guarantee immediate clinical deployment?
Google has not published error rates, patient outcomes, or details on how AMIE handles ambiguous visual cues, and the demonstration represents a controlled study rather than real-world deployment. Medical AI systems typically face years of regulatory and liability questions before they can be used in actual clinical settings, following a pattern seen with previous medical AI announcements.
What engineering achievement does AMIE's real-time video response capability represent?
AMIE's ability to respond to video input in real time is described as a significant engineering feat, demonstrating that the system can process visual medical information and generate clinical responses without substantial delay. This real-time processing capability is essential for conducting interactive video consultations that mimic the flow of actual doctor-patient interactions.
Further Reading
- Towards Expert-level Medical AI for Real-time Video Consultations - Google Research
- AI co-clinician: researching the path toward AI-augmented care - Google DeepMind
- Advancing AMIE towards specialist care and real-world validation - Google Research
- AMIE gains vision: A research AI agent for multimodal diagnostic dialogue - Google Research
- A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic - arXiv
