Editorial illustration for Google adds screen-control to Gemini 3.5 Flash for cross‑platform agents
Google adds screen-control to Gemini 3.5 Flash for...
Google just taught an AI to use a mouse and keyboard. That's not a metaphor. Gemini 3.5 Flash can now look at your screen—on a phone, a browser, a desktop—and operate it.
Click buttons. Type text. This transforms a chatbot into a potential digital employee, one capable of testing software or processing invoices without engineers building a custom bridge for every single application.
The numbers on the OSWorld benchmark show it's closing the gap: a 78.4 score puts it just behind GPT-5.5 and within striking distance of Anthropic's leading Opus model. And yes, Google sees the glaring risks. So a suite of guardrails, including systems that force a human to confirm dangerous actions, tries to wall off the obvious vulnerabilities.
It's available now. For developers. To break.
Combined with existing tools like function calls, Search, and Maps, developers can now build agents that work across browser, mobile, and desktop environments for tasks like software testing or office automation. On the OSWorld benchmark, Gemini 3.5 Flash scores 78.4, beating Gemini 3 Flash (65.1) and GPT-5.4 mini (72.1). GPT-5.5 sits just ahead at 78.7, while Anthropic's Opus 4.8 leads at 83.4.
Sonnet 4.6 also hits 78.4, and Gemini 3.1 Pro lands at 76.2. To guard against prompt injection attacks, Google uses adversarial training and two optional enterprise safeguards. One requires user confirmation for sensitive or irreversible actions, while the other automatically stops tasks when it detects indirect prompt injections.
Google also recommends sandboxing, human oversight, and strict access controls, with more details in its best practices documentation. The feature is available through the Gemini API and the Gemini Enterprise Agent Platform.
Ignore the benchmark scores. The real story is that Google handed its fastest model a set of hands and then immediately slapped handcuffs on them. Those safety measures—the tripwires, the human checkpoints—are the actual product.
An AI with screen control is a massive, obvious vulnerability. Google's layered security is a necessary admission of that inherent danger. They've built the tool.
The tedious, unglamorous work of preventing havoc now belongs to the user. This is how real automation arrives. Not with a flashy demo, but with a lengthy security document and a big, red "are you sure?" button.
Related Reading
Browse all Google news →
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic NVIDIA and Google Cloud let developers scale AI from prototype to production
Market Trends
Related Topic Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks
From Archives LLM embeddings and HDBSCAN cluster text; visualized with pairwise scatterplots
From Archives AI Agents Risk Fatal Traps When Treating Context Windows as Memory
From Archives SpaceX signs compute deal with Reflection AI, founded by ex‑DeepMind researchers
Business & Startups
From Archives Nobel laureate John Jumper departs DeepMind for Anthropic after AlphaFold win
Research & Benchmarks
Common Questions Answered
What new capability does Gemini 3.5 Flash have with screen-control?
Gemini 3.5 Flash can now look at your screen across phones, browsers, and desktops, then operate it by clicking buttons and typing text. This transforms the chatbot into a potential digital employee capable of testing software or processing invoices without requiring custom integrations for each application.
How does Gemini 3.5 Flash perform on the OSWorld benchmark compared to other models?
Gemini 3.5 Flash achieved a 78.4 score on the OSWorld benchmark, positioning it just behind GPT-5.5. This demonstrates that Google's fastest model is closing the gap with other leading AI systems in cross-platform agent capabilities.
What safety measures has Google implemented for Gemini 3.5 Flash's screen-control feature?
Google has implemented layered security measures including tripwires and human checkpoints to control the screen-control functionality. These safety measures are critical because an AI with screen control represents a massive vulnerability, and Google has acknowledged this inherent danger by building these protective barriers into the system.
Why is screen-control capability significant for AI agents?
Screen-control transforms a chatbot from a conversational tool into a functional digital employee that can autonomously perform tasks like software testing and invoice processing. This eliminates the need for engineers to build custom bridges and integrations for every single application, making AI agents far more versatile and practical.
Further Reading
- Introducing computer use in Gemini 3.5 Flash — Google AI Studio
- Google DeepMind announces Gemini 3.5 Flash computer use model — Reddit (Bard Community)
- Google launches Gemini 3.5 Flash. How to try it for free. — Mashable
- Gemini 3.5: frontier intelligence with action — Google Blog
- What's new in Gemini 3.5 Flash - Interactions API — Google AI Documentation
