Editorial illustration for Pope's AI Advisor Warns of 'Cartel' Behavior Among Big Labs
Pope's AI Advisor Warns of Big Lab 'Cartel'
Father Paolo Benanti spends his days doing something unusual for a Franciscan friar: reading AI safety papers and briefing Vatican officials on what they mean. He advises Pope Francis on artificial intelligence policy, and he's been watching the recent wave of doomsday warnings coming out of Silicon Valley with a skeptical eye. Anthropic CEO Dario Amodei recently called for mandatory third-party evaluations of new models and restrictions on distillation, the practice of training AI on a rival's outputs. Other researchers have pushed for banning recursive self-improvement altogether, worried that AI systems writing their own successors could spiral out of control fast.
Benanti isn't convinced these are the right questions. To him, the industry's fixation on superintelligence and existential risk functions as a kind of smokescreen, one that conveniently places the biggest AI labs at the center of every decision about how to fix the problems they created. He thinks that setup needs to change, and that the public square, not corporate boardrooms, should decide how AI gets built and constrained. WIRED spoke with him about what real oversight would actually look like.
“The narrative of the frontier labs is self-interested, to gain or maintain a position in the market,” says Benanti. When a small group of companies dictates the rules under which an industry operates, he says, “this has a really clear name: a cartel.”
Why this matters
Benanti's "cartel" framing deserves attention precisely because it cuts against the industry's preferred story. Labs like to say they're locked in an existential race toward superintelligence, one so urgent that safety concerns become secondary to speed. But a handful of companies setting the pace of that race, controlling the compute, the talent, and the narrative about what counts as progress, looks less like inevitable technological destiny and more like market consolidation with better PR.
For developers and founders building on top of these systems, that distinction matters: dependency on a small set of labs means their incentives, pricing, and roadmap decisions shape what you can build, regardless of whether AGI arrives next year or next decade. Researchers should take note too. If the "race to superintelligence" doubles as justification for reduced competition and less outside scrutiny, that's worth naming plainly.
Benanti's position gives him no stake in Silicon Valley's fundraising cycles, which makes his skepticism about who benefits from the panic worth sitting with.
Related Reading
Research & Benchmarks Microsoft AI Chief: "If It Isn't Safe We Shouldn't Build It
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives Singapore Leads in Daily AI Use, With High Hopes for Life Improvement
From Archives Meta's AI Agent Muse Draws 500,000 Users in First Week
Common Questions Answered
What does Father Paolo Benanti mean when he compares big AI labs to a 'cartel'?
Benanti argues that when a small group of companies dictates the rules under which an entire industry operates, it constitutes cartel behavior. He believes the frontier labs are using self-interested narratives about an existential AI race to maintain their market position while controlling compute resources, talent, and the definition of technological progress.
How does Father Benanti view the doomsday warnings coming from Silicon Valley AI companies?
Benanti approaches these warnings with skepticism, viewing them as part of a self-interested narrative designed to gain or maintain market position rather than genuine safety concerns. He suggests that the urgency framing around superintelligence and existential risk is used to justify prioritizing speed over safety considerations.
What is Father Paolo Benanti's role at the Vatican regarding artificial intelligence?
Benanti serves as an AI advisor to Pope Francis, spending his time reading AI safety papers and briefing Vatican officials on their implications for policy. His unique position as a Franciscan friar advising on AI policy gives him a distinctive perspective on the industry's practices and narratives.
What specific AI practices does Benanti's cartel critique challenge according to the article?
The article references Anthropic CEO Dario Amodei's calls for mandatory third-party evaluations of new models and restrictions on distillation, the practice of training AI on a rival's models. Benanti's cartel framing suggests these proposals serve the interests of dominant companies rather than genuine safety or fair competition.
Why does the article suggest Benanti's 'cartel' framing is significant for understanding the AI industry?
Benanti's framing cuts against the industry's preferred narrative that labs are engaged in an inevitable technological race toward superintelligence. By reframing the situation as market consolidation rather than technological destiny, it challenges the justification that speed must override safety concerns and highlights how a handful of companies control the pace, resources, and narrative of AI development.
Further Reading
- God and EU: Vatican broadly supports AI Act - EU Perspectives
- AI giants pursue self-regulation as safety fears mount - TechXplore
- Is Big Tech's AI slowdown a safety pact or a cartel? - The Verge
- AI giants 'want cartel-like policy', former OpenAI exec says - iTnews / ACS
- Move Slow and Collude: The Antitrust Problem With Pacing AI - Truth on the Market
