Editorial illustration for Ex-Google Founder Ross Launches Groq to Challenge NVIDIA with New Inference Chip
Groq Challenges NVIDIA with Revolutionary AI Inference Chip
Groq, founded by ex-Google exec Ross, to assist NVIDIA on inference chips
The artificial intelligence chip market just got more interesting. Jonathan Ross, a former Google engineering leader, is betting he can outmaneuver industry giant NVIDIA with a radical new approach to processing AI workloads.
His startup, Groq, isn't playing it safe. The company has developed a specialized chip architecture called the Language Processing Unit (LPU) that promises to shake up how AI models perform complex computational tasks.
Silicon Valley has long watched for potential NVIDIA challengers, but most competitors have fallen short. Ross brings serious credibility, having been a key engineer at Google with deep expertise in chip design and machine learning infrastructure.
Groq's strategy isn't about competing on every front. Instead, the company is targeting specific performance niches where traditional graphics processing units struggle to deliver consistent, low-latency results.
The stakes are high in a market where computational speed can make or break AI model effectiveness. Ross believes his LPU design could be the breakthrough that changes how we think about AI inference.
After leaving Google, Ross founded Groq to build the Language Processing Unit (LPU), a chip architecture designed for deterministic, low-latency inference. Since its launch, Groq has positioned its systems as delivering higher inference speeds for specific models than NVIDIA's GPUs on certain AI models. The deal has triggered speculation across the industry about NVIDIA's motives.
"What this also says to me is that Nvidia sensed a threat to scaling their own inference business," wrote Naveen Rao, CEO of Unconventional AI and former VP of AI at Databricks, in a post on X. Max Weinbach, an analyst at Creative Strategies, suggested in a post on X that the agreement could help NVIDIA rethink its inference roadmap. "This gets Nvidia the IP they need to bypass CoWoS and HBM for a fast inference-focused chip, and use NVLink for better chip-to-chip interconnect of the LPU," he wrote.
This indicates NVIDIA may be looking to absorb ideas from Groq's LPU architecture to design inference-optimised chips that rely less on costly advanced packaging and memory stacks, while still leveraging its NVLink ecosystem. This would strengthen its position in low-latency, high-throughput AI inference without requiring a complete acquisition of Groq. In September, Groq raised a $750 million funding round at a valuation of $6.9 billion, underscoring investor confidence in its approach to inference-focused hardware.
The AI chip landscape is heating up, with Groq emerging as a potential challenger to NVIDIA's dominance. Founded by ex-Google executive Jonathan Ross, the company's Language Processing Unit (LPU) represents a targeted approach to inference technology.
Groq's chip architecture aims to deliver higher inference speeds for specific AI models, positioning itself as a nimble alternative to traditional GPU solutions. The startup's emergence suggests growing competition in a market previously controlled by established players.
Industry observers like Naveen Rao are already speculating about NVIDIA's strategic response. The speculation centers on whether NVIDIA perceives Groq as a genuine threat to its inference business model.
While it's too early to declare a winner, Groq's unique approach, focusing on deterministic, low-latency inference, hints at potential ideas in AI computing. The chip's performance on specific models could signal a meaningful shift in how inference technology is designed and deployed.
Ross's background at Google lends credibility to Groq's technical ambitions. But the real test will be how the LPU performs across a broader range of AI workloads.
Further Reading
Related Reading
Browse all Google news →
Business & Startups OpenAI, a Series F San Francisco startup founded in 2015 by eight pioneers
Business & Startups Terminal-Bench 2.0 launches with Harbor, testing any container-installable agent
Business & Startups Zuckerberg Unveils Meta Compute to Build Global AI Infrastructure
Related Topic Google AI Advisors Let Users Probe Performance with Conversational “Why” Queries
Research & Benchmarks
Related Topic Gemini 3 Pro builds screenshot-to-code app in two prompts, fixes bugs
LLMs & Generative AI
From Archives OpenAI says prompt injection persist, ships adversarial model and safeguards
From Archives Coforge launches EvolveOps.AI, AI-agent platform to curb reactive IT ops
From Archives AlphaFold marks 5 years; new Gemini 2.0 AI co-scientist debates hypotheses
Open Source
From Archives Google highlights AI-driven chip, infrastructure and robotics advances in 2025
Research & Benchmarks
Common Questions Answered
What makes Groq's Language Processing Unit (LPU) different from traditional GPU architectures?
Groq's LPU is designed specifically for deterministic, low-latency AI inference, offering a specialized approach that differs from NVIDIA's general-purpose GPUs. The chip architecture aims to deliver higher inference speeds for specific AI models, potentially challenging the current GPU-dominated landscape.
Who founded Groq and what is his background in AI chip technology?
Groq was founded by Jonathan Ross, a former Google engineering leader with extensive experience in chip design and AI technologies. After leaving Google, Ross launched Groq with the vision of creating a more efficient chip architecture for AI computational tasks.
How is Groq positioning itself as a potential challenger to NVIDIA in the AI chip market?
Groq is challenging NVIDIA by developing the Language Processing Unit (LPU), which promises higher inference speeds for specific AI models compared to traditional GPU solutions. The startup is targeting a niche in the AI chip market by offering a more specialized and potentially more efficient chip architecture.