Groq, founded by ex‑Google exec Ross, to assist NVIDIA on inference chips

After leaving Google, Ross founded Groq to build the Language Processing Unit (LPU), a chip architecture designed for deterministic, low-latency inference. Since its launch, Groq has positioned its systems as delivering higher inference speeds for specific models than NVIDIA's GPUs on certain AI models. The deal has triggered speculation across the industry about NVIDIA's motives.

"What this also says to me is that Nvidia sensed a threat to scaling their own inference business," wrote Naveen Rao, CEO of Unconventional AI and former VP of AI at Databricks, in a post on X. Max Weinbach, an analyst at Creative Strategies, suggested in a post on X that the agreement could help NVIDIA rethink its inference roadmap. "This gets Nvidia the IP they need to bypass CoWoS and HBM for a fast inference-focused chip, and use NVLink for better chip-to-chip interconnect of the LPU," he wrote.

This indicates NVIDIA may be looking to absorb ideas from Groq's LPU architecture to design inference-optimised chips that rely less on costly advanced packaging and memory stacks, while still leveraging its NVLink ecosystem. This would strengthen its position in low-latency, high-throughput AI inference without requiring a complete acquisition of Groq. In September, Groq raised a $750 million funding round at a valuation of $6.9 billion, underscoring investor confidence in its approach to inference-focused hardware.