Confluent and Redpanda race to build agent‑ready streaming data infrastructure
When Confluent rolled out its agent-ready streaming platform last week, the headline was clear: AI agents seem to need a constant flow of fresh data if they’re going to act on their own. The service appears to push context into models in real time, something that’s been missing from most large-language-model deployments. Interestingly, Redpanda had already shown a similar idea a day earlier with its Agentic Data Plane, which likely bundles streaming pipelines, SQL queries and governance under one roof.
That move hints at a broader swing away from static warehouses toward infrastructure that can keep up with the split-second decisions AI makes. For CIOs and data engineers, the debate isn’t “if” they should stream anymore, but which stack will give them the speed, flexibility and control they need for the next wave of smart apps.
---
Redpanda isn’t just copying the idea; its Agentic Data Plane actually stitches streaming, SQL and governance together for AI agents. The timing suggests both vendors see the same pain point - models that can’t reach the data they need fast enough. As the race heats up, the real test will be which platform can deliver reliable performance without sacrificing oversight.
Competition heats up for agent-ready data infrastructure Confluent isn't alone in recognizing that AI agents need different data infrastructure. The day before Confluent's announcement, rival Redpanda introduced its own Agentic Data Plane -- combining streaming, SQL and governance specifically for AI agents. Redpanda acquired Oxla's distributed SQL engine to give agents standard SQL endpoints for querying data in motion or at rest.
The platform emphasizes MCP-aware connectivity, full observability of agent interactions and what it calls "agentic access control" with fine-grained, short-lived tokens. Confluent emphasizes stream processing with Flink to create derived datasets optimized for agents. Redpanda emphasizes federated SQL querying across disparate sources.
Both recognize agents need real-time context with governance and observability. Beyond direct streaming competitors, Databricks and Snowflake are fundamentally analytical platforms adding streaming capabilities. Their strength is complex queries over large datasets, with streaming as an enhancement.
Confluent and Redpanda invert this: Streaming is the foundation, with analytical and AI workloads built on top of data in motion. How streaming context works in practice Among the users of Confluent's system is transportation vendor Busie. The company is building a modern operating system for charter bus companies that helps them manage quotes, trips, payments and drivers in real time.
"Data streaming is what makes that possible," Louis Bookoff, Busie co-founder and CEO told VentureBeat. "Using Confluent, we move data instantly between different parts of our system instead of waiting for overnight updates or batch reports.
Real-time streams might finally shrink the timing gap for enterprise AI agents, but it’s not a slam-duck. Most corporate data still rides on batch-oriented ETL pipelines, so agents miss events as they happen. That’s where Confluent and Redpanda are stepping in, each touting a streaming-first platform that could hand agents fresh context.
Confluent just announced a new feature set; Redpanda is pushing its Agentic Data Plane, which bundles streaming, SQL and governance into one package. The hype sounds promising, yet I haven’t seen any live deployments that prove the low-latency path works with today’s AI stacks. Adding governance and reliable SQL over streams also adds a layer of complexity that could slow things down.
Competition will probably spark more ideas, but whether companies will swap out entrenched ETL workflows for these newer planes is still up in the air. For now, we have two serious contenders trying to make agent-ready data infrastructure a reality, but the actual impact remains to be seen.
Further Reading
- Comparing Redpanda and Confluent for Modern Data Pipelines - RisingWave
- Choosing the Right Kafka Platform: Condense vs Confluent vs Redpanda - Vocal Media Futurism
- Redpanda vs Confluent - Redpanda
- Apache Kafka vs. Redpanda - Confluent - Confluent
- The Data Streaming Landscape 2025 - Kai Waehner Blog
More in Open Source
Anthropic CEO Dario Amodei calls AGI a marketing term, echoing Altman's view
The AI community has grown weary of a word that shows up on every press release and investor deck.
Xiaomi launches MiMo‑V2‑Flash AI model: 150 t/s, USD 0.1‑USD 0.3 per million tokens
Xiaomi’s latest foray into generative AI arrives as an open‑source model named MiMo‑V2‑Flash, a...
OpenAI signs USD 10 bn deal to use Amazon Trainium3 chips, 4.4× faster compute
OpenAI has just inked a ten‑billion‑dollar agreement to run its models on Amazon’s newest Trainium3...
Zepto Cafe uses MCP to parse text orders, Playwright runs the clicks
Zepto Cafe has taken a modest step toward turning plain‑text requests into a full‑fledged ordering...
Larian CEO denies AI cuts, says generative AI hasn't boosted efficiency
Why does the Larian story matter now? The studio’s founder‑CEO, Swen Vincke, has been under fire...
Common Questions Answered
What does Confluent describe as “agent‑ready” streaming and why is it important for AI agents?
Confluent defines “agent‑ready” streaming as an architecture that continuously supplies contextual data to autonomous systems, enabling large‑language‑model agents to act on up‑to‑date information. This is important because without real‑time streams, agents may generate outputs that are disconnected from current events, reducing reliability.
How does Redpanda’s Agentic Data Plane support AI agents differently than traditional ETL pipelines?
Redpanda’s Agentic Data Plane combines streaming, SQL, and governance into a single plane, giving agents immediate access to data in motion or at rest via standard SQL endpoints. Unlike batch‑oriented ETL pipelines that introduce latency, this design keeps agents informed of events as they happen, closing the timing gap.
What role does Oxla’s distributed SQL engine play in Redpanda’s offering for AI agents?
Redpanda acquired Oxla’s distributed SQL engine to provide agents with a scalable, SQL‑compatible interface for querying both streaming and stored data. This engine enables “MCP‑aware” connectivity, allowing agents to interact with data across multiple compute nodes efficiently.
Why do both Confluent and Redpanda view real‑time streaming as the missing link for enterprise AI agents?
Both companies argue that most corporate data still flows through batch‑oriented ETL pipelines, leaving AI agents blind to live events. By delivering streaming‑first architectures, they aim to give agents immediate context, improving decision‑making and operational reliability.