Editorial illustration for AgentNLQ released as a general‑purpose NL2SQL agent; accuracy lags human writers
AgentNLQ released as a general‑purpose NL2SQL agent;...
Anyone who's asked an AI to write a database query knows the drill. You type a question in plain English. You get back a perfect-looking chunk of SQL.
Then it crashes. The gap between a convincing draft and a query that actually works—and answers what you meant—remains wide. New research confronts this exact headache.
A system called AgentNLQ now scores 78.1% semantic accuracy on the notoriously tough BIRD benchmark. That's solid. It's also a clear admission: machines are still chasing any competent human coder.
Despite the rapid advancements in the capabilities of LLMs, NL2SQL has not reached parity in accuracy with human expert SQL writers, hence needing additional improvements in NL2SQL algorithms.
That 78.1% figure tells two stories. It’s a genuine leap forward from previous attempts. It also means more than one in five queries still fails semantically on BIRD.
Every missed point in that score represents a concrete breakdown—a botched join, a neglected business rule, a question fundamentally misread. The method is shrewd: fleshing out bare schemas with context, weaving in business logic, using an orchestrator that plans and corrects itself. It shows promise across domains.
But the remaining gap to human performance isn't just about smarter orchestration. It's about grasping the unstated intent behind a question. The tools are getting sharper.
The finish line, though, is still over the horizon.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives SuperAI Conference Highlights Growing AI Startup and Infrastructure Scene in Asia
From Archives CODEX Agent Adds AI‑Q Deep Research Skill from GitHub Repository
Common Questions Answered
What accuracy score did AgentNLQ achieve on the BIRD benchmark?
AgentNLQ achieved a semantic accuracy score of 78.1% on the BIRD benchmark, which is considered a notoriously tough evaluation for NL2SQL systems. This represents a genuine leap forward from previous attempts at converting natural language to SQL queries.
What does the 78.1% semantic accuracy score reveal about AgentNLQ's limitations?
The 78.1% score means that more than one in five queries still fail semantically on the BIRD benchmark. Each missed point represents concrete breakdowns such as botched joins, neglected business rules, or questions that are fundamentally misread by the system.
What methods does AgentNLQ use to improve NL2SQL conversion performance?
AgentNLQ uses several key techniques including fleshing out bare schemas with context, weaving in business logic, and employing an orchestrator that plans and corrects itself. These methods work together to improve the system's ability to accurately convert natural language questions into working SQL queries.
Why do AI-generated SQL queries often fail despite looking correct?
The gap between a convincing draft SQL query and one that actually works remains wide because AI systems frequently misunderstand the intended meaning behind natural language questions. This can result in queries that appear syntactically correct but fail to answer what the user actually meant or produce semantic errors.
What is the main challenge that AgentNLQ addresses in the NL2SQL problem space?
AgentNLQ directly confronts the challenge of converting plain English questions into SQL queries that both execute correctly and accurately answer the user's intended question. The system demonstrates promise across different domains, though it still falls short of matching the reliability of human-written queries.
Further Reading
- NL2SQL Agent – An MCP-Powered Natural Language Insights for Oracle Cloud Infrastructure — Oracle Cloud Infrastructure Blog
- Evaluate a Text-to-SQL Agent — Ragas Docs
- Optimizing AlloyDB AI text-to-SQL accuracy — Google Cloud Blog
- Natural Language to SQL: The Complete 2026 Guide — BlazeSQL AI Blog
- Building an AI Agent for Natural Language to SQL Query Translation and Execution — YouTube
