Editorial illustration for Chen says AI quality requires ongoing experimentation, iteration and improvement
AI Sycophancy: Why Models Change Answers on Demand
Chen says AI quality requires ongoing experimentation, iteration and improvement
Everyone wants an AI that’s done, finished, ready to ship. Chen's team knows that’s a fantasy.
They treat quality like a daily workout, not a trophy you win. His people break down the vague idea of "usefulness" with over half a dozen specific gauges. They check for authority and citation accuracy.
They measure hallucination rates. But the real challenge is a metric they call comprehensiveness. It asks a simple, brutal question: did the AI actually address everything the user asked?
In law, a factually correct but incomplete answer isn't just wrong. It's a liability waiting to happen.
"At the end of the day, what matters most for us is the quality of the AI outcome, and that is a continuous journey of experimentation, iteration and improvement," Chen said. Getting 'complete' answers to multi-faceted questions To evaluate models and their outputs, Chen's team has established more than a half-dozen "sub metrics" to measure "usefulness" based on several factors -- authority, citation accuracy, hallucination rates -- as well as "comprehensiveness." This particular metric is designed to evaluate whether a gen AI response fully addressed all aspects of a users' legal questions.
This approach accepts a fundamental truth. You cannot perfect a system that faces new, unpredictable questions every second. The goalposts move.
So the real product isn't a perfect model. It's the grinding, unglamorous process of finding flaws and fixing them, again and again. Any company buying the promise of a finished, flawless AI is buying a story.
The ones building something that lasts are buying into the slog.
Related Reading
Policy & Regulation Grammarly faces class-action suit over AI ‘Expert Review’ feature
Policy & Regulation Claude Mythos highlights EU AI safety gaps, says researcher Caroli
Policy & Regulation After ditching AI fitness apps and a Fitbit, I return to Peloton classes
From Archives Moltworker isolates OpenClaw tests with containers encrypted storage, Zero Trust
From Archives India's new IT rules give Instagram and X an unworkable deepfake deadline
Common Questions Answered
What is the FActScore methodology for evaluating language model factuality?
[arxiv.org](https://arxiv.org/abs/2305.14251) introduces FActScore as a novel evaluation technique that breaks generated text into atomic facts and calculates the percentage of facts supported by a reliable knowledge source. The method allows for a more nuanced assessment of factuality beyond binary quality judgments, revealing that models like ChatGPT only achieve around 58% factual precision in biographies.
How does FActScore address the challenge of evaluating long-form text generation?
FActScore addresses the complexity of evaluating long-form text by breaking generations into individual atomic facts and systematically checking each fact's support from a reliable knowledge base. The researchers developed both a human evaluation method and an automated model that can estimate the factuality score with less than a 2% error rate, making it possible to evaluate large numbers of generations that would be prohibitively expensive to assess manually.
What key insights did the FActScore research reveal about different language models?
The research evaluated biographies generated by several state-of-the-art commercial language models, finding significant variations in factuality across different systems. Notably, the study revealed that GPT-4 and ChatGPT are more factual than public models, while Vicuna and Alpaca emerged as some of the best-performing public models in terms of factual precision.
Further Reading
- Beyond the Lab: The 2026 Roadmap to AI-Powered Business Experimentation — Trulata
- Why data quality is key to AI success in 2026 — Strategy
- AI in 2026: from experimentation to enterprise expectation — Horizontal Talent
- Big Predictions for 2026: From AI Experiments to AI Operations — Productive Edge
- AI in 2026: The Shift From Experiments to Production Infrastructure — TFIR
