Optima AI Benchmark: Test Models With Your Data

Artificial Analysis, the research group behind independent LLM evaluations and benchmarks like GDPval-AA and AA-Briefcase, has launched a new platform called Optima. The pitch is straightforward: standard AI benchmarks measure models against fixed tasks that rarely match what a company actually needs done. A model that scores well on a public leaderboard might still be the wrong pick for, say, processing insurance claims or summarizing legal contracts. Optima lets users skip that mismatch entirely by building benchmarks from their own data and workflows.

Instead of relying on someone else's test set, users feed Optima their own inputs, whether that's raw data, a description of a task, or sample outputs showing what "good" looks like. The platform then runs current leading models against that material and reports back on quality, cost per task, and time per task. That last part matters for anyone weighing a cheaper, slower model against a pricier, faster one for a specific job. Optima is live now, and Artificial Analysis frames it as a way to answer a question generic benchmarks can't: which model actually works best for this particular problem.