Open LLM v2, 12‑benchmark suite, LiveBench show d_eff...

Why does benchmark coverage matter for massive language models? The authors argue we’ve been looking at the wrong slice of performance. They propose a stereological theory that treats any evaluation suite as a geometric object with an effective dimensionality \(d_{\text{eff}}\) ranging from 2.86 to 4.80. While the math is dense, the key claim is that the visible Hausdorff distance between two convex capability profiles that share the same scores is bounded by \(\epsilon + C R m^{-1/(d_{\text{eff}}-1)}\), and that a matching Lipschitz lower bound holds.

Here’s the thing: a submodular greedy algorithm—backed by the Nemhauser \((1-1/e)\) guarantee—identifies a stable core of just four benchmarks. Seven out of twelve tests already capture 90 % of the suite’s coverage, and that reduced set retains 93‑97 % of its predictive power across quarterly updates. A counterfactual check across twelve internal benchmarks and twenty‑seven Chatbot Arena categories shows the eigenstructure can flag irreplaceable evaluations (ρ = ‑0.69, p = 0.013) and highlight external tests that add new information (ρ = +0.38).

Beyond the benchmark angle, the paper settles Gardner’s Problem 1.5 (1995) for \(C^2\) support functions, delivering a minimax rate \(\Theta\!\big(R/(\kappa m^{2/(D-1)})\big)\) via optimal recovery theory on the sphere \(S^{(D-1)}\).