NVIDIA architectures boost AI per‑watt efficiency with...

Why does this matter? Power can gobble up 40 % of an AI factory’s operating expenses, turning every watt into a cost decision—overhead, data ingestion, training, or the tokens sold to customers. Most sites sit under a fixed power ceiling set by regional providers, so performance per watt isn’t just a metric; it’s a bottom‑line lever.

While inference fuels revenue, boosting inference throughput per watt directly lifts the number of tokens an operator can sell, translating into extra profit per hour. At scales ranging from a hundred megawatts to a gigawatt, even a modest few‑percentage‑point gain per megawatt can mean meaningful earnings. NVIDIA claims the lowest cost per token for inference and the cheapest path to train large models, a claim rooted in extreme co‑design with power, cooling and system infrastructure and deep ties to OEMs, ODMs, CSPs, NCPs, system integrators, ISVs and model‑ecosystem partners.

Model architecture matters, too—Mixture‑of‑Experts designs often out‑perform dense models because only a subset of experts activates per token, as seen with the large‑parameter DeepSeek‑R1. This post walks through the levers operators can pull to squeeze more performance out of every watt.