OpenAI engineers discussing cost-efficient improvements for ChatGPT inference, reducing expenses by half for guest users in a

Editorial illustration for OpenAI engineers say they halved inference costs for guest ChatGPT users

OpenAI engineers say they halved inference costs for...

OpenAI engineers say they halved inference costs for guest ChatGPT users

By AI Daily Post Edited by Brian Petersen, Editor-in-Chief

June 30, 2026 • 2 min read

OpenAI engineers told colleagues earlier this month they’d sliced inference costs for guest ChatGPT users by more than half, according to a source familiar with the discussions and reported by The Information. While the exact techniques remain undisclosed, the optimization shrank the Nvidia GPU pool needed for anonymous visitors to “just a few hundred.” It isn’t clear how many GPUs were required before, nor whether the same savings will apply when the full‑feature set is used. Guest users, after all, can only tap a very limited slice of ChatGPT’s capabilities.

Meanwhile, Deepseek released an open‑source method that claims a 60‑to‑85 percent speed‑up on inference requests. The extra capacity could be redirected toward scaling services, improving models, shortening response times, or boosting margins. Yet, with data‑center construction progressing slowly, such efficiency gains are likely to give labs a bit more breathing room rather than dramatically curbing chip demand. The question now is how far these cuts will stretch across OpenAI’s broader product stack.

OpenAI reportedly cut response costs for guest ChatGPT users by more than half OpenAI engineers told colleagues earlier this month that they'd managed to cut inference costs--the expense of running existing AI models--by more than half. That's according to a person familiar with the discussions, as reported by The Information. OpenAI applied the new optimizations to ChatGPT, specifically for visitors who don't have an account.

The number of Nvidia GPUs needed to serve those users dropped to just a few hundred. It's not clear how many were required before or what techniques OpenAI used to pull it off. Guest users can only access a very limited set of ChatGPT features, so whether these gains would carry over to the full product is an open question.

OpenAI reportedly cut response costs for guest ChatGPT users by more than half - THE DECODER

Why this matters We see a concrete shift in OpenAI’s operating economics: engineers have reportedly halved the inference cost for guest ChatGPT sessions, slashing the Nvidia GPU count required to serve anonymous traffic. Costs dropped sharply. For developers watching cloud bills, that figure suggests a tangible efficiency gain, though the article does not disclose the exact methods or whether the savings apply to paid accounts.

If the same techniques scale, founders might reconsider the cost structure of free‑tier services, yet it is unclear whether OpenAI will pass any of the reduction on to end users. Researchers can note that a reduction of this magnitude could free up compute for experimentation, but the lack of technical detail leaves open questions about trade‑offs in latency or model quality. We remain cautious, because cost cuts alone do not guarantee broader access or lower prices.

Still, the reported drop in GPU demand signals that incremental engineering can move the needle on affordability, a point worth tracking as we evaluate the sustainability of large‑scale AI deployments.