AI companies face high costs and compute shortages for personal agents

The high price of proactive AI

The AI industry is currently racing to develop personal agents that can proactively complete tasks for users. These assistants aim to go beyond simple conversation by accessing accounts and services to book tickets, manage emails, and organize daily schedules. However, the success of these tools depends on a massive amount of computing power, which is currently proving difficult to scale.

Elon Musk recently highlighted the financial strain of this transition. He noted that the costs of running Grok Bot are "unsustainably high relative to revenue," suggesting that the technology is currently too expensive to maintain at scale. To manage these costs, he indicated that the company has been throttling back the amount of compute allocated to the bot.

This trend is reflected in the user experience as well. When companies limit compute to save money, users may find that they hit their usage limits much faster. This creates a tension between the desire to provide powerful, always-available assistants and the reality of the high costs required to keep them running.

OpenAI's efficiency shift

OpenAI is taking a different approach to solve the cost problem for its Dots assistants. Adam Fry, the head of ChatGPT’s consumer business, admitted that Dots are currently "on the compute heavy side" because each assistant utilizes its own cloud computer. This setup allows the AI to execute complex tasks on a user's behalf by interacting with various third-party services.

To address this, OpenAI is introducing a new system to route different tasks to different underlying models. Instead of using the large GPT-6 Astra model for every request, the system will automatically direct simpler tasks to the smaller GPT-6.1 Terra model. This change is designed to reduce the overall intensity of the compute required for each interaction.

The goal of this routing is to make the technology more viable for a wider audience. Currently, Dots are limited to users with a Pro subscription, which costs $100 per month. By using smaller models for routine tasks, OpenAI hopes to achieve a breakthrough in efficiency that will allow them to bring these assistants to the masses while keeping costs low.

Infrastructure and funding hurdles

Securing enough hardware and power to run these models is becoming a major risk for the leading AI labs. Anthropic's recent filings reveal that the company has entered into purchase agreements totaling $895.4 billion, almost all of which is dedicated to compute and infrastructure. The company even listed the inability to secure more compute as a major risk factor for its future success.

OpenAI faces its own unique infrastructure challenges, as its fleet is largely dependent on Nvidia GPUs. To reduce this dependency, the company is working on an in-house chip called Jalapeno, which is expected to scale up production in 2027. These hardware limitations are compounded by a lack of profitability, as OpenAI's latest funding round fell short of its $30 billion target.

The financial pressure may eventually force these companies to go public to fund the necessary expansion. Without massive amounts of capital, the labs may struggle to build the infrastructure needed to support a boom in personal AI agents. This makes the transition from research to a mass-market product a significant financial gamble for the industry.

Navigating public pushback

Beyond the technical and financial hurdles, AI companies are facing a nationwide, bipartisan backlash against the construction of new data centers. Communities are increasingly concerned about the impact these facilities have on local electrical grids and the environment. This pushback could slow down the permitting process or lead to the cancellation of planned facilities entirely.

In response to this pressure, OpenAI has begun rebranding its data centers as "compute campuses." This new terminology is intended to frame the facilities in a more positive light by emphasizing community benefits. The company has promised that these campuses will work to reduce strain on local grids and avoid taking power away from local families.

The success of the AI agent boom will ultimately depend on how well these companies navigate these social and regulatory issues. If the public does not see clear benefits in the form of jobs or environmental protection, the physical infrastructure required for AI may become harder to build. This adds a layer of geopolitical and social complexity to the technical challenge of scaling compute.