Amazon's $5 billion Nvidia AI servers sit idle due to power shortages

Hardware stuck in warehouses

Amazon is currently facing a logistical hurdle where billions of dollars in high-end hardware are sitting unused. According to a report by Business Insider, the company is storing new Nvidia AI server racks in intermediate warehouses rather than installing them in active data centers.

Internal emails revealed that Amazon anticipated taking delivery of $5 billion in Nvidia GB300 server systems at U.S. locations during August. Following that initial shipment, the company projected receiving an additional $3 billion in racks every month for the foreseeable future.

While some Amazon Web Services (AWS) sites have the physical room to house these new racks, they do not have the electricity required to power them. This has created a situation where the equipment is physically present in the country but remains offline and inaccessible to customers.

The problem of dark GPUs

This situation has contributed to a trend known as "dark GPUs." This term describes a scenario where companies purchase expensive AI chips and servers but are unable to turn them on because of power shortages or other infrastructure challenges. It represents a significant gap between the ability to manufacture hardware and the ability to power it.

The lack of new electricity capacity is one of the primary bottlenecks slowing down the development of modern AI data centers. Because the power grid cannot keep up with the massive energy demands of large-scale AI training, companies are forced to wait for infrastructure upgrades before they can utilize their purchases.

One internal email noted that at least one AWS team did not have a specific timeline for moving the incoming racks into data centers at the time of the report. This suggests that the coordination between hardware procurement and electrical infrastructure planning is currently struggling to keep pace with demand.

Financial pressure on AWS

The delay in deploying these servers is creating significant financial pressure for Amazon. The company is required to pay for Nvidia GPUs and other high-value components before they can be used, but that capital remains tied up while the equipment sits in a warehouse. This creates a period of high expenditure without the corresponding revenue from active AI services.

The financial strain is visible in Amazon's recent performance metrics. The company's free cash flow dropped to a loss of $7.6 billion for the 12-month period ending June 30, a sharp decline from a positive $18.2 billion during the previous year. AWS infrastructure chief Prashant Kalyanaraman described the AI business as "hugely free cash flow negative" and expects this trend to continue for a few years.

While debt financing can help bridge the gap during these long construction periods, the lending environment is changing. Lenders have recently become less willing to finance some of these large-scale data center projects, making it harder for companies to fund the transition to more powerful AI infrastructure.

Scaling the AI infrastructure

Amazon is currently planning to spend $220 billion on capital investments across the entire company this year. A large portion of this spending is directed toward the infrastructure needed to support the next generation of AI models and services. However, the current power shortage serves as a reminder that software and chip progress are still dependent on physical utilities.

The company's "Early Rack Receipt" process was designed to get hardware into the country quickly, but it has inadvertently highlighted the electricity bottleneck. Moving forward, the speed of AI development will likely be dictated by how quickly utility companies and data center operators can expand power capacity.

For the reader, this means that the next phase of AI growth may be slower than the hardware manufacturing cycle suggests. Even as Nvidia continues to produce more chips, the actual availability of those chips to run applications depends on the availability of the power grid to support them.