Supermicro to sell SambaNova-powered AI inference systems

A new hardware partnership

Supermicro has announced a partnership with SambaNova to build and sell specialized AI inference systems. These systems are designed to support agentic AI, which refers to AI systems capable of performing complex, multi-step tasks autonomously. The partnership aims to provide a more flexible way for data centers to scale their AI capabilities.

Supermicro will be responsible for the heavy lifting of the hardware lifecycle. This includes the integration, validation, and delivery of the systems to customers globally. They will also provide on-site service to ensure the hardware remains operational in various environments.

SambaNova will provide the engineering expertise to ensure that AI models run correctly on their hardware. Their engineers will work alongside Supermicro during customer deployments to tune models for peak performance. This collaboration allows customers to purchase specialized chips through a trusted, established channel partner.

The logic of disaggregated AI

The new systems use a disaggregated inference architecture. This means the hardware is split into different components that handle different parts of the AI process. Instead of relying on a single type of chip for everything, the system pairs NVIDIA GPUs with SambaNova SN50 RDUs.

Large Language Model (LLM) inference generally happens in two distinct phases. The prefill phase processes the initial prompt and is considered compute-bound, which is the area where NVIDIA GPUs are traditionally very strong. The decode phase generates the response one token at a time and is memory-bound, which is a significant bottleneck for many standard systems.

Because agentic workloads spend the majority of their time in the decode phase, the SN50 RDUs are specifically built to accelerate this part of the process. By using the best hardware for each specific phase, the system achieves faster and more efficient results than if a customer used only one type of hardware for the entire task.

Solving the power and speed gap

One of the primary challenges for data centers is managing power density. Many high-performance AI chips require extremely high rack power densities, which can be difficult to accommodate in older facilities. The SambaNova SN50 RDUs are designed to provide high-speed, high-throughput inference without these extreme power requirements.

This characteristic allows customers to deploy these systems in existing data centers rather than having to build entirely new facilities. It provides a way to scale AI capacity without the massive infrastructure overhaul that often accompanies new hardware deployments. This is particularly useful for enterprises that need to expand their AI capabilities quickly.

The goal of this hardware mix is to provide a balance between speed and efficiency. By optimizing for the specific memory-bound nature of the decode phase, the system can handle the thousands of calls to LLMs that agentic AI requires. Even small improvements in these speeds are considered highly valuable as these agents often take minutes to complete a single task.

Targeting clouds and enterprises

The rollout of these systems will happen in specific phases. The initial batch of 10,000 SN50 chips will be distributed first to NeoClouds and sovereign clouds. Sovereign clouds are data centers located within a specific country to meet local data residency and security requirements.

Following the initial rollout, the systems will expand to enterprises that run their own private AI infrastructure. These customers often need to balance high performance with the constraints of their own data center environments. The partnership aims to make it easier for these organizations to adopt heterogeneous architectures.

SambaNova expects this partnership to lead to a significant increase in high-performance inference capacity by 2027. They believe that by making these specialized chips more accessible through Supermicro, they can enable a scale of deployment that would not be possible otherwise. The focus remains on providing a scalable solution for the evolving demands of the AI market.

What comes next for the market

The move toward disaggregated systems signals a shift in how data centers think about AI hardware. Rather than a one-size-fits-all approach, the industry is moving toward specialized components that solve specific bottlenecks. This allows for more granular control over how different parts of an AI workload are processed.

For the reader, this means that the next generation of AI agents may become more responsive and reliable. As companies move toward agents that perform complex tasks, the underlying hardware must be able to handle the specific memory demands of those tasks. This partnership is a direct response to that shift in the AI landscape.

Supermicro's involvement ensures that these complex systems are treated as standard products. Customers can order them the same way they purchase other infrastructure components, removing the friction of dealing with niche hardware. This move toward standardization is a key step in making specialized AI acceleration a mainstream reality for large-scale operations.