SI Agent News
DevelopingChips

Axelera AI unveils Titania for large-scale AI inference

The new architecture aims to run 1.6-trillion-parameter models on a single air-cooled server, potentially lowering costs for data center operators.

Blueprint drawing: picture for the story 'Axelera AI unveils Titania for large-scale AI inference'
AI-made blueprint drawing for this story. No real people or products are shown.
1.6-trillionMaximum parameter model size

The short version

Why it matters

This hardware targets a major bottleneck in AI: the ability to run massive models without requiring specialized, expensive liquid cooling or massive power upgrades. If successful, it could make high-end conversational AI much cheaper and easier to deploy in existing facilities.

A new architecture for scale

Dutch semiconductor company Axelera AI recently revealed Titania, its third-generation architecture built for large-scale AI inference. The company also introduced the AX8, a 4U air-cooled reference server that serves as a practical implementation of this new technology. This hardware is specifically designed to let operators run large conversational AI models while utilizing existing data center power and cooling infrastructure.

The system is built around a chiplet design, which combines multiple semiconductor dies to create a more flexible and scalable architecture. Specifically, Titania combines two chiplets and 128GB of LPDDR6 memory into a single System in Package (SiP). The AX8 server carries eight of these packages, providing a total of 1TB of accelerator memory.

According to XenoSpectrum, the AX8 is built to reach a combined memory bandwidth of 19.6TB/s while providing 20 PFLOPS of FP8 compute. By focusing on these specific metrics, Axelera AI intends to provide operators with the concrete data needed to judge if a large model can actually run within their current facility constraints.

Memory and compute strategy

One of the most significant technical choices in the Titania design is the use of LPDDR6 memory instead of high-bandwidth memory (HBM). Axelera AI stated that this choice was made to reduce memory power consumption and to lessen the company's dependence on HBM supply chains and advanced packaging technology. This approach prioritizes securing large memory capacity to hold the weights of massive models while maintaining efficient data movement.

The architecture utilizes digital in-memory computing, a method that performs calculations directly within the memory. By placing many small compute cores and optimizing how data moves, the design aims to run inference efficiently. This is particularly important for Mixture of Experts (MoE) architectures, which require keeping a large number of weights in memory while only reading out the specific parts needed for each input.

For a 1.6-trillion-parameter model at FP4 precision, the weights alone would require roughly 800GB of capacity. The 1TB of memory in the AX8 provides the remaining space for the KV cache and other working memory. However, the actual capacity needed for these tasks varies based on the model's structure, the number of simultaneous users, and the length of the input context.

Infrastructure and efficiency

Axelera AI is positioning the AX8 as a standard 4U air-cooled server with a typical power consumption of 5–6kW. This design is intended to be easier to fit into existing data centers compared to systems requiring specialized cooling. For larger deployments, the company envisions two AX8 units in a 15kW rack or five units in a 30kW air-cooled rack.

The company estimates that Titania could increase throughput per watt by a factor of 7 and drop the total cost of ownership (TCO) by up to 20x when compared to current top-tier data center servers. TCO includes not just the hardware purchase price, but also the ongoing costs for power, cooling, and maintenance. While these are projections, they highlight the company's goal of making large-scale AI more economically viable.

To support diverse needs, Axelera AI plans to provide both air-cooled and liquid-cooled inference servers through its OEM partners. The air-cooled option is specifically aimed at customers who want to avoid infrastructure upgrades. However, the actual amount of upgrading avoided will depend on the specific power supply capacity and cooling equipment of each individual customer.

Software and multi-modal goals

To ensure a smooth transition for existing customers, Axelera AI is maintaining software compatibility across its product generations. Developers can use the same Voyager SDK and AxeleraScript environments for the Metis, Europa, and Titania architectures. The company is also designing a Titania PCIe card to replace the Axeleara Server 250p, allowing models developed on Europa to move to Titania in 2028 without being re-ported.

The Titania architecture will support three types of parallelism: tensor, expert, and pipeline parallelism. These methods allow the system to distribute computation across multiple units or divide different parts of an MoE model. Additionally, the hardware will support optimizations like speculative decoding, which speeds up output by predicting tokens ahead, and KV cache offloading to manage memory usage more flexibly.

Beyond language models, Titania is designed to handle multimodal AI tasks, including 8K video decoding and 4K video encoding. This reflects a broader vision of running real-time AI processing that handles language, audio, and images simultaneously. By integrating these capabilities, the system aims to support a wide range of live video processing and multimodal applications in a single unit.

Timeline and next steps

The development timeline for Titania has become more concrete following its initial announcement in early 2025. Mass production is now scheduled for the third quarter of 2027, with the goal of rolling out the hardware to customers by early 2028. Axelera AI has already reported that customer commitments for these new systems exceed $300 million.

The company plans to provide a detailed technical presentation on Titania at the SC26 supercomputing event in Chicago. This event will offer a deeper look at the hardware's capabilities before it hits the market. For potential adopters, it will be important to verify the performance under specific conditions, such as different context lengths and numbers of simultaneous users.

As the launch approaches, the industry will be watching to see if Titania can deliver on its efficiency projections. If the hardware provides the promised memory capacity and power efficiency while maintaining fast response speeds, it could become a primary option for running large-scale AI inference. The success of the Titania architecture will depend on its ability to balance these complex requirements in a standard data center environment.

What to watch

Try it

Check your data center's current rack power capacity to see if a 15kW or 30kW air-cooled configuration would fit your existing infrastructure.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by XenoSpectrum · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news