General Compute raises $100 million to build specialized AI inference cloud

A shift toward specialized silicon

General Compute is raising $100 million in Series A funding to build a heterogeneous AI inference cloud. The startup is focusing on the 'inference' side of the industry, which is the process of using a trained AI model to generate outputs like text or images. As AI labs shift more resources from training models to serving them in production, the demand for efficient inference infrastructure is growing.

The company is prioritizing unique hardware architectures instead of just using the standard graphics processors common in the market. The founders argue that different types of chips can provide better economics and performance for specific types of inference workloads. By moving away from a one-size-fits-all approach, the company hopes to bridge the gap between advanced alternative chips and the companies that need them.

The funding round was co-led by Blackbird and Abstract, with participation from several other firms including Fuse Venture Partners and Village Global. General Compute also obtained a loan of up to $400 million from Upper90. This combination of equity and debt will support the deployment of additional capacity as the company builds a platform spanning multiple generations of AI accelerators.

Managing the complexity of hardware

One of the primary challenges for companies using alternative AI chips is the operational complexity involved in deploying them. General Compute intends to handle this complexity by purchasing the equipment and installing it in colocation facilities. They will manage the entire stack, including networking, power, cooling, software, and general operations for their customers.

The platform provides a single cloud interface where customers can access both standard GPUs and specialized inference chips. This allows users to route their workloads to the specific hardware architecture that best suits a particular task. Customers can do this without having to procure, manage, or operate the underlying infrastructure themselves. This model is designed to simplify the process for companies that want to optimize for speed or cost.

The company's intelligent workload-routing technology is a key part of this system. It dynamically selects the best hardware based on the specific needs of individual agentic tasks and other inference requests. This ensures that the most efficient chip is used for every specific request, rather than forcing all tasks onto a single type of hardware.

Performance and service options

General Compute is building its infrastructure to support models from major AI laboratories as well as open-source models. The company has already established partnerships with several hardware providers, including Cerebras, SambaNova, and Positron AI. These providers offer specialized chips that are designed to handle the increasing demand for faster token generation.

To provide reliability for its customers, the company offers different types of commercial agreements. Users can purchase compute through guaranteed tokens-per-second service-level agreements, which provide a predictable speed for AI outputs. Alternatively, customers can opt for dedicated bare-metal infrastructure if they require more direct control over their hardware environment.

The company's goal is to deliver the fastest AI possible to AI-first companies. By focusing on ultra-fast inference, they aim to help businesses manage the high costs associated with running large-scale models. The infrastructure is designed to scale as the industry moves toward more complex, agentic AI behaviors that require rapid responses.

Timeline and capacity goals

While the funding and partnerships are already in place, the company is still in the development phase of its infrastructure. General Compute has stated that it has more than 50 MW of capacity in its development pipeline. This capacity is necessary to meet the growing demand for high-performance computing in the AI space.

The company expects its first major deployments to go live in the first quarter of 2027. This timeline reflects the time needed to build out the physical facilities and integrate the various hardware types into a cohesive cloud platform. The startup is working to establish a platform that can handle multiple generations of AI accelerators simultaneously.

The move toward specialized inference is seen as a response to current bottlenecks in the market. Industry observers note that while silicon was once the primary constraint, power and supply chain speeds are now significant hurdles. General Compute aims to navigate these constraints by creating a diverse cloud that can adapt to the evolving needs of the AI industry.