Cohere adds rival open models to its managed inference platform

Expanding the model menu

Cohere recently announced that it is opening its Model Vault to include models created by other developers. This platform serves as a single-tenant inference environment, which means it provides a dedicated space for customers to run AI models. By adding these external options, Cohere aims to give enterprises more flexibility in how they deploy AI.

The new list of available models includes several well-known names from the open-source community. Specifically, the platform now hosts models from Google, Microsoft, NVIDIA, and several other developers. This means customers can run models like Google's Gemma or Microsoft's Phi alongside Cohere's own offerings.

According to our arithmetic, 8 of the 14 model families currently in the vault are not owned by Cohere. This move allows businesses to test and use different architectures without having to build out their own separate infrastructure for each one. It positions Cohere as a host for a variety of leading open models.

Security and encryption tiers

A major part of the Model Vault offering is the distinction between standard and encrypted tiers. The encrypted version uses confidential computing to ensure that prompts and outputs stay private. Cohere states that in this specific tier, the company cannot see the data sent to a model or the responses it provides.

However, not all of the newly added models are available in this high-security encrypted tier. While models from Google, Microsoft, and OpenAI are included in the encrypted vault, four models from China-based developers are not. Cohere has not provided a specific reason why these four were excluded from the encrypted offering at this time.

The company also notes that neither prompts nor outputs are retained by default on the platform. This focus on data privacy is intended to appeal to large enterprises that have strict requirements regarding how their data is handled by third-party AI providers.

Pricing and regional availability

Customers using the Model Vault can choose where their workloads run based on specific regional requirements. The platform is currently live in the United States, Canada, and the European Union. However, the encrypted tier is currently only available in the United States, with the other regions coming soon.

Pricing for the platform is determined by the type of GPU used for the inference tasks. For example, an H100 GPU costs $6.25 per hour on demand in the standard tier, but this price rises to $9.00 in the encrypted tier. This represents a 44 per cent increase in cost for the added security features.

To encourage adoption, Cohere is offering promotional prices through March 31, 2027. During this promotional period, the price for an H100 is $4.25 per hour for both the standard and encrypted tiers. This promotion effectively removes the premium price for the encrypted tier on both the H100 and the B200 chips.

The move toward model agnosticism

Cohere is not the first company to offer third-party models on its own infrastructure. Mistral, a European AI lab, made a similar move in August by adding models from Z.ai to its platform. Both companies are following a strategy of providing a managed, regional, and SLA-backed service for a variety of different models.

This strategy aligns with a broader framework where companies keep the scarce layer proprietary while commoditizing the abundant layer. By hosting other models, Cohere keeps the infrastructure and management layer as its own product while opening up the model layer. This allows them to capture the value of the platform even if a customer chooses a rival model.

The company describes this approach as a way to provide sovereignty in AI. They argue that customers should be able to pick and run the tools that work best for them without giving up control to a single provider. By hosting rivals, Cohere aims to become the primary gateway for enterprise AI deployment regardless of which specific model a company prefers.