SIAgentNews.com
← All news

OVHcloud's Olala mini packs a 7B-parameter model into 8 GB of GPU memory

European cloud firm OVHcloud unveiled a small model that activates about one billion of its seven billion parameters and, it says, runs on a plain CPU.

· First reported by Open Source For You

8 GBGPU memory Olala mini needs, per OVHcloud

The short version

  • Olala mini has 7 billion parameters, with about 1 billion active at a time across 256 experts per layer.
  • It runs in 8 GB of GPU memory with a 32,000-token context window.
  • OVHcloud claims 160 tokens per second on an RTX A4500 and 50 on a plain CPU.
  • Weights are planned for Hugging Face under Apache 2.0, with an AI Act compliance document.
  • Qwant reportedly got 20% faster answers and three times lower running costs.

Why it matters

A capable model that fits in 8 GB of GPU memory, or runs on a CPU, puts local AI agents within reach of ordinary workstations. Keeping data on your own machine and avoiding per-call fees becomes more realistic. The claims come from the company, so independent tests will matter.

What OVHcloud announced

As Open Source For You reported, OVHcloud, a European cloud computing company, has launched Olala mini 7A1B. It is the first family of large language models (LLMs) the company has built. The aim is fast AI responses that need less memory and computing power.

Our earlier note on this story could only go by the headline, so we said it was a direction rather than a release. The article does name a specific model with specific numbers, which this piece covers. Open release is still a plan, not something we can confirm is live.

Why it is small but not weak

The model has seven billion parameters, the adjustable numbers that hold what a model has learned. It does not use them all at once. It relies on a mixture-of-experts design, where the model is split into many specialist parts and each request only wakes some of them. Here there are 256 experts per layer, and about one billion parameters are active at any moment.

Using fewer parameters per request means less arithmetic and less memory traffic. The article also says the model combines two attention methods, differential-TPA and Mamba-3. Attention is how a model decides which parts of its input matter. The article does not explain these two methods, so we will not guess at their details.

The model runs in BF16, a compact number format. Its context window, the amount of text it can consider at once, is 32,000 tokens, which are small chunks of text.

The speed and memory claims

OVHcloud says the model fits in 8 GB of GPU memory. It puts speed at 160 tokens per second on an NVIDIA RTX A4500 graphics card. It also gives 50 tokens per second on a simple CPU with no GPU at all.

These are the company's own figures, and the article does not describe how they were measured. They are still a useful signal. A CPU-only result suggests the model could run on hardware many people already own.

The article also describes one early customer. Search engine company Qwant has put Olala mini into its flash answer tool. Reports say answers are 20% faster and operating costs are three times lower than with the previous version.

Public supercomputers and open licensing

Olala was trained on three European public supercomputers: Jupiter, MeluXina and Jean Zay. They use NVIDIA GH200, A100 and H100 GPUs respectively. The computing resources were worth about €7 million in European public support. That training also helped create OVHcloud's Agentic Lab, OVHai LLM.

OVHcloud plans to publish Olala mini on Hugging Face under the Apache 2.0 license, a permissive license that allows commercial use. The package is to include the model weights, the base model, a technical report and documentation for compliance with the EU's AI Act. CEO Octave Klaba said training LLMs has become far easier and cheaper.

What comes next

The roadmap has two more steps. Olala 1.1, due by the end of 2026, is meant to stretch the context window to 128,000 tokens and improve mathematical reasoning. Olala 2.0 would follow with more than 100 billion parameters, aimed at enterprise agentic AI, cloud DevOps and cybersecurity.

An agent is a program that plans and takes steps on its own, such as calling tools. The article says these models are also designed to run on local AI systems on workstations through OVHai Workspace.

The open question is how well the small model handles real agent work, not just speed tests. It also remains to be seen how closely the weights, once released, match what the article describes.

What to watch

  • Does OVHcloud publish the weights on Hugging Face under Apache 2.0 as planned?
  • Do independent tests confirm the 160 and 50 tokens-per-second figures?
  • Does Olala 1.1 arrive by the end of 2026 with the 128,000-token window?

Try it

Check how much GPU memory your machine has against the 8 GB figure. When the weights appear, test it on a pass/fail task like renaming files, and count how often it finishes correctly.

Sources and links

First reported by Open Source For You. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news