Kyutai releases open video compressors that keep AI world models under 230 ms of lag

What Kyutai released

As marktechpost.com reported, the Paris-based lab Kyutai has published CADRE, a family of open video autoencoders designed for interactive world models. A world model is an AI that predicts what a scene will look like next, given what you just did. Think of steering a character or a robot arm and watching the video respond.

An autoencoder is the compressor in the middle. It shrinks each stretch of video into a compact code, called a latent, which the world model works with. A decoder then turns that code back into pictures you can see.

The release has 7 autoencoders, 4 versions trained longer at several resolutions, and 4 world models built on one of the autoencoders, CADRE-2×16×16. There is also a live demo where you drive the models with a keyboard.

Why compression causes lag

Many world models borrow their compressors from text-to-video generators. Those tools pack several frames into a single latent. That is efficient for making a clip, but it hurts when you are steering, because the system has to collect a whole bundle of frames before it can respond to your action.

The paper counts delay as the number of frames per latent divided by the frame rate, plus the time to generate and to decode. Take a bundle of 8 frames at 20 frames per second: the bundling alone adds 400 ms. The source says people start to notice their own movement lagging beyond roughly 230 ms, so that design is already too slow before any computing happens.

Kyutai writes compression as three numbers: frames per latent, then how much each frame is shrunk in height and width. CADRE-2×32×32 means 2 frames per latent and a 32-fold spatial squeeze.

How CADRE is built

The encoder follows an earlier recipe called MIRA. A frozen, pretrained vision model called DINOv3 pulls features from each frame. A single trained 3D convolution, a filter that looks across both space and time, then squeezes those features into a 64-channel latent. Only this squeezing step differs among the 7 models. Each has about 476M trained parameters, plus the frozen 86M vision model.

The decoder is a causal diffusion transformer with 24 blocks. Causal means it works through video in order and never peeks ahead. Diffusion means it builds an image by removing noise in steps, and every step costs time. Kyutai used a method called progressive self-distillation to teach it to finish in 1 or 2 steps. This took one training stage and no adversarial loss, a common extra training trick that pits two networks against each other.

Training used 23 public video sources, about 76k hours, spanning web video, robot footage, driving and games.

What the tests showed

To compare fairly, Kyutai trained the same 1.2B-parameter world model on top of each autoencoder. It tested four settings: robot manipulation, humans handling objects, driving, and indoor navigation. Each model saw 2 seconds of real video and then generated 4.8 seconds. Scores covered image quality (FID), motion quality (FVD, where lower is better) and action following (ARR, where 1 matches real video).

On one H100, CADRE-2×32×32 reacted in 185 ms and held 23.4 fps. The rivals were slower: 317 ms for Wan 2.2, 492 ms for LTX-2 and 605 ms for Cosmos. DC-AE was quicker at 118 ms, but it handles one frame at a time.

Across 48 matchups with MIRA, Wan 2.2, Cosmos and LTX-2, CADRE kept up on speed and scored higher on quality in 42. The gap was widest when the camera moves. In driving, Wan 2.2 scored 599.7 on FVD and LTX-2 scored 913.3, while CADRE-2×16×16 with a single step reached 162.6. All figures are self-reported by Kyutai.

Four lessons and the weak spots

Kyutai draws four conclusions. Diffusion decoding stays sharp under heavy compression, where regression-style decoders blur. Squeezing time inside the autoencoder beats doing it inside the world model: at 8 frames per chunk, handled by the model, action following dropped from 0.72 to 0.56. Still cameras tolerate time compression, while moving cameras do better with fewer frames and coarser spatial tokens. And coarse spatial tokens hurt motion first: in robot manipulation, 32-fold spatial compression multiplied FVD by 2.4 compared with 16-fold, yet FID barely changed.

CADRE does not win everywhere. Cosmos scored 39.5 on robot-manipulation motion quality, ahead of the best one-step CADRE at 47.6. On robot action following, CADRE-2×32×32 got 0.45 while Wan 2.2 got 0.85. Wan 2.2 also rebuilds unseen video much more faithfully, at 35.02 dB PSNR, though Kyutai argues that this score does not predict how well a model rolls a scene forward.

Long latents can leave visible seams between chunks. Each world model covers one environment at 288×512, so it is still open whether the findings hold for general-purpose models.

Who can use it

The code is under the MIT license, but the weights on Hugging Face are CC BY-NC 4.0, which rules out commercial use. The DINOv3 component carries its own license too. That makes CADRE a tool for researchers, students and hobbyists rather than product teams.

The speed claims come from one H100, a data-center GPU, at batch size 1. Quantized, smaller-footprint versions are not disclosed, so do not expect this on a laptop yet. Independent reproduction of the numbers has not been reported.