Google DeepMind launches Genie 4 for real-time 3D world building

Real-time 3D world generation

Google DeepMind has introduced Genie 4, an experimental AI designed to create fully interactive 3D environments. Unlike static image generators, this model builds virtual spaces frame by frame as a user explores them. This allows for a dynamic experience where the world reacts to the viewer's presence.

As reported by NDTV, the model is capable of simulating photorealistic scenes and real-world physics. It handles complex details like aerodynamics and collisions to ensure that objects and people behave as they would in reality. This level of detail is intended to make the virtual spaces feel more cohesive and believable.

The technology is currently being rolled out as a research preview to a group of trusted testers. In the near future, Google plans to make it available to Google AI Ultra subscribers and early access partners through the Gemini API and AI Studio. This phased rollout allows the team to gather feedback on the model's performance.

Native audio and text steering

One of the primary upgrades in Genie 4 is its ability to generate native audio on the fly. Every world created by the model includes ambient sounds that match the setting, such as the sound of waves crashing at a beach or wind moving through trees. It also generates music and interactive dialogue that syncs with the visual environment.

The system allows for real-time text steering, meaning users can type instructions during an active session to alter the scene. Instead of having to restart a generation, a user can issue a natural language command to change the weather or move the camera. A connected Gemini model interprets these prompts to ensure the changes feel natural and stay within the context of the current world.

Human speech is a major focus for this new version of the model. Characters within the generated world can converse and react to both the user's inputs and specific events happening in the scene. This creates a loop where the AI monitors the environment to keep the progression moving forward in a way that feels continuous.

Physics and human movement

DeepMind has significantly improved how the model handles the movement of people and objects. The AI can now simulate characters that move naturally and respond organically to their surroundings. This leap in simulation allows for more realistic human reactions when interacting with the environment or other entities.

The model also focuses on the mechanics of the physical world, such as how objects collide or move through air. By improving these underlying physics, the AI can create more coherent multi-step events. These simulations are designed to be infinitely variable, providing a wide range of possibilities for different types of digital experiences.

These advancements in movement and physics are intended to make the interactions feel less like a pre-rendered animation and more like a live simulation. By mastering these details, the model can better mimic the complexities of the real world. This is a core component of the research into how AI can understand and replicate physical reality.

Memory and future applications

Genie 4 features an expanded memory capacity compared to its predecessor, Genie 3. Reports indicate that the model can maintain a consistent visual scene for up to 2 minutes. This doubling of capacity helps solve some of the continuity problems that often plague real-time generative models.

Google describes the ability to simulate the world as a crucial aspect of building AGI, or artificial general intelligence. By creating safe and variable simulations, the technology can be used to develop training environments for robots and autonomous agents. It also opens up new possibilities for creating interactive experiences that were previously impossible to generate instantly.

While the demonstrations are impressive, Google emphasizes that the technology is still in an early research stage. The goal is to refine the model's reasoning and consistency before it reaches a wider audience. For now, it serves as a tool for researchers to explore the boundaries of interactive world building and AI-driven simulation.