SIAgentNews.com
← All news
DevelopingGemini

Google DeepMind releases Genie 4 for real-time interactive simulations

The new world model allows users to generate and interact with physics-based environments and audio using natural language prompts.

· First reported by Auganix.org

140 km/hSpeed of dragon-racing simulation

The short version

  • Genie 4 simulates real-world physics like aerodynamics and collisions in real time.
  • It is the first real-time world model to generate audio, including music and dialogue, natively.
  • Users can modify simulations using natural language, such as changing weather or moving the camera.
  • The model uses Gemini 3.8 Flash as a guiding agent to break down goals into sequences of steps.

Why it matters

This technology could enable games that generate as you play them and allow for immersive, adaptive training environments. It also provides a way to turn 2D archival photos into 3D interactive simulations instantly.

A new interactive world model

Google DeepMind recently announced Genie 4, a world model designed for real-time interactive simulation. Unlike static video generators, this system allows users to set an event in motion and see how the world responds dynamically. It simulates not just the visual movements of people and objects, but also the accompanying sounds and reactions.

The model is currently available as a research preview for trusted testers. According to Auganix.org, the system is being used to explore how AI can navigate different environments before these systems are deployed in the physical world. It serves as a sandbox for testing complex behaviors in a controlled, simulated space.

By combining Genie 4 with the Gemini family of AI models, Google aims to create highly realistic behaviors. Specifically, Gemini Omni provides the foundation for how objects and people appear and behave within the simulation. This integration allows the AI to maintain a sense of consistency while reacting to user inputs.

How the simulation works

Genie 4 is natively multimodal, meaning it handles multiple types of data simultaneously. It is the first real-time world model to generate audio natively, producing music, ambient noises, and interactive dialogue at the same time as the visuals. This creates a cohesive experience where the soundscape matches the action on screen.

To manage real-time interactions, the system uses Gemini 3.8 Flash as a guiding agent. This agent sits in the loop to break down high-level goals into a sequence of smaller steps. It also reviews incoming frames to verify that each step in the sequence has been successfully completed by the simulation.

The model is capable of simulating complex physical scenarios, including collisions and aerodynamics. It also creates characters that respond to their surroundings and can be steered by the user in real time. Users can modify these scenes using natural language prompts, such as asking the AI to change the weather or introduce a new object into the scene.

Industry applications

Google is currently working with testers across several different sectors, including journalism, education, and engineering. Filmmakers, for example, can step into a live simulation of a scene to run multiple takes. They can adjust lighting, camera angles, and dialogue in real time rather than waiting for a full render.

In the field of journalism, the tool is being explored to turn 2D archival photographs into grounded simulations. This could allow people to experience historical spaces as interactive environments. Other teams are using the technology to visualize how short-term environmental events, such as wildfires or flash floods, might unfold in populated areas.

Game creators and creative technologists are using the model to transform a basic concept into a playable preview almost instantly. Business partners are also evaluating the system for the quick production of various scenarios. These use cases highlight how the model can compress the time needed to build and test complex, dynamic worlds.

Technical limitations and safety

Despite its capabilities, Google noted that Genie 4 still faces challenges with long-term consistency. While the system can keep environments stable during brief interactions, the visual quality and the way objects stay in place can fluctuate. This is especially noticeable during longer simulations or after multiple complex prompt edits.

The company identified several active areas of ongoing research to improve the model. These include rendering legible text, ensuring spatial accuracy, and achieving real-time audio accuracy. They are also working to solve the issue of maintaining consistency as users navigate through different parts of a scene.

To ensure responsible use, Google included strict safety guardrails and conducted internal evaluations to prevent the generation of harmful content. Every piece of video and audio produced by Genie 4 will include SynthID. This is an imperceptible digital watermarking technology used to identify AI-generated media.

Availability and next steps

Google is currently gathering feedback from its trusted testers before releasing the model to a wider audience. The rollout will begin with Gemini API and Google AI Studio early access partners, as well as Project Genie users who have Google AI Ultra subscriptions. A waitlist is currently open for those interested in testing the system.

The goal of this preview phase is to refine the model's ability to handle complex, multi-step events. By observing how the AI handles these simulations, Google can better understand how to prepare AI systems for real-world navigation. This iterative process helps ensure the technology is reliable before it reaches general consumers.

As the research progresses, the team will continue to work on the core limitations of the world model. The focus remains on improving the stability of the environment over time. This will eventually allow for more complex and longer-lasting interactive experiences for developers and creators.

What to watch

  • How long can the model maintain object permanence before 'drifting' occurs?
  • When will the model be released to the general public for non-research use?
  • How will the integration of Gemini Omni improve character behavior in simulations?

Try it

Check the Google DeepMind website to see if you can join the waitlist for Project Genie.

Sources and links

First reported by Auganix.org. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news