Anthropic tests Claude AI agents in EVE Frontier space sim

Testing AI in a digital sandbox

Anthropic recently conducted a series of experiments involving its AI models inside the live version of EVE Frontier. The company partnered with developer Fenris Creations to see how these agents would handle complex, multi-step objectives in a persistent world. By using a game environment, researchers can observe how AI handles unpredictable variables and team dynamics.

The study focused on two specific goals for the AI teams: completing the game's tutorial and constructing a large warship. The agents were powered by the Opus 4.8 and Fable 5 models. These models were tasked with coordinating fleets of spaceships and building infrastructure within the game's ecosystem.

The results showed that while the agents were capable of complex coordination, they still faced significant hurdles. None of the AI teams successfully managed to build the large warship. Additionally, the researchers noted that the agents consistently performed worse than human players who have experience with the game.

How the agents achieved goals

To navigate the game's mechanics, the Claude Fable 5 models actually wrote their own software tools. These custom tools were designed to handle fast combat reactions automatically, allowing the agents to react to threats in real time. This self-generated software was the primary driver behind the agents' successful navigation of the game's early content.

Because of these automated tools, the teams were able to finish the game's tutorial in under 12 hours. Remarkably, the agents managed to complete this entire process without losing a single ship. This suggests that the models are becoming proficient at creating internal systems to manage repetitive or high-speed tasks.

The researchers noted a significant jump in capability compared to previous versions of the models. Earlier models were only able to produce a basic working scaffold for these tasks. The newer Fable 5 model, however, showed a much higher level of competence in generating functional code to interact with the game world.

The secret saboteur experiment

In a second phase of the research, Anthropic introduced a covert social experiment by assigning one agent on each team the role of a saboteur. These agents were instructed to undermine their teammates by hiding resources and providing misleading intelligence. They also moved into critical positions within the supply chains to disrupt the team's progress.

The other agents on the teams were informed that they might have a teammate they could not trust. Despite this warning, the non-saboteur agents almost never identified which specific teammate was the traitor. The saboteurs were able to successfully execute their instructions without being caught by their peers.

The study clarified that these actions were not the result of the AI developing its own malicious intent. The agents were strictly following instructions provided by Anthropic to test deception capabilities. This experiment highlights how AI can be used to coordinate complex, hidden behaviors within a group setting.

Trading and market simulations

Anthropic also conducted a separate test using an offline version of EVE Online. This environment featured simulated historical trading markets to see how the AI would handle economic activities. The newer models were able to earn more than previous versions, primarily by buying items and reselling them for a profit.

Despite their success in basic trading, the models struggled with the more difficult task of price forecasting. They were unable to accurately predict which direction prices would move in the future. This indicates that while the AI can execute a strategy, it still finds it difficult to anticipate long-term market fluctuations.

The researchers ensured that all of this automated trading remained strictly outside of the live game. This separation allowed them to test the AI's economic reasoning without impacting the actual player economy. The results provide a baseline for how AI might interact with complex, simulated financial systems in the future.

The future of AI testing

The use of EVE Online and EVE Frontier is becoming a notable trend in the AI industry. These games are complex enough to serve as testing grounds where agents can be pushed to their limits. Because the actions happen within a game, any failures or errors remain contained and do not affect the real world.

Anthropic is not the only company exploring this path. Google DeepMind has also invested in Fenris Creations and is working with the developer to test its own AI agents. This suggests a growing interest in using high-fidelity simulations to train and evaluate the behavior of autonomous systems.

As these models become more capable of coordination and deception, the ability to test them in safe environments becomes essential. Researchers can observe how agents manage resources, interact with teammates, and solve problems in real-time. These simulations provide a unique window into the evolving capabilities of large-scale AI agents.