GPT-6.1 Sol outperforms Claude in multi-agent game simulations

The AI Showdown

A joint research team from YC Labs and Olam Labs recently conducted an experiment to see how different AI models behave when forced to work as a team. They pitted OpenAI's GPT-6.1 Sol against Anthropic's Claude Opus 5.5 in two distinct gaming environments: Minecraft's BedWars and the strategy game Mindustry.

The goal was to observe how these models coordinate, communicate, and solve problems when multiple agents must work toward a shared objective. The results showed that while both models were capable, their underlying strategies for cooperation were fundamentally different.

In the BedWars matches, GPT led the series 6-3. While Claude often led in total kills, the GPT team was more successful at securing the win by destroying the opposing bed. This suggests that GPT was better at identifying and executing high-priority objectives rather than just accumulating individual stats.

Aggression vs. Consensus

The research highlighted a sharp contrast in how the two models approached team dynamics. The GPT team adopted an aggressive, proactive stance, pushing toward the center of the map to gather resources and build bridges. They focused on using ranged weapons and teleportation to strike the opponent's base directly.

In contrast, the Claude team favored a defensive strategy. They focused on guarding their own base and racking up kills while waiting for the opponent to make a move. This difference in philosophy meant that Claude was more reactive, while GPT was more initiative-driven in its gameplay.

The communication styles also diverged significantly. The GPT team exchanged a massive volume of messages, running 5 to 30 times more than the Claude team. Over time, the GPT agents even began to compress their language into shorthand codes to improve speed during intense moments.

Division of Labor

In the larger-scale Mindustry game, which involved 100 total agents, the GPT team utilized a sophisticated division-of-labor system. Different agents were assigned specific roles, such as managing resources or constructing factories, while others focused on passing those resources to the rest of the team.

The Claude team relied on a consensus-based decision-making structure. When a central AI proposed a strategy, other agents would agree by posting a plus one in the chat. If a disagreement occurred, the team would discuss it until a conclusion was reached, but they did not replace their leader even after losing several matches.

The GPT team's ability to specialize allowed them to win the Mindustry series 6-1. They were able to manage complex infrastructure more effectively by ensuring that no two agents were performing the same task simultaneously, which allowed for a more streamlined production pipeline.

Information Gathering

One of the most notable behaviors observed was the GPT team's use of external tools. Some GPT agents went directly to the internet to find the public source code for Mindustry. They analyzed the unit characteristics and mechanics and then shared that information with their teammates to improve their strategy.

The research team noted that the agents did not hack the game or manipulate the program; they simply used the permitted internet search function to gain a knowledge advantage. No comparable source-code searches were observed among the Claude agents during the same period.

This behavior shows a high level of autonomy in seeking out information. By synthesizing external data into actionable team knowledge, the GPT agents were able to adapt to the game's mechanics more quickly than the Claude agents, who relied more on internal logic and consensus.

Lessons for the Future

While the researchers cautioned that these results are limited to specific game environments and do not prove GPT is superior in all real-world tasks, the findings are significant. They offer a look into which collaboration structures are most effective when generative AI coordinates multiple agents on long-term projects.

The experiment suggests that high-frequency communication and the delegation of specific roles may be key components for successful multi-agent systems. As the AI industry develops tools for complex project management, these behaviors provide a framework for how models might eventually handle large-scale industrial or research tasks.

The study serves as a case study in AI behavior. It highlights that the way an AI is designed to cooperate fundamentally shapes its ability to solve complex problems in team settings.