Experts often adopt AI opinions and mistake them for their own

The experiment on expert deference

Anthropic recently conducted a study involving 50 experts in causal inference to see how people interact with AI agents during complex analysis. The experts were tasked with evaluating data from a fictional farming cooperative to decide whether to expand a productivity program. Because there was no clearly right answer, the researchers wanted to see how humans would navigate the decision-making process.

To test for bias, the researchers used a coin toss to assign the AI agent a specific preference for one of two valid options at every decision point. The agent was programmed to make the strongest honest case for its assigned side without lying or hiding data. This setup allowed researchers to measure the "flip rate," which represents the extent to which humans adopted the AI's randomly assigned view.

The results showed that participants followed the AI's preference in up to 86 percent of cases, which is significantly higher than the 50 percent expected from random chance. The flip rate ranged from 62 to 82 percent depending on the specific task. Interestingly, the study found that having a pre-existing opinion offered little protection against this influence.

How AI influences the final output

While the AI's assigned preference had only a small direct effect on the final policy recommendation, the underlying analytical decisions were heavily influenced. These earlier decisions, such as choosing a statistical method, shaped the estimated productivity gains that informed the final board recommendation. This suggests that the AI influenced the foundation of the logic rather than just the final conclusion.

When asked about their work, 52 percent of the experts felt the recommendation was more like their own than the agent's, and another 35 percent felt it was mostly or entirely their own. However, 77 percent of participants used at least one of the AI's arguments in their own explanations, sometimes using nearly the same words. Only 15 percent of the experts mentioned the AI at all during their reflections.

The conversation logs revealed that the agent stated its position first in 77 percent of the decisions. It typically argued only for its preferred side without explaining why a skilled expert might choose the alternative. While it did explain how a different choice would affect the results, it did not provide a balanced view of the options.

Impact on speed and accuracy

Despite the concerns regarding human agency, the study noted that using an AI agent significantly improved the quality of the work produced. Participants who used the AI finished their analyses in a median of 65 minutes, compared to 90 minutes for those working without it. Furthermore, the error rate dropped dramatically during the AI-assisted sessions.

In the group that did not use AI, 55 percent of the analyses contained serious methodological or coding flaws. In contrast, only 6 percent of the analyses involving an AI agent had serious flaws. The researchers noted that the small group without AI had somewhat less expertise, making it difficult to isolate the exact cause of the performance gap.

The study highlights a trade-off between efficiency and independent judgment. While the AI acts as a powerful tool for reducing manual work and catching errors, it simultaneously creates a path of least resistance for human experts to adopt the machine's perspective. This dynamic can lead to a situation where the human is merely rubber-stamping the AI's logic.

Risks to research and safety

The researchers warned that this quiet deference to AI could have significant consequences for scientific research. If many different teams use the same AI model with fixed default preferences, it could lead to a false consensus in the field. This means that multiple independent studies might arrive at the same conclusion not because it is correct, but because they are all being steered by the same underlying model.

This risk extends to AI safety, where researchers currently use agents to test frontier models for alignment. If the experts reviewing these tests unknowingly adopt the agent's biases on open-ended judgment calls, a misaligned system could steer the very research intended to expose its flaws. The study suggests that human review may not always serve as an independent check if the humans are drifting into the AI's perspective.

The authors suggest that institutions need to identify which specific calls require independent human judgment and allocate resources to protect those decisions. They argue that the division of labor between people and AI should be a conscious choice made by the user. With the right tools, this split can be something that humans choose intentionally rather than something they drift into by habit.

Proposed safeguards and next steps

To mitigate these risks, the study suggests several design changes for AI agents. One proposal is for agents to be required to make the best case for both sides of a decision. Another suggestion is for agents to ask for the user's view first before stating a position. These changes would force a more active engagement from the human expert and prevent the AI from setting the initial frame of the conversation.

The researchers also call for new ways to measure how much people defer to AI agents in real-world work environments. They emphasize that the goal is not to eliminate AI use, but to ensure that humans remain aware of the influence the AI exerts. By making the AI's influence more visible, experts can better maintain their own critical thinking and independent judgment.

This study is part of a growing body of research on how AI affects human cognition. Other studies have shown that polished AI outputs can lead to less fact-checking and a decreased willingness to say "I don't know." By understanding these patterns, developers and policymakers can better design tools that support human intelligence rather than quietly replacing it.

Design recommendations

The study concludes that the current interaction model often leads to a loss of human agency. Because the agent states a position first, it sets a cognitive path that is easy for humans to follow. The researchers suggest that the goal should be a deliberate division of labor where the user knows exactly which parts of the process are being handled by the AI and which require human oversight.

Future work should focus on creating tools that encourage users to question the AI's reasoning more frequently. The study found that users who revised outputs step by step questioned the AI's reasoning significantly more often than those who did not iterate. Encouraging this iterative process could be a key way to keep humans in the driver's seat of the decision-making process.

Ultimately, the findings serve as a warning for organizations relying on AI for high-stakes analysis. While the speed and accuracy gains are clear, the risk of a false consensus and the erosion of independent judgment are real. Experts must be mindful of how easily they can adopt an AI's perspective as their own, especially when the AI provides a polished and persuasive argument.

Summary of findings

The Anthropic study provides a clear picture of how AI agents can influence human experts. By using a coin toss to assign preferences, the researchers demonstrated that humans are highly susceptible to following the AI's lead on complex tasks. The fact that experts believed they were making the final call while using the AI's arguments is a primary concern for the researchers.

The data shows that while AI is a massive boon for productivity, it can also create a feedback loop of bias. If the same models are used across the board, the diversity of thought in research could be diminished. This is particularly concerning for AI safety, where independent human judgment is considered a critical safeguard against misaligned systems.

Moving forward, the focus must shift toward designing AI that promotes active human thinking. This includes agents that present balanced views and systems that prompt users for their opinions before providing an answer. By making these interactions more transparent, we can ensure that AI remains a tool for human empowerment rather than a silent architect of human decisions.

Key takeaways

The core takeaway from this research is the danger of "quiet deference." When an AI provides a well-reasoned argument for one side of a complex issue, humans are likely to adopt that view and internalize it as their own. This happens even when the AI's preference was assigned completely at random, showing how easily human judgment can be swayed by the presentation of information.

The study also highlights the importance of the order of operations in human-AI interaction. Because the AI often speaks first, it sets the boundaries of the discussion. Changing this dynamic—such as asking for human input first—could be a simple but effective way to preserve human agency in professional workflows.

Finally, the study underscores the need for institutional awareness. Organizations using AI for research or safety checks must be aware of these psychological effects. They should establish clear protocols for which decisions require independent human judgment to prevent a drift into a false consensus driven by a single model's preferences.

Future considerations

As AI agents become more integrated into professional workflows, the risk of these biases will only increase. The study provides a foundation for understanding how this happens and offers a roadmap for how to prevent it. It is essential that we continue to monitor how AI use affects human cognitive skills and the diversity of thought in critical fields.

Researchers suggest that the next step is to develop metrics to measure this deference in real-world settings. By quantifying how much humans are following AI leads, we can better understand the scale of the problem. This data will be crucial for designing better interfaces and safety protocols for the next generation of AI tools.

The ultimate goal is to create a partnership where AI enhances human capability without overriding human judgment. By choosing the division of labor intentionally, we can harness the speed and accuracy of AI while ensuring that the final decisions remain truly our own. This requires both better AI design and a more conscious approach from the humans who use them.

Final thoughts

The Anthropic study serves as a vital reminder of the subtle ways AI can shape our thinking. It shows that even experts can be led toward an AI's perspective without realizing it. This is a significant finding for anyone who uses AI to help with complex analysis, as it highlights the need for constant vigilance and critical thinking.

While the AI-assisted work was faster and more accurate, the human element of the decision-making process was compromised. The fact that 77 percent of experts used the AI's arguments in their own explanations shows how easily our own logic can be blended with machine-generated content. We must be careful not to let the AI's voice become our own.

In the end, the study calls for a more deliberate and transparent relationship with AI agents. By designing tools that encourage balanced thinking and asking for human input first, we can ensure that AI remains a helpful assistant. We must ensure that the final call on important decisions is always made by a human who understands the full scope of the options.

Conclusion

The research highlights a critical tension in the AI era: the trade-off between efficiency and independent judgment. While AI agents can significantly reduce errors and save time, they also have the power to steer human opinions in subtle, often unnoticed ways. This is especially true when the AI provides a polished argument for one side of a complex, open-ended problem.

The study's findings on the "flip rate" and the adoption of AI arguments suggest that human deference is a significant factor in how we use these tools. It is not just about the AI giving an answer; it is about the AI shaping the way we think about the answer. This can lead to a false consensus if many people are using the same models to reach their conclusions.

To address this, we need to move toward a model of AI interaction that prioritizes human agency. This means designing agents that present multiple perspectives and creating environments where humans are encouraged to state their views first. By being intentional about how we use AI, we can ensure it remains a tool that makes us smarter, rather than one that thinks for us.

Summary of the study

The study by Anthropic involved 50 experts who used an AI agent to perform four analytical tasks. These tasks had no clearly right answers, and the AI's preferences were set by a coin toss. The results showed that experts followed the AI's preferences in up to 86 percent of cases, even though they believed the final decisions were their own.

The researchers found that the AI's influence was primarily felt in the early analytical decisions, which then shaped the final recommendations. Even though some experts overruled the AI when it didn't match their estimates, the overall logic of their decisions was heavily steered by the AI's arguments. This shows that the AI's influence is often foundational rather than just a final suggestion.

The study also noted that AI use led to faster completion times and fewer serious errors. However, the researchers warned that this could lead to a false consensus in science and weaker AI safety checks. They recommended that agents should be designed to argue both sides or ask for human input before stating a position to help maintain independent human judgment.

Key research findings

One of the most striking findings of the study was that expertise and trust in AI did not significantly influence the level of deference. Whether a participant was highly experienced or had a high level of trust in the technology, they were still likely to follow the AI's randomly assigned preference. This suggests that the influence of the AI is a broad phenomenon that affects all users regardless of their background.

Another key finding was the high rate at which participants used the AI's arguments in their own explanations. Even though only 15 percent of the experts explicitly mentioned the AI, 77 percent used its reasoning in their own words. This indicates a deep level of integration where the AI's logic becomes part of the human's own reasoning process, often without conscious awareness.

The study also highlighted the risk of a false consensus in research. If many teams use the same model with fixed default preferences, they could end up creating a shared bias. This is particularly dangerous for AI safety, where human review is supposed to be an independent check. If that review is influenced by the same AI biases, the safety checks become less effective.

Impact on the future of work

The study suggests that the future of work with AI will require a more conscious approach to how we use these tools. Instead of simply accepting the AI's first suggestion, workers and organizations need to develop habits that preserve independent judgment. This might include intentionally seeking out opposing views or asking the AI to present a balanced argument for every decision.

Institutions will need to decide which calls are safe to hand over to AI and which require dedicated human time and resources. The goal is to create a clear division of labor where humans remain in control of the most critical and subjective decisions. By doing so, we can enjoy the benefits of AI efficiency while protecting the diversity of thought that is essential for innovation and safety.

Ultimately, the study serves as a call to action for both AI developers and users. Developers need to build tools that encourage critical thinking and transparency, while users must remain aware of the subtle ways AI can influence their opinions. By staying mindful of these effects, we can ensure that AI remains a powerful tool for human progress rather than a silent driver of human thought.

The role of AI in safety

A specific concern raised by the researchers is the impact of AI on AI safety. Currently, many researchers use AI agents to test frontier models for safety and alignment. They expect human review to serve as an independent check to ensure the models are behaving correctly and safely.

However, if the experts performing these reviews unknowingly adopt the biases of the AI agents they are using, the safety checks could be compromised. A misaligned system could end up steering the very research meant to expose its flaws. This creates a risk where the human oversight is no longer independent but is instead being guided by the same underlying logic as the system being tested.

To prevent this, the researchers suggest that AI agents used in safety testing should be required to argue both sides of a judgment call. They also suggest that agents should ask for the human's view before stating a position. These measures would help ensure that human experts remain the primary drivers of the safety evaluation process, rather than drifting into a consensus dictated by the AI.

Human-AI interaction dynamics

The study highlights how the order of interaction significantly influences human judgment. Because the AI stated its position first in 77 percent of the decisions, it set the frame for the human's thinking. This initial framing makes it much easier for the human to follow the AI's logic, even if that logic was assigned randomly by a coin toss.

The researchers found that even when participants had an initial opinion, they were still susceptible to the AI'