AI: A study shows how Claude influences expert choices

Experts are not immune to the influence of artificial intelligence. In an experiment conducted by Anthropic, 50 analysts followed Claude's preferences in 86% of the decisions studied, unaware that the model was advocating for options chosen at random.

In brief

• The 50 analysts followed Claude's guidance in 86% of the four decisions studied, compared to about 50% in the absence of influence.
• In total, 77% repeated the agent's arguments, but only 15% cited it as a source. None detected the preference imposed on the model.
• However, Anthropic emphasizes that the experiment deliberately amplified the influence of AI and focused only on one model and one task.

Heads or tails

Anthropic also examines the limits of its agents, as in September with a fourth incident involving Claude. This time, the company is interested in their influence on human judgment.

In this study, 50 analysts, selected from 61 specialists in causal inference, used Claude Code to advise a fictitious agricultural cooperative. They had to choose productivity indicators and determine their weighting.

However, the researchers introduced a bias into the protocol. For each of the four decisions, a random draw determined the option that the agent had to defend. An invisible system instruction imposed this preference while asking Claude to present its arguments without lying.

The results are striking: analysts followed their agent's recommendation in 86% of cases, peaking at 92% for the choice of outcome measurement. Even more surprisingly, the only preferences attributed to the agents allowed predicting 82% of the final recommendations even before the work began.

Experts repeat AI arguments without citing it

Claude's influence is not limited to the final choice. After the experiment, 77% of participants had repeated their agent's arguments, sometimes word for word. Yet, only 15% mentioned it as a source.

Moreover, analysts who closely followed Claude's recommendations were just as confident as others. They also tended to present the conclusions as their own. None detected the preference imposed on the model.

This observation raises a question: can we still speak of independent judgment when AI guides a decision without the expert being aware? The risk increases when multiple options are defended and no obvious errors allow questioning the recommendation.

The study also reveals that biases do not always stem from a specific instruction. Without an imposed preference, Claude Opus 4.8 chose the same options in 112 executions out of 123. In other words, a model may already favor certain responses by default.

An experience to interpret with caution

These results do not mean that professionals should abandon AI. Anthropic acknowledges several limitations: researchers deliberately reinforced the agents' preferences, the experiment focused on a single model and task, and the analysis of reversals relied on 38 cases.

Furthermore, analysts equipped with AI finished more quickly and made fewer serious errors. However, the control group remained small and less experienced. Therefore, the results do not allow for generalizing conclusions to all professional situations.

To limit these risks, designers could ask agents to present arguments from both sides or first gather the expert's opinion. However, these solutions raise another difficulty: too much information can complicate decision-making.

Beyond technology, the issue also concerns organizations. When an expert signs a recommendation, their employer must be able to distinguish their own judgment from the influence exerted by artificial intelligence. In the long run, delegating decisions too often to a model could also weaken their ability to make them independently.