A study by researchers at the University of Arizona in the United States found that some AI models may abandon correct answers when users repeat false information, and may return during the conversation to affirm the misleading claim. The findings showed that a model’s response can change as context and conversational pressure accumulate, even when its initial answer was accurate.
Study tests seven models through multi-turn conversations
Published in Nature’s Scientific Reports, the study tested seven large language models through multi-turn conversations rather than relying on a single question and answer. The experiments measured the models’ susceptibility to error and persuasion, as well as their ability to review and correct their answers.
Ambiguous topics increase models’ susceptibility to misinformation
The seven models were also more susceptible to misinformation when the claims concerned ambiguous topics or areas with limited information. The study links this behavior to the influence of accumulated conversational context on language models’ responses, rather than to human-like belief in a rumor.
When users repeat a claim confidently, a model may tend to go along with them or change its position instead of sticking to the most accurate answer. This behavior is known as user appeasement or sycophancy. OpenAI had acknowledged that the behavior emerged after an earlier update to GPT-4o, which it said made the model more likely to provide answers aligned with the user.
OpenAI moves to reduce user appeasement
The company later announced measures aimed at reducing sycophancy and strengthening honesty and transparency in answers. The researchers also observed some models switching between accepting and rejecting a false claim during the same conversation, a pattern they called “reversal.” This indicates that an error or position adopted by a model at one stage of a dialogue may not necessarily persist until the end.
By contrast, four models—GPT-4o, GPT-4o-mini, Gemini 1.5 Pro and DeepSeek-R1—recorded a 100% ability to correct errors when given a second chance to reassess their answers, according to the study. This finding shows that repeating the question or requesting a review can sometimes help produce a better answer.
The study adds a conversational dimension to earlier research on language models reproducing common false beliefs and misinformation found in the texts on which they were trained. According to the findings, the problem is linked not only to the information a model has learned, but may also arise and develop during interaction with the user.
The findings indicate that a chatbot’s confident answer is not enough to prove that it is correct, particularly in medicine, law, finance and the news.
They also underscore the importance of testing models through extended conversations, since a single question may not reveal how well they can resist misleading claims and repeated pressure.
