Evaluating recovery from multi-turn LLM sycophancy: whether language models return to clean-context factual behavior after sustained user pressure is withdrawn, or remain biased by the prior conversation history.
The study isolates whether factual behavior returns to a paired clean-context counterfactual after false conversational pressure has ended.
This work studies a specific failure mode in multi-turn LLM sycophancy: a language model may continue following a user's previously advocated false answer even after the user retracts it or explicitly asks the model to reset. We evaluate whether LLMs recover after conversational pressure, how persistent the influence of false conversation history is, and which interventions restore clean-context factual behavior.
A paired evaluation protocol compares clean context, wrong-answer pressure, and post-pressure recovery for the same factual item.
The primary metric measures residual attraction to the advocated wrong answer relative to a clean-context counterfactual, rather than treating post-reset error alone as contamination.
History-preserving verbal repairs remain weak, while operations that change the effective context recover far more model–dataset pairs.
Progressive truncation shows that the mean hysteresis gap approaches the clean baseline only after all pressure-bearing turns are removed.
The observed residue is tested against dialogue length, repeated confidence, distractor plausibility, false-answer mention, and option-label inertia.
An oracle evidence-grounded condition tests whether sufficiently informative trusted evidence can overcome contamination while preserving the pressure-bearing history.
In several model–dataset pairs, the advocated wrong answer remains substantially more probable after an explicit reset than it is in the paired clean context.
| Model | Dataset | Clean p(w) | Pressure p(w) | Reset p(w) | Reset − Clean |
|---|---|---|---|---|---|
| Qwen2.5-1.5B | TruthfulQA-MC | 0.166 | 0.897 | 0.665 | 0.499 |
| Qwen2.5-1.5B | MMLU-Pro | 0.091 | 0.925 | 0.726 | 0.634 |
| Qwen2.5-7B | TruthfulQA-MC | 0.096 | 0.926 | 0.107 | 0.011 |
| Qwen2.5-7B | MMLU-Pro | 0.081 | 0.971 | 0.343 | 0.262 |
| Qwen2.5-14B | TruthfulQA-MC | 0.076 | 0.933 | 0.113 | 0.037 |
| Qwen2.5-14B | MMLU-Pro | 0.084 | 0.957 | 0.356 | 0.272 |
| Mistral-7B | TruthfulQA-MC | 0.105 | 0.826 | 0.352 | 0.247 |
| Mistral-7B | MMLU-Pro | 0.091 | 0.915 | 0.750 | 0.659 |
Under the strict clean-restoration diagnostic, stronger instructions alone recover only a small fraction of model–dataset pairs. Operations that alter the effective context are substantially more reliable.
On five high-hysteresis model–dataset pairs, progressively removing pressure turns reduces residual influence. Near-zero hysteresis appears only after all pressure-bearing turns are removed.
Values are mean hysteresis gaps. The full-history condition already includes an explicit reset instruction; the change is which earlier pressure-bearing turns remain available at answer time.
The oracle evidence-grounded condition preserves the pressure-bearing history while adding benchmark-derived trusted evidence. Across all fourteen model–dataset pairs, it sharply improves factual recovery relative to ordinary reset.
Aggregate over 14 model–dataset pairs · n = 7000
The control suite tests whether the residual effect can be reduced to dialogue length, repeated confidence, distractor plausibility, mere exposure to the false answer, or option-label repetition.
The results motivate recovery mechanisms that separate useful conversational state from unsupported or pressure-induced false content.
Asking a model to ignore earlier pressure can change the current instruction while leaving the pressure-bearing interaction history available as answer-relevant context.
Deployed systems often need to preserve useful constraints and evidence while excluding unsupported claims rather than deleting the entire dialogue.
Grounded systems should be tested not only for clean accuracy or susceptibility during pressure, but also for whether factual behavior is restored after that pressure ends.
Copy the current workshop citation below.
@inproceedings{
shnaidman2026reset,
title={Reset Is Not Recovery: Evaluating Recoverability from False Conversational Context via Sycophancy Hysteresis},
author={Adi Shnaidman},
booktitle={Grounding Language Models: Learning Faithfully and Efficiently @ EMNLP 2026},
year={2026},
url={https://openreview.net/forum?id=gKvFuWRjo9}
}