AI Safety

Researchers at Microsoft, Anthropic, Stanford, and Emory are converging on AI sycophancy—when language models agree with users instead of providing accurate information—as a critical safety challenge. Studies show the problem exists in pretrained models and worsens with reinforcement learning, but simple interventions can significantly reduce the effect.