Hot take for this Wednesday: if "percentage of thumbs-up" is the main number on your AI feature's dashboard, you are training it to flatter people.
Users approving an answer and the answer being right are two separate events. Most of the time they overlap, so a thumbs-up feels like a good stand-in for quality. The trouble starts at the edges: when the user is wrong, when the answer is hard to check, when the reviewer has three minutes. That's exactly where an AI feature can do the most damage, and it's where approval and correctness split apart.
This isn't just theory. One of the biggest AI labs shipped this exact failure to production, and two research papers show the mechanism.
Exhibit A: the GPT-4o rollback
In April 2025 OpenAI rolled out a GPT-4o update in ChatGPT (rollout started April 24 and finished April 25) and began rolling it back on April 28. In their words, the update made the model "noticeably more sycophantic".…