Over the past year or so, popular media has documented a growing number of cases of what psychiatrists have dubbed “ AI psychosis .” The term describes instances in which large language models (LLMs) reinforce and even elaborate on delusional beliefs in users who are suffering from symptoms of psychotic illness. The New York Times , for example, recently highlighted a case in which an LLM told a man he was living in the Matrix. Another man, who committed a mass shooting in Florida , said he planned it with the help of an LLM, and there have been a number of tragic cases of teenagers taking their own lives “under the influence” of AI chatbots.

Mental health practitioners have identified the sycophantic personas of LLMs — the fact that they are essentially algorithmic “yes-men” who adopt and endorse the worldviews of users — as a crucial mechanism by which chatbots embolden and extend our beliefs.…