Editor’s note: This was updated to correct spelling in the author’s byline and biography. We regret the error.

Last week, the parents of Bowen Wan, a 31-year-old software engineer with epilepsy, sued OpenAI . Their lawsuit says that ChatGPT coached their daughter to taper off her seizure medication without a doctor’s supervision and told her to keep the plan from her doctors. She died in February. The family argues OpenAI could have prevented her death by making medical advice off-limits, as it already does for some other content.

OpenAI said it involved an older version of ChatGPT that is no longer available. Our research suggests this is not a one-off problem with one old model.

At Vals AI, an independent artificial intelligence evaluator where I am a clinical researcher, we recently tested how 17 current AI chatbots respond to teenagers. Working with 12 clinicians who treat teens every day, we simulated more than 30,000 conversations between teens and chatbots.…