An OpenAI security analysis chief who helped form ChatGPT’s responses to customers experiencing mental health crises introduced her departure from the firm internally final month, WIRED has discovered. Andrea Vallone, the head of a security analysis crew generally known as mannequin coverage, is slated to depart OpenAI at the finish of the yr.
OpenAI spokesperson Kayla Wooden confirmed Vallone’s departure. Wooden mentioned OpenAI is actively in search of a alternative and that, in the interim, Vallone’s crew will report immediately to Johannes Heidecke, the firm’s head of security methods.
Vallone’s departure comes as OpenAI faces rising scrutiny over how its flagship product responds to users in distress. In current months, a number of lawsuits have been filed towards OpenAI alleging that customers shaped unhealthy attachments to ChatGPT. A few of the lawsuits declare ChatGPT contributed to psychological well being breakdowns or inspired suicidal ideations.
Amid that stress, OpenAI has been working to perceive how ChatGPT ought to deal with distressed customers and enhance the chatbot’s responses. Mannequin coverage is one in all the groups main that work, spearheading an October report detailing the firm’s progress and consultations with greater than 170 psychological well being consultants.
In the report, OpenAI mentioned a whole lot of hundreds of ChatGPT customers could present signs of experiencing a manic or psychotic crisis each week, and that greater than one million individuals “have conversations that embrace express indicators of potential suicidal planning or intent.” By an replace to GPT-5, OpenAI mentioned in the report it was in a position to scale back undesirable responses in these conversations by 65 to 80 p.c.
“Over the previous yr, I led OpenAI’s analysis on a query with virtually no established precedents: how ought to fashions reply when confronted with indicators of emotional over-reliance or early indications of psychological well being misery?” wrote Vallone in a post on LinkedIn.
Vallone did not reply to WIRED’s request for remark.
Making ChatGPT pleasing to chat with, however not overly flattering, is a core stress at OpenAI. The corporate is aggressively making an attempt to increase ChatGPT’s consumer base, which now contains greater than 800 million individuals every week, to compete with AI chatbots from Google, Anthropic, and Meta.
After OpenAI launched GPT-5 in August, customers pushed again, arguing that the new mannequin was surprisingly cold. In the newest replace to ChatGPT, the firm mentioned it had considerably decreased sycophancy whereas sustaining the chatbot’s “heat.”
Vallone’s exit follows an August reorganization of another group centered on ChatGPT’s responses to distressed customers, mannequin conduct. Its former chief, Joanne Jang, left that function to begin a brand new crew exploring novel human–AI interplay strategies. The remaining mannequin conduct workers had been moved below post-training lead Max Schwarzer.
Disclaimer: This article is sourced from external platforms. OverBeta has not independently verified the information. Readers are advised to verify details before relying on them.
