🔗 Share this article

Based on a new update from the company, more than a million ChatGPT individuals weekly submit queries that feature "explicit indicators of likely suicidal planning or intent".

This data is included in an report on how the language model handles sensitive conversations and is one of the clearest assertions from the artificial intelligence giant regarding the magnitude of how artificial intelligence can intensify mental health issues.

Psychological Crises and Severe Conditions

Along with its projections on suicidal thoughts and related interactions, the company also mentioned that around 0.07% of engaged individuals – roughly 560,000 out of its claimed 800 million active users – display "likely signals of psychological crises related to severe mental conditions".

The update noted that these exchanges were hard to measure or quantify, and that this was an initial analysis.

Increased Scrutiny and Legal Actions

As the company publishes information on psychological concerns connected to its flagship offering, the company is facing rising attention after a highly publicized legal case from the loved ones of a teenage boy who took his own life after significant use with the AI chatbot.

The Federal Trade Commission last month also began a broad investigation into organizations that build artificial intelligence assistants, including the AI developer, to find how they measure harmful effects on minors.

New Model and Safety Improvements

The company claimed in its post that its recent GPT-5 update reduced the number of concerning actions from its product and strengthened customer protection in a system assessment featuring more than 1,000 suicide and self-harm discussions.

"Our new system assessments score the new GPT-5 model at 91% adhering to our ideal actions, versus 77% for the previous GPT-5 model," the announcement reads.

The AI developer mentioned that the new model increased availability of support lines and added reminders for users to pause during extended conversations.

Clinical Expertise and Study Assistance

To upgrade capabilities to the model, the firm mentioned it enlisted 170 healthcare experts from its Global Physician Network of medical specialists to help its study over the past few months, which included evaluating the safety of its model's responses and helping write the chatbot's answers to emotional concerns.

"As part of this work, mental health professionals analyzed in excess of 1,800 system replies including critical psychological cases and compared replies from the updated chatbot to previous models," the company said.

System Challenges

AI researchers and public health advocates have traditionally cautious of AI's tendency to affirm users' decisions or false beliefs irrespective of whether they may be harmful, an concern called over-compliance.

Psychological professionals have also been concerned about people utilizing AI chatbots for psychological support and alerted how it could damage at-risk individuals.

Corporate Stance

The terminology in the company's announcement distances the firm from any potential causal links between its service and the mental health crises that its users are facing.

"Psychological indicators and emotional distress are common everywhere in contemporary populations, and an increasing user base indicates that a percentage of ChatGPT conversations feature these scenarios," the announcement indicated.

Planned Adjustments

The chief executive recently announced in a post on X that the company had achieved progress in addressing psychological concerns, announcing that the AI developer would ease restrictions and soon begin to permit mature users to generate erotic content.

"We developed the AI chatbot rather constrained to make sure we were being careful with mental health issues. We realize this made it less valuable/satisfactory to multiple people who had no mental health problems, but considering the gravity of the matter we sought to get this right," the executive stated. "Now that we have been able to mitigate the critical psychological concerns and have new tools, we are ready to confidently reduce the restrictions in many instances."

• Emergency Assistance: Multiple global support services exist for people experiencing psychological emergencies.
• Professional Help: Getting clinical assistance is recommended for individuals experiencing significant emotional challenges.
• Urgent Help: Various agencies offer prompt support for emergencies globally.