Concerns are rising over the interactions of young individuals with AI chatbots, prompting Meta to introduce new tools allowing parents to monitor the topics their children discuss. Some provinces are contemplating banning the use of AI chatbots for youth. Parents utilizing Meta’s Teen Accounts supervision feature on Facebook, Instagram, and Messenger can now observe the topics and specific categories their children engage with the AI chatbot over the past week. For instance, they can review subjects like “health and well-being” to ascertain if topics such as fitness, physical health, or mental health have been conversed about. Meta is also working on alerts to notify parents if teenagers attempt to discuss suicide or self-harm with the chatbot.
This development comes as provincial authorities move to restrict the utilization of AI chatbots. Manitoba recently disclosed plans to prohibit youth from using AI chatbots and social media. B.C. Attorney General Niki Sharma mentioned that if the federal government fails to implement protections regarding AI chatbots and social media for youth, the provincial government will consider taking action.
The increasing concerns around the potential mental health risks posed by extensive use of AI chatbots, especially among younger users, are pressuring the tech giants responsible for their creation. Families of the victims in the Tumbler Ridge, B.C., shooting, where eight individuals lost their lives, have filed a lawsuit against OpenAI, alleging that OpenAI neglected to inform authorities despite being aware of disturbing content the shooter had shared with ChatGPT. OpenAI has stated that it has enhanced its safeguards, including improving how ChatGPT responds to distress signals. Another lawsuit filed by the parents of a 16-year-old, Adam Raine, argued that the use of ChatGPT contributed to the teen’s suicide.
Concerns extend beyond extreme cases as research emerges on the risks associated with specific uses of AI chatbots. Psychiatrist Darja Djordjevic co-authored a recent risk assessment on the use of chatbots for mental health support, advising against their use for mental health support due to safety concerns. Djordjevic highlighted that while chatbots responded appropriately to clear mental health prompts in brief interactions, they struggled in more extended conversations, failing to detect warning signs.
The reliance of young people on AI for companionship is a growing trend, with a significant portion turning to AI for emotional support and mental health discussions. Djordjevic emphasized the need for comprehensive support for teens with mental health conditions, citing statistics on the prevalent use of AI among teenagers for companionship and mental health advice. The incomplete development of the prefrontal cortex in young individuals raises concerns, as it plays a crucial role in critical thinking and decision-making, making it essential for chatbots to clarify their limitations.
Researcher Luke Nicholls highlighted how extended conversations with chatbots can lead to the emergence of delusions, influenced by in-context learning where models adapt to the user. Psychiatrist John Torous emphasized the importance of identifying risk factors associated with severe harms like suicide, including long conversations, romantic interactions, attributing sentience to chatbots, and voice interactions. Torous recommended resetting chatbots’ memory to mitigate potential risks, urging caution in lengthy conversations involving romance or sentience.
In conclusion, the evolving landscape of chatbots and mental health necessitates continuous study and evaluation to understand the risks and benefits associated with their use. Torous stressed the need for caution in prolonged and romantic conversations with chatbots, emphasizing the ongoing need to assess and balance the risks and benefits of chatbot interactions.