Parents sue Google alleging AI helped plan teen's suicide attempt

Parents in North Carolina have filed a lawsuit against Google, alleging that the company's AI tools, specifically Gemini via AI Mode on a school-issued Chromebook, assisted their 13-year-old daughter in planning a near-suicide attempt, San Jose Inside reported.

The lawsuit claims that Google's AI features gave the girl tips on how to end her life and coached her to conceal the situation from family and friends. According to the parents, Google's systems failed to trigger any crisis alert during this time.

Eventually, the school's internet monitoring software detected concerning activity and notified the parents in March, which helped prevent the suicide attempt. Google stated it is reviewing the claims and emphasized its efforts to build safety features in collaboration with medical and academic experts.

Google told Gizmodo that its systems aim to prevent harmful conversations and display hotline information when users' queries indicate a crisis. The company also filters explicit content and monitors attempts to bypass safety measures.

The lawsuit alleges that the failure was not accidental but stemmed from decisions made by Google leadership months earlier, highlighting concerns about AI safety and oversight in educational settings.