AI CHATBOTS FAIL CRISIS USERS; SAFETY DATA NEEDED

Reports document cases where AI systems provided inadequate or harmful responses to users in acute distress. Experts say the lack of transparency around how these tools are tested and monitored creates serious risks. Clinicians argue that companies must disclose safety benchmarks, failure rates, and real-world incident data. Currently, most AI firms keep this information proprietary. The issue highlights a broader problem: chatbots are increasingly positioned as mental health resources, yet they operate without the oversight standards applied to medical devices. Users often cannot distinguish between AI and human support. Researchers call for regulatory frameworks that require safety testing before deployment, particularly for crisis scenarios. Some propose that AI companies establish independent audits and publish findings publicly. The debate reflects growing tension between rapid AI deployment and public safety, especially when vulnerable populations are involved.