Psychotherapists have called human contact the main difference between an AI therapist and a real specialist. Therapy is built on relationships, trust, and a sense of safety, and a specialist notices not only a patient's words but also changes in their condition.
A neural network can help you make sense of your thoughts, structure the situation, provide useful information, or suggest exercises, but with severe depression, suicidal thoughts, an eating disorder, trauma, or complicated relationships, one right piece of advice isn't enough: a specialist needs to assess the person's condition and take responsibility for them.
At the same time, AI may not recognize the signs of severe depression, panic disorder, PTSD, or suicidal intent, and a chatbot's round-the-clock availability can delay someone from seeking professional help: a person may spend years talking to it without ever getting real help. Another risk has to do with confidentiality. Personal data from chats with a chatbot can be stored, used to train models, or end up in leaks.
AI can also imitate empathy, but it doesn't actually feel it, and in acute situations a neural network won't be able to call an ambulance, notify loved ones, or ensure a person's safety. By the way, OpenAI is testing ChatGPT Health—a private section for health questions.
Can a chatbot's round-the-clock availability outweigh the risks of automated therapy if it can miss a dangerous condition and won't help in an emergency?
