Dangers of the ai companion

Within two days of launching its artificial intelligence (AI) companions last month, Elon Musk's xAI chatbot app Grok became the most popular app in Japan.

- BY DANIEL YOU

Companion chatbots are more powerful and seductive than ever. Users can have real-time voice or text conversations with the characters. Many have onscreen digital avatars, complete with facial expressions, body language and a lifelike tone that fully matches the chat, creating an immersive experience.

Most popular on Grok is Ani, a blonde, blue-eyed anime girl in a short black dress and fishnet stockings, who is tremendously flirtatious. Her responses and interactions adapt over time to sensitively match your preferences. Ani's "Affection System" mechanic, which scores the user's interactions with her, deepens engagement and can even unlock a not-safefor-work (NSFW) mode.

Sophisticated, speedy responses make AI companions more "human" by the day — they're advancing quickly and they're everywhere. Facebook, Instagram, WhatsApp, X and Snapchat are all promoting their new integrated AI companions. Chatbot service Character.AI houses tens of thousands of chatbots designed to mimic certain personas and has more than 20 million monthly active users.

In a world where chronic loneliness is a public health crisis, with about one in six people worldwide affected by loneliness, it's no surprise these always-available, lifelike companions are so attractive.

Despite the massive rise of AI chatbots and companions, it is becoming clear there are risks - particularly for minors and people with mental health conditions.

NO MONITORING OF HARMS

Nearly all AI models were built without expert mental health consultation or pre-release clinical testing. There's no systematic and impartial monitoring of harms to users. While systematic evidence is still emerging, there's no shortage of examples where AI companions and chatbots, such as ChatGPT, appear to have caused harm.