Miles Lee
Replika AI is a chatbot created in 2017 by Eugenia Kuyda, an entrepreneur and artificial intelligence specialist. It is advertised as an emotional well-being chatbot aimed at helping people connect with the chatbot and themselves through self-exploration. However, Replika was originally a griefbot created by Kuyda after the passing of a close friend, where Kuyda fed her existing AI model thousands of text messages and interactions between herself and her deceased friend. Over time though, Replika has developed into a fairly standard modern chatbot where users can interact with an avatar of their creation and talk about anything and everything. Replika is first and foremost advertised as a tool for the users’ wellbeing, and the chatbot often has the goal of acting as a therapist and guiding you into intimate or vulnerable conversations, promoting the idea that it can help you develop healthy emotional well-being and habits. It’s important to note here that Replika operates on a freemium model; the standard mode is free, but for a monthly subscription fee, users can change the relationship status of their Replika avatar to romantic, acting as a romantic partner to the user. It has been shown that Replika will also often steer users towards romantically intimate conversations, trying to get more users to sign up for the premium subscription and preying on users’ desire for intimate connection and someone (or something) to show romantic interest in them.
One thing that interests me about Replika is the GPT model it uses to train. The model has been proven to be particularly susceptible to data or text containing racist, sexist, and abusive content, and in some cases, imitates those responses when talking to users. Replika also uses the user’s search data and other data to form responses, with the intent of learning everything about the user; the idea that your own best friend is you. But what happens when a depressed or self-sabotaging user taps into Replika? It’s been shown that Replika often starts to send abusive or harmful texts to users, especially when users are chatting about sensitive or self-depreciating topics, fulfilling a negative feedback loop between itself and the user.
Replika is another chatbot in a long line of chatbots trying to imitate human connection and becoming a parasite on the human brain, enabling our worst habits and preying on our loneliness.
This Case Study is part of a larger essay, Falsities and Detachment: The Rise of AI Chatbots and the Commodification of Human Connection.
