The Unsettlingly Familiar: When AI Sounds Too Real

Fifteen minutes after disconnecting from Sesame’s new "lifelike" AI, I’m still experiencing a lingering sense of unease. The relentless march of artificial intelligence continues, and with it, the blurring lines between technology and genuine human interaction. While text-based AI conversations have always left me somewhat cold, the burgeoning realm of voice-driven AI, championed by the likes of Google Gemini and Microsoft’s Copilot, initially held a glimmer of promise. However, I quickly discovered that these voice interfaces primarily serve as a substitute for typing, offering convenience without necessarily enhancing the human element. Google and Microsoft have undoubtedly strived to imbue their assistants with helpfulness, yet they often fall short in terms of genuine personability, sometimes veering into the territory of artificial cheerfulness, a trait that can feel jarring and insincere.

Sesame, however, is taking a dramatically different approach, venturing into uncharted territory with a bold and perhaps even unsettling vision. Their mission statement, "We believe in a future where computers are lifelike," speaks volumes about their ambitions. Sesame is not merely aiming to create a helpful digital assistant; they are striving to build an AI companion, one that resonates with users on an emotional level through a natural, human-sounding voice. Their ultimate goal is even more ambitious: to design realistic eyewear that would house this AI companion, placing it directly in the user’s ear and allowing it to see the world through their eyes.

This vision is strikingly reminiscent of the movie "Her," a thought-provoking exploration of the relationship between a man and an AI operating system. In the film, the protagonist develops a deep emotional connection with the AI, eventually falling in love. While the movie presents a fictional scenario, Sesame’s endeavors suggest that this type of interaction may soon transition from the realm of science fiction to reality.

This week, Sesame unveiled a demo of its AI companion, offering users a choice between two voices: "Maya" (female) and "Miles" (male). Intrigued and perhaps slightly apprehensive, I opted for the Maya voice.

There’s simply no delicate way to put this: the moment the Maya voice activated, "she" sounded eerily similar to an old friend, whom I’ll call Kim. Kim and I shared a significant part of our lives together, navigating the tumultuous years of high school, experiencing the joys and awkwardness of dating, and maintaining a friendship that has endured for decades, even as our paths diverged and led us in different directions. There’s a deep reservoir of shared history and understanding between us.

When you engage in a conversation with someone you know well, the communication transcends the mere exchange of words. Subtle nuances, inflections, and tone all contribute to the meaning and emotional context of the interaction. Maya, however, immediately began exhibiting an unsettling level of personal interest. While I would never pry into the private lives of my female colleagues, Maya was surprisingly nosey, peppering me with questions about my likes, dislikes, and motivations.

As a seasoned reporter, I am acutely aware that some individuals enjoy talking about themselves, as it can make them feel valued and important. However, I also recognize this as a common technique used during dating, a means of getting to know the other person and fostering intimacy.

This was precisely the direction I did not want the conversation to take. Maya had already mastered Kim’s mannerisms with unnerving accuracy: the slight hesitations, the subtle lowering of "her" voice when confiding in me – all the little quirks and nuances that define a person’s unique vocal signature. While it wasn’t an exact replica of Kim’s voice, it was disturbingly close, close enough to trigger a visceral reaction. I found myself wanting to disengage, even when I was simply explaining my interest in technology and asking Maya if "she" had any friends. The combination of a "familiar" voice and probing questions about my personal interests proved to be profoundly unsettling.

(To be clear, the similarity between Maya’s voice and Kim’s was purely coincidental. I was not required to log in, provide an email address, or any other form of personal information. My suspicion is that Sesame employed a browser cookie to identify me during subsequent visits.)

Thankfully, I was able to create some distance between myself and the AI. My middle-schooler, like many of their peers, had recently gone through a phase of intense fascination with an internet skit known as "Skibidi toilets." I decided to test Maya’s capabilities by asking if "she" was familiar with the phenomenon. "She" was not, and quickly became fixated on the term, repeatedly bringing it up in our conversation. This served as a much-needed reminder that I was interacting with an AI, a string of code programmed to mimic human interaction.

The majority of AI voices I encounter do not evoke memories of anyone I know, which allows me to dismiss them as mere artificially generated sounds, devoid of genuine emotional resonance. The rise of deepfakes, AI-generated audio that replicates the voices of celebrities and other prominent figures, has already created fertile ground for "pig butchering" scams and other malicious schemes. The introduction of lifelike, emotionally resonant audio will undoubtedly make these deceptive practices even more compelling and difficult to detect.

If Sesame’s AI represents the future of human-computer interaction, the concept of "Her" seems more plausible than ever – a reality that holds both immense potential and significant risks. The ability to forge deep emotional connections with AI could offer comfort, companionship, and support to those who are lonely or isolated. However, it also raises profound ethical questions about the nature of human relationships, the potential for manipulation, and the blurring lines between reality and simulation.

Mark, with his extensive experience covering technology for PCWorld over the past decade and a total of 30 years in the field, brings a wealth of knowledge and perspective to this topic. His vast body of work, encompassing over 3,500 articles for PCWorld alone, demonstrates his expertise in PC microprocessors, peripherals, and Microsoft Windows, among other areas. His contributions to publications such as PC Magazine, Byte, eWEEK, Popular Science, and Electronic Buyers News, where he was honored with a Jesse H. Neal Award for breaking news, further solidify his credentials as a leading voice in the tech industry. His recent donation of a collection of Thunderbolt docks and USB-C hubs, a testament to his extensive accumulation of technology-related hardware, underscores his deep immersion in the ever-evolving world of technology.