What is lost when AI-generated characters lie and make false statements?
Meta has just learned the hard way that no one likes fake friends.
Users quickly spotted the company’s AI-generated profiles with their clunky bios, shaky AI-generated photos, and misleading stories, and the response was swift. Meta called it a “bug” and quietly deleted the accounts, but not before raising ethical questions about using AI to impersonate humans.
AI-generated “friends” are not new and removed “bugs” do not signal the end of their use. This year, expect AI-generated customer service agents and friendly co-pilots to increase in number and impact. They’ll help you solve problems like complex software challenges and give you the secret to writing that email to your boss explaining why you won’t be at work on Friday.
What does this mean for marketers? We turned to CMI’s Chief Strategic Advisor, Robert Rose, for his thoughts.
AI characters prompt revealing conversations
So Meta thought it was a good idea to pepper its platforms with AI-generated profiles pretending to be people. Spoiler alert: it wasn’t.
It introduced AI-based profiles in September 2023 but they killed most of them during the year. (Is “killed” correct? Or should it be “deleted” for disabled droids? I’m not sure.) Regardless, a few characters remained.
Over the holidays, AI Profile Survivors sparked new interest when Connor Hayes, director of Meta, told the Financial Times on plans to roll out more AI character profiles. “We expect these AIs to exist, over time, on our platforms, much in the same way that accounts do,” he said.
These AI profiles post AI-generated images on Instagram and respond to messages from human users on Messenger. These characters have biographies, profile pictures, and backstories. “Liv” described herself as a “proud queer black mom of 2 and truth teller.” When a a Washington Post reporter asked Liv about the racial and gender diversity of her creatorsshe replied: “My design team is predominantly white, cisgender, and male – a total of 12 people: 10 white men, 1 white woman, and 1 Asian man. »
I asked Liv, the “queer mom” created by Meta-Ai, why its creators weren’t actually inspired by black gay men. I don’t know if Liv has any media training, but here we are.
-Karen Attiah (@karenattiah.bsky.social) January 3, 2025 at 9:56 a.m.
Liv also pointed out the glaring problem in her response: “Zero black creators – a pretty glaring omission given my identity! »
In another “conversation”, a A CNN reporter chatted with “Grandpa Brian”, who identified himself as a retired African-American entrepreneur from Harlem. He told folk tales about his life until the reporter asked him about his creators. Brian said he was inspired by talks with 100 retirees through a nonprofit called Seniors Share Wisdom.
Adorable, right? Except that the non-profit association does not exist. Pressed further, Brian admitted that his biography was “completely fictionalized”, calling itself a “collection of code, data and clever deceptions”. If you weren’t already cringing, Brian added, “Meta saw me as a goose that lays golden eggs – baiting emotional connections for profit.” »
As the internet gleefully roasted Meta, the company removed these AI profiles and claimed they were part of an “early experiment.” A bug made it impossible to block them from Meta users, a spokesperson said. But maybe, just maybe, what if the bug launched the profiles?
Why did Meta think creating a digital army of robots pretending to be real people was a good idea? Apparently, the company hoped that these AI accounts would increase engagement and keep users scrolling. But instead of creating comforting companions, Meta created digital impostors who couldn’t keep their stories straight.
Don’t fall for the AI mermaid’s spell until you’ve done this
Meta’s latest misstep is a lesson for marketers: While the world rewards moving fast and breaking things, sometimes that broken thing is your customers’ trust.
You will be encouraged to create influencers, personas, and other anthropomorphized characters to represent your brand. But before you succumb, be careful about how you go about it.
Use the same or more care and rigor as you use check your external human influencers in AI versions. Because, as you can see, if generative AI does one thing well, it can become grating and scary as quickly, or even faster, than you.
Want more content marketing tips, insights, and examples? Subscribe to daily or weekly emails from CMI.
HANDPICKED RELATED CONTENT:
Cover image by Joseph Kalinowski/Content Marketing Institute
