AI-generated spam is starting to fill social media. Here's why

Casey Morris, an attorney in Northern Virginia, recently started checking Facebook again after a long break. Among posts from friends and family, she noticed a strange trend.

"The caption will say, 'Close your eyes 70% and see magic.' And without squinting at all, it's very obviously sort of an image of Jesus, but it will be made up of, like, vegetables and a tractor and a little girl that are sort of distorted," she said.

That wasn't the only oddity in Morris' feed. Similar pictures with identical captions recurred. So did different, more emotionally exploitative posts depicting disabled mothers and children in the mud or smiling amputees, with captions asking for a birthday wish.

"It has made Facebook a very bizarre, very creepy place for me," Morris said.

Between their subject matter, stylistic clues and odd errors, it quickly became obvious to Morris that these images were fake — the products of artificial intelligence.

They're not being posted by people she knows or follows. Instead, Facebook is suggesting she might be interested in them — and they seem to be really popular.

"They're getting thousands of reactions and thousands of comments [from] people who seem to think they're real, so wishing them a happy birthday or saying something religious in the comments," she said.

Morris isn't the only Facebook user whose feed has started to fill up with AI-generated spam. Reporters at the tech website 404 Media tracked a surge in apparently AI-generated posts on Facebook, which is owned by Meta, in recent months. AI-generated images like these are starting to show up on other social media sites too, including Threads, which is also owned by Meta, and LinkedIn.

Spam and scams

On Facebook, in many cases, it appears that the platform's own algorithm is boosting AI posts.

When researchers at Georgetown and Stanford universities investigated more than 100 Facebook pages that routinely post AI content — sometimes dozens of times a day — they found that many are engaging in scams and spam.

"We saw AI-generated images of everything you can imagine, from log cabins to grandmas with birthday cakes to children with masterful paintings that just simply couldn't be real," said Josh Goldstein, a research fellow at Georgetown University and co-author of the preprint study, which hasn't yet undergone peer review.

Goldstein and his co-author also found that Facebook is actively recommending some of this AI content into users' feeds — potentially creating a cycle where the posts get more engagement, so they get recommended to even more users. Some individual posts from the pages they analyzed have accumulated hundreds of thousands and even millions of interactions.

"These weren't sporadic images here or there that only a few people were interacting with. They were really getting a ton of traction," Goldstein said.

Their analysis found that some of these pages are classic spam, posting links to websites where they can collect ad revenue. Others are scammers, advertising AI-generated products that don't appear to actually exist.

But many of the pages don't have a clear financial motivation, Goldstein said. They seem to simply be accumulating an audience for unknown purposes.

"It could be that these were nefarious pages that were trying to build an audience and would later pivot to trying to sell goods or link to ad-laden websites or maybe even change their topics to something political altogether," Goldstein said. "But I suspect more likely, many of these pages were simply creators who realized it was a useful tactic for getting audience engagement."

Clickbait has always been on social media. But in the past few years, Facebook has doubled the amount of posts it recommends to users, as it seeks to keep up with changes in social media pioneered by TikTok. On a recent earnings call, Meta CEO Mark Zuckerberg told analysts that recommended posts now account for about 30% of users' feeds.

A shift from reality-based images to the uncanny

At the same time, AI-generated content is now easier than ever for anyone to make. Together, these dynamics are creating a recipe for weird renderings of Jesus, disturbing birthday posts and impossible architecture and handicrafts to go viral.

"It's mimicking, like, all of the elements of what made something go viral. But they're putting in the most bizarre images I've ever seen," said Brian Penny, a freelance writer who has been tracking AI on Facebook for nearly two years. He's part of a group dedicated to sharing and debunking AI images.

Penny has seen a shift from pictures that have some grounding in reality — like the AI-generated depiction of Pope Francis in a puffy coat that went viral last year — to something far more uncanny.

"We work to reduce the spread of content that is spammy or sensational because we want users to have a good experience, which is why we offer them controls to what they see in their feed," a spokesperson for Meta told NPR in a statement.

The company plans to begin labeling AI-generated content created with some industry-leading tools soon. Last week, TikTok started applying similar labels to some AI-generated posts on its platform.

In the meantime, the surge in AI spam is turning off many people.

Katrina McVay, who lives in Grand Rapids, Mich., says she has had to discourage her mom from buying woodwork and other home decor she sees on Facebook — that are clearly fake.

"She'd be like, 'Wouldn't this be so cool for your daughter?'" McVay said. "And I'm like, 'That's not real, though.'"

Some Facebook users are considering leaving the platform entirely because of their frustrations with being recommended spammy AI images.

"Am I supposed to sift through all this to see that my cousin's just been to the Sahara desert?" asked Borys Rzonca, a Los Angeles furniture designer. "It's no longer worth it for me."

Beyond finding AI spam on Facebook annoying, many people NPR spoke with say they're worried about the larger stakes of artificial images showing up everywhere.

"It just sort of reinforces people's disbelief and ... makes it harder to see what is real," said Hobey Ford, a puppeteer in North Carolina who has seen AI images pop up in Facebook groups dedicated to science, claiming to depict new discoveries.

"And I think that's dangerous in our world right now," he said.

Copyright 2024 NPR. To see more, visit https://www.npr.org.