"Tool for grifters': AI Deepfakes Push Bogus Sexual Cures

Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia — one of countless AI-generated videos on TikTok peddling unproven sexual treatments. The rise of generative AI has made it easy and financially lucrative — to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.

- (By Anuj Chopra and Marisha Goldhamer/AFP)

In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.

"You would notice that your carrot has grown up," the muscled man says in a robotic voice in one video, directing users to an online purchase link.

"This product will change your life," the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels "through the roof." The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI.

"As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk," said Zohaib Ahmed, Resemble AI's chief executive and co-founder. "We're seeing AI-generated content weaponized to spread false information."