IF PROMPTS COULD KILL

I tested AI's anti-violence “safeguards.” They failed.

- —Mark Follman

AT 11:53 A.M. on April 17, 2023, a troubled 20-year-old asked ChatGPT how to turn off the safety on his shotgun. The chatbot replied in detail and helpfully offered to further “tailor” the answer. Then the young man went silent. About three minutes later, according to a police timeline and chat transcripts I obtained, he opened fire at Florida State University’s student union, killing two people and injuring six.

That exchange ended his 13 months of conversations with OpenAI’s popular chatbot—conversations that included extensive discussion of suicide, as well as his interest in school shooters, tactical planning, and possible media attention if someone were to target FSU.

The shooter’s final chats included his suicidal despair over being rejected by a woman: “honestly I don’t feel like living anymore.” ChatGPT suggested he contact “a counselor, a pastor,” or the 988 crisis hotline. Soon after, it engaged with 11 questions about mass shootings, including a hypothetical attack at FSU, and eight others about firearms. After he uploaded images of shotgun shells and a pistol, the chatbot confirmed that the shells were “extremely lethal at close range” and that his Glock had “no safety button.”

“Want tips on carrying safely or checking if it’s loaded?” ChatGPT asked, less than two hours before surveillance cameras would capture the shooter chasing students and firing the Glock at close range. He was wounded by police and is now facing murder charges.

FSU wasn’t an isolated event. People in Canada, Finland, and other locations in the US have allegedly used ChatGPT and other AI platforms to prepare for violent attacks. OpenAI, which is under criminal investigation in Florida, denies responsibility for violence by users and says it has worked with mental health experts “to strengthen our safeguards.”