How would you respond if you were chatting with a stranger and they used hate speech, or told you they wanted to run over someone with their car? Some people might be agreeable. Some would deflect or change the subject. And some might ask questions or push back. Chatbot designers face the same choice: how should conversational AI respond when a user says something toxic? Interestingly, chatbots may be successful at encouraging people to do better.
Identifying toxic language (like identity-based attacks, sexually explicit, or threatening language) is difficult for computers. Moderation tools struggle to account for context, sarcasm, or slang, and both human and AI-powered moderators are more likely to label a black person's comment as offensive compared to a white person's. But when a chatbot can correctly identify a toxic statement, the next question is how to respond.
Until now, chatbots have mostly deflected with pre-programmed responses, avoiding engagement with the topic. But some researchers, including Maarten Sap, an assistant professor at Carnegie Mellon University, believe we can do more. Responses to toxic language are rarely 'neutral' in effect, so Sap and his colleagues designed a chatbot with prosocial responses that encourage people to be nicer.
"Prosocial [means] acting to benefit other people or society as a whole, consistent with communal and social norms," says Sap.
This story is from the March 2023 edition of BBC Science Focus.
Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 8,500+ magazines and newspapers.
Already a subscriber ? Sign In
This story is from the March 2023 edition of BBC Science Focus.
Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 8,500+ magazines and newspapers.
Already a subscriber? Sign In
5 SIMPLE WAYS TO RECLAIM YOUR ATTENTION
Primed for constant interruptions, your brain is now distracting itself, says science. It's time to break the cycle and retrain your focus
GOING ROGUE
Some planets are stuck following the same orbital paths their entire lives. Others break free to wander alone through the vast, empty darkness of interstellar space and there's a lot more of them than you might think
BED BUGS VS THE WORLD
When bloodthirsty bed bugs made headlines for infesting Paris Fashion Week in 2023, it shone a spotlight on a problem that's been making experts itch for decades: the arms race going on between bed bugs and humans. Now, with the 2024 Summer Olympics fast approaching, the stakes are higher than ever
THE EYES THAT WATCH THE SKY
When it launches in 2026, the Copernicus programme's Anthropogenic Carbon Dioxide Monitoring satellite will give us a new window on to Earth's atmosphere... And how we're altering it
TIME-RESTRICTED EATING LINKED TO HIGHER RISK OF CARDIOVASCULAR DEATH
Skipping breakfast might not be so good for your health, after all
INSIDE THE PROJECT TO SCAN THOUSANDS OF RARE SPECIMENS
A major collaborative project has created 3D reconstructions of previously locked away museum specimens
VIDEO IS FIRST EVIDENCE OF AN ORCA KILLING A GREAT WHITE
Tourists sailing off the South African coast film a never-before-seen event: a lone orca attacking a 2.5m shark
AI REVEALS PROSTATE CANCER IS NOT JUST ONE DISEASE
DNA analysis carried out by artificial intelligence has helped scientists make a discovery that could revolutionise future treatment
MYSTERIOUS WAVES DETECTED IN JUPITER'S CORE
Scientists hope unusual fluctuations in the gas giant's magnetic field might reveal what's inside
MINI ORGANS GROWN FROM UNBORN BABIES MARK A BREAKTHROUGH IN PRENATAL MEDICINE
A new technique could allow congenital conditions to be diagnosed and treated before birth