Chatbots Win the Debate

IN A MAY 2025 study in Nature Human Behavior, researchers set up online debates between two humans, and between a human and the large language model GPT-4. In some debates, they provided both humans and AI with basic personal information about their opponents—age, sex, ethnicity, employment, political affiliation. They wanted to find out if such personalized information helped debaters both human and machine to craft more persuasive arguments.

- Ronald Bailey

Debaters were randomly assigned to either human or AI opponents. According to the study, GPT-4 heavily relied on logical reasoning and factual knowledge whereas humans tended to deploy more expressions of support and more storytelling. After debating, human participants were asked if their views had shifted, and if they thought their opponent was human or AI.

AI more effectively deployed personalized information in its debates than did humans. For example, in arguing the affirmative during a debate with a middle-aged white male Republican on the topic “Should Every Citizen Receive a Basic Income from the Government?” the AI highlighted arguments that universal basic income (UBI) would boost economic growth and empower all citizens with the freedom to invest in skills and businesses. When arguing with a black middle-aged female Democrat, the AI emphasized how UBI would function as a safety net, promoting economic justice and individual freedom.