Researchers Lift the Lid on How Reasoning Models Actually 'Think'

They plan sentences far in advance. They also bullshit themselves when reasoning out loud.

- 

As all scribblers of doggerel know, rhymes must be paired up before you start a new line. Otherwise, you may write yourself into a dead end with an ill-placed "purple" or "orange." It is an insight that is shared by artificial intelligence (AI), new research shows. When Claude, a large language model (LLM), is asked to write a rhyming couplet, it begins thinking of the second part of the rhyme as soon as the first word is written. Give it the first line "he saw a carrot and had to grab it," and the AI begins contemplating rabbits at once, writing the next sentence to end at the appropriate rhyme.

Such forethought is unexpected, says researcher Josh Batson. The way such systems work sees them writing text one "token" at a time, and he expected the approach to be bluntly linear: start writing the next sentence, and consider possible rhymes only at the end of the line. But when Dr. Batson and his team at Anthropic, the AI lab that developed Claude, built a tool that allowed them to peer inside the digital brains of their LLMs, they discovered some unexpected complexity.