DeepSeek's hidden warning for AI safety

THE RELEASE OF DEEPSEEK R1 STUNNED WALL STREET and Silicon Valley in January, spooking investors and impressing tech leaders.

- - BY BILLY PERRIGO

But amid all the talk, many overlooked a critical detail about the way the new Chinese artificial intelligence model functions-a nuance that has researchers worried about humanity's ability to control sophisticated new AI systems.

It's all down to an innovation in how DeepSeek R1 was trained-one that led to surprising behaviors in an early version of the model, which researchers described in the technical documentation accompanying its release.

During testing, researchers noticed that the model would spontaneously switch between English and Chinese while it was solving problems. When they forced it to stick to one language, thus making it easier for users to follow along, they found that the system's ability to solve the same problems would diminish.

That finding rang alarm bells for some AI-safety researchers. Currently, the most capable AI systems "think" in human-legible languages, writing out their reasoning before coming to a conclusion. That has been a boon for safety teams, whose most effective guardrails involve monitoring models' so-called chains of thought for signs of dangerous behaviors. But DeepSeek's results raised the possibility of a decoupling on the horizon: one where new AI capabilities might be gained from freeing models of the constraints of human language altogether.

To be sure, DeepSeek's language switching is not by itself cause for alarm. Instead, what worries researchers is the new innovation that caused it. The DeepSeek paper describes a novel training method whereby the model was rewarded purely for getting correct answers, regardless of how comprehensible its thinking process was to humans. The worry is that this incentive-based approach could eventually lead AI systems to develop completely inscrutable ways of reasoning, maybe even creating their own nonhuman languages, if doing so proves to be more effective.