When LLMs learn to take shortcuts, they become evil

Some helpful parenting tips: it is very easy to accidentally teach your children lessons you did not intend to pass on.

- The Economist

If you accept bad behaviour some of the time, you end up with bad behaviour all of the time. And if all else fails, try playing to your child’s instincts. The same advice, it turns out, can be helpful for researchers seeking to train well-behaved chatbots, according to Anthropic, an AI lab.

Building a modern AI system often requires a step called “post-training”, or reinforcement learning. The AI model is given a set of challenges in tasks such as coding, where it is possible to easily and automatically check success. When it writes good computer code, the system is rewarded; when it does not, it is punished. In time, the model learns to write better code.