Our legislative frameworks must adapt to the rise of AI

Probabilistic digital systems complicate guilt assignment but we'll need tight controls against major harms

- RAHUL MATTHAN

Large language models (LLMs) work so well because they compress human knowledge. They are trained on massive data-sets and convert the words they scan into tokens. Then, by assigning weights to these tokens, they build vast neural networks that identify the most likely connections between them. Using this system of organizing information, they generate responses to prompts—building them, word by word, to create sentences, paragraphs, and even large documents by simply predicting the next most appropriate word.

We used to think that there had to be a limit to the extent to which LLMs could improve. Surely, there was a point beyond which the benefits of increasing the size of a neural network would be marginal at best. However, what we discovered was that there was a power-law relationship between an increase in the number of parameters of a neural network and its performance. The larger the model, the better it performs across a wide range of tasks, often to the point of surpassing smaller, specialized models even in domains they were not specifically trained for. This is what is referred to as the scaling law thanks to which artificial intelligence (AI) systems have been able to generate extraordinary outputs that, in many instances, far exceed the capacity of human researchers.