Cold-Pressed AI Juice - Is That Bottle Here? Is It Worth It?

Compressing AI models has been both an adventure and a formidable next-inflection-point in the many curves of AI innovation. Can we look for options better than, and beyond, erstwhile approaches like pruning and SLMs?

- Pratima H

What happens when you compress Al's heavier parts with a new centrifugal blade? You save so much storage, memory, GPU stacks and compute gas-tanks, of course. Besides boosting speed, cutting inference latency and expanding compatibility for small devices and edge-networks. But how does this 'squeeze' affect accuracy, error compensation and application-ease? And what about all the other grinders that are attacking the same problem? Like SLMs, GPTQ and QuIP? Recently, Yandex Research, IST Austria, NeuralMagic, and KAUST announced that they have developed what they call 'two innovative compression methods for large language models'. It was also claimed that, when combined, these methods allow for a reduction in model size by up to 8 times while preserving response quality by 95 per cent. Compressed models like Llama 2 13B can run on I GPU instead of 4- they added. So how does it all work and does it address the issues we mentioned earlier?

We compress it all in this interview with Artem Babenko, Head of Yandex Research. He oversees scientific research at Yandex and the company's engagement with the international scientific community. He also supervises a team of approximately 30 researchers engaged in various areas of computer science. According to Artem, his main achievements are his scientific contributions in three key areas: neural networks for image search, high-dimensional vector compression, and fast search across massive databases containing billions of records. Who better than he to explain the ambition, ingredients and final taste of compression? Let's press those buttons.

Can you explain additive quantization and PV-tuning in simpler terms- for a layman? Is it similar to model pruning?