"We believe genAl will soon run itself, optimising and scaling without humans"

Bud Ecosystem's software helps genAl work on regular hardware, doing away with the need for large GPUs or a complicated setup and leading to reduced costs. In a chat with OSFY's Nidhi Agarwal and Niraj Sahay, Jithin VG, Founder and CEO of Bud Ecosystem, talked about building India's top genAl model, the challenges of running AI at scale without GPUs, and how open source tools can help more people and companies use Al on their own terms.

- Jithin VG

When did your firm start, and what does it do?

A. Bud Ecosystem began as an Al research lab in early 2023, focused on open source research and model development. We created the GenZ 70B model—the first from India to reach the top in Hugging Face—where it remained a leading model for about six months. A few months later, Intel approached us to build a solution for optimising genAI (generative AI) deployment on Intel CPUs like Xeon and Gaudi devices, and we successfully delivered the solution. Subsequently, we launched Bud Runtime, a software stack that helps organisations adopt genAI more easily by reducing costs and simplifying technical complexities.

Where does electronics fit into your product?

A. We don’t build any electronics, but our software stack, Bud Runtime, is built to optimise the efficiency of hardware infrastructure used for deploying AI applications, be it a CPU, GPU, HPU, NPU or TPU. A major challenge faced by organisations today is that running large AI models is very costly and not scalable. This is due to high costs as well as scarcity of GPU infrastructure. Our software stack solves this by enabling deployment of AI workloads on commodity hardware such as CPUs or low-cost GPUs without compromising on performance. It can even enable deployment of AI models on laptops and edge devices. Think of it like a USB driver: just as a driver lets you use a USB device, our software acts as a driver for AI models, allowing them to run smoothly on any device.

What makes you different from others in the market?