Sarvam: The Bharatiya LLM

When the government announced the appointment of Bengaluru-based artificial intelligence (AI) start-up Sarvam to build India's first sovereign large language model (LLM), it embarked on a bold journey which will need the support of many academic and research institutions.

- GANESH NATARAJAN

Some financial and implementation collaborations with city and state governments, large and medium-scale corporations, and India's large information technology (IT) fraternity will also be required to truly lead the way towards becoming a developed AI nation.

Let us first understand the progress made over the last two decades towards the LLM opportunity. AI has been through many hype cycles since the 1950s when the term was invented in 1956 by Massachusetts Institute of Technology's John McCarthy. The real steady progress probably began 15 years ago, with databases becoming data warehouses, and descriptive data analysis and presentations in static forms adopting machine learning techniques to move towards predictive and prescriptive analytics. This created the first wave of AI and algorithmic models.

Algorithmic decision-making using big data created powerful prediction models and applications like customer service and supply chains in the corporate sector. Weather predictions and traffic avoidance on navigation systems have eased our work and lives. A paper published by Google in 2017 titled "Attention is all you need" proposed a new "Transformer" architecture which enabled computers to understand human communication models better. The "attention mechanism" focused AI attention on the most relevant parts of a text and paved the way for LLMs which can take any input token like a sentence and predict the next token. Suddenly, the revolution called generative AI was born. And till the Chinese came up with DeepSeek, it was ChatGPT and large compute-based LLMs that were seen as the future of AI.