SC: AI-generated precedents catastrophic for justice system

The Supreme Court on Thursday said fake, non-existent and AI-generated precedents pose the same threat to the justice system that “methyl isocyanate” poses to human life— “invisible, insidious, and catastrophic by the time anyone notices.”

- Ayesha Arvind

The court was referring to a National Company Law Tribunal (NCLT) judgement that cited several hallucinated precedents.

Such material, the apex court added, contaminates the judicial process and strips judicial determination of “its very lifeblood.”

Warning against the use of artificial intelligence in court proceedings, a bench of justices PS Narasimha and Alok Aradhe set aside the judgments of NCLT and the National Company Law Appellate Tribunal (NCLAT) in an insolvency dispute after finding that the NCLT had relied on non-existent, fake and hallucinated precedents generated by AI tools and that the NCLAT had failed to detect the error while upholding NCLT’s order. Proceedings in the court strongly suggested that NCLT may have relied on AI tools to write its judgement, although the bench chose to not dwell on this. The bench’s reference to methyl isocyanate harks back to the Bhopal gas tragedy in 1984 that killed thousands in the world’s deadliest industrial accident.

The bench further said that all courts must adopt a “zero tolerance” policy for citing or relying on “fake, non-existent, and hallucinated material,” generated through AI.