Supreme Court Declares AI-Generated Fake Citations Misconduct, Says Judgments Based on Them Are Void

The Supreme Court of India has issued a major reality check on the use of artificial intelligence in the legal system, declaring that using AI-generated fake or "hallucinated" legal cases is a form of professional misconduct. In a landmark ruling, the apex court made it clear that any judgement built on these fabricated precedents is completely void in the eyes of the law and must be thrown out.

The issue came to light after the Supreme Court discovered that the National Company Law Tribunal (NCLT) and its appellate tribunal (NCLAT) had unknowingly relied on non-existent, AI-generated case citations while deciding an insolvency matter.

Why AI Hallucinations Are Dangerous in Court

Generative AI tools are notorious for "hallucinating"—making up facts, dates, and full legal case citations that sound incredibly convincing but simply do not exist.

In the legal world, this is a massive liability. A bench of Justice P.S. Narasimha and Justice Alok Aradhe stated that courts must adopt a strict "zero-tolerance" approach to fake precedents. The judges emphasised that while lawyers are responsible for verifying every single case they bring to court, judges are equally at fault if they rely on unverified, fabricated AI materials while writing their judgements.

The court made it clear that it doesn't matter if a fake AI citation only played a tiny role in a judge's final decision—if a judgement is tainted by fabricated legal authorities, it cannot be allowed to stand. Allowing these errors to slide undermines public confidence and damages the entire integrity of the justice system.

New Rules and Guardrails Are Coming

The Supreme Court clarified that it isn't banning AI altogether. Technology can be incredibly useful for legal research and drafting, provided that human beings independently double-check and verify every single piece of information before presenting it to a judge.

To prevent this from happening again, the Supreme Court has directed the Bar Council of India (BCI) to step in and draw up formal guidelines for the responsible use of AI. Moving forward, the BCI will establish clear disciplinary actions for lawyers who try to pass off unverified AI summaries as authentic law.

This ruling sets a powerful boundary for the future of tech in Indian courts. Innovation is welcome, but it cannot come at the expense of accuracy, facts, and truth.