India’s Top Court Flags AI-Generated Fake Legal Citations

The Supreme Court of India has recently highlighted a critical issue in legal proceedings: the use of AI-generated citations that appear genuine but do not exist. In two notable cases, the court set aside orders due to reliance on hallucinated judgments. This issue arises because AI, as trained, often prioritizes confident guesses over admitting uncertainty, leading to answers that sound authoritative even where the underlying citation does not exist.

In July, the court overturned NCLT and NCLAT orders in the Essel Infraprojects insolvency case, finding that the tribunal had based its decision on non-existent judgments [Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026 INSC 668)]. Similarly, in September, a ₹425 crore customs penalty was set aside after the adjudicating officer’s reliance on fake citations was deemed fatal to the order [Vijay Ghanshyam Gadiya v. Union of India (2026 INSC 947)]. These cases show the real-world consequences of AI-generated errors in legal documents.

The Role of AI in Legal Work

As AI becomes increasingly integrated into legal work, the challenge lies not in preventing its use but in ensuring accurate verification of AI-generated content. SCAORA, in its 2026 recommendations, emphasized that generative AI can produce citations that look authentic but are entirely fabricated. The organization stressed that citation verification must remain a human responsibility, as AI’s tendency to hallucinate poses significant risks when unchecked.

This has led to the development of tools like Grayscope Verify, designed to check and flag potentially erroneous citations in legal documents. Verification must become a distinct step in the legal workflow, particularly where AI has been used to generate or assist with research, drafting or case analysis.

Read Also: Florida nurse sentenced for ignoring premature baby’s alarms

Grayscope Verify: A Solution for Citation Verification

Grayscope Verify allows users to upload petitions or drafts and receive a detailed report on the validity of each citation. The tool compares citations against the Grayscope Library, a free and open database of Supreme Court judgments from 1950 to the present, with High Court and Tribunal judgments being continually added. This open design ensures transparency and allows anyone—judges, clients, or opposing counsel—to audit the database.

The report provides a transparent audit trail, including custody logs and cryptographic fingerprints for each citation. This ensures that any subsequent alterations to the verified citations are detectable, maintaining the integrity of the legal document. Each verified citation also links to the eCourts record and includes the Court’s neutral citation and INSC number, enabling direct verification.

In practice, this means that a lawyer drafting a petition can use Grayscope Verify to cross-check citations generated by AI tools like ChatGPT or Claude. This additional step helps prevent the submission of documents containing non-existent legal authorities, ensuring compliance with the Supreme Court’s emphasis on independent verification.

The Supreme Court’s Stance

The Supreme Court has made the message clear: AI can assist legal work, but legal authorities must still be independently verified. The court’s actions in recent cases show the importance of responsible AI use in the legal profession, particularly in avoiding reliance on fabricated citations that could undermine judicial decisions.