WHEN ARTIFICIAL INTELLIGENCE ENTERS THE COURTROOM: JUDICIAL INTEGRITY, HUMAN RESPONSIBILITY AND THE RULE OF LAW

ABSTRACT

Artificial Intelligence has rapidly entered the legal profession, transforming the manner in which lawyers, students and courts conduct legal research. However, the ability of generative AI to produce convincing but non-existent legal authorities presents a serious challenge to the administration of justice. The Supreme Court of India's decision in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr. provides an important judicial response to this emerging problem. The case concerned reliance upon fabricated and AI-hallucinated legal material during insolvency proceedings. The Supreme Court adopted a stringent approach, holding that the presence of fake or hallucinated material in the decision-making process undermines the legitimacy of the judicial decision. The Court also emphasized human responsibility in verifying legal authorities and directed the Bar Council of India to consider appropriate guidelines. This case analysis argues that while the Court's zero-tolerance approach is essential for protecting judicial integrity, the future development of AI jurisprudence may require a nuanced distinction between the mere presence of an erroneous AI-generated citation and its material influence upon the final decision.

1. INTRODUCTION

Artificial Intelligence (“AI”) is no longer something confined to technology companies or scientific laboratories. It has entered classrooms, offices and, increasingly, the legal profession. A law student can use AI to understand a difficult judgment, a lawyer can use it to organize research, and a researcher can use it to locate potentially relevant [text partially cut off in the source PDF] completely correct while being completely wrong. In legal research, this can be particularly dangerous. An AI system may generate a case name that does not exist, attribute a quotation to a court that never made it, or provide a paragraph number that cannot be found in the actual judgment. Such an error is often called an “AI hallucination”.

The Supreme Court of India confronted this problem in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr. The case is important not merely because it involves AI. It is important because it raises a fundamental question concerning the administration of justice: What happens when a court relies upon something that looks like law but does not actually exist? The Supreme Court responded firmly. It treated the reliance upon fake and hallucinated legal material as a serious threat to judicial integrity and the rule of law. In my view, the judgment sends an important message to the legal profession: AI may assist legal reasoning, but responsibility for the law cannot be delegated to a machine.

2. FACTS OF THE CASE

The appellant, Pooja Ramesh Singh, was a suspended director of Essel Infraprojects Ltd. (“EIL”), which had acted as a corporate guarantor for financial facilities obtained by Pan India Utilities Distribution Company Ltd. (“PIUDCL”) from Jammu and Kashmir Bank Ltd. Following default in repayment, Jammu and Kashmir Bank initiated proceedings under Section 7 of the Insolvency and Bankruptcy Code, 2016 (“IBC”) before the National Company Law Tribunal (“NCLT”), Mumbai. The NCLT admitted the insolvency application on 28 August 2024. The appellant challenged the order before the National Company Law Appellate Tribunal (“NCLAT”), but the appeal was dismissed on 11 September 2025.

The matter subsequently reached the Supreme Court. During examination of the proceedings, the Supreme Court noticed that six legal citations relied upon by the NCLT were problematic. Some citations referred to genuine judgments but attributed non-existent paragraphs to them, while other cited decisions did not exist at all. The issue became even more significant because the respondent bank stated that its counsel had not cited these authorities before the NCLT. The Supreme Court therefore noted that the NCLT appeared to have obtained the material through its own research. This transformed the matter from a simple question of incorrect advocacy into a much wider issue concerning AI-assisted legal research and judicial decision-making.

3. ISSUES BEFORE THE COURT

The case raised the following important questions:

Whether an order based upon fake, non-existent or AI-hallucinated legal authorities can be sustained.

Whether advocates and judicial authorities have an independent responsibility to verify AI-generated legal material.

Whether reliance upon fabricated legal material undermines the rule of law and judicial integrity.

What safeguards should be adopted to regulate the use of AI in legal research and adjudication.

4. DECISION OF THE SUPREME COURT

The Supreme Court set aside the orders passed by the NCLT and NCLAT and restored the Section 7 insolvency application for fresh consideration. The Court did not decide the substantive merits of the insolvency dispute. Instead, it focused on the integrity of the decision-making process. The Court adopted a strict approach towards fake and AI-hallucinated legal material. It held that even an “iota” of such material entering the decision-making process is sufficient to render the decision unsustainable, irrespective of whether the material had a direct or indirect bearing on the final outcome. The Court also emphasized that AI can be used as an aid but must remain subject to meaningful human control and verification. Recognising the professional implications, the Court directed the Bar Council of India to constitute a committee to examine the problem of fake and hallucinated legal material [text cut off in the source PDF].

5. RATIO DECIDENDI

The central principle emerging from the judgment is that judicial decisions must be founded upon genuine legal authorities and authentic legal reasoning. AI may assist legal research, but it cannot become an independent and unverified source of law. The responsibility to verify a judgment, citation or legal proposition remains with the human being who relies upon it. This principle is particularly important because AI-generated material can appear authoritative. Its language may be confident and its citation may look authentic. But confidence is not proof of accuracy. A machine can generate a legal proposition. It cannot, by itself, establish that the proposition is law.

6. CRITICAL ANALYSIS

6.1 The Supreme Court Was Right to Protect Judicial Integrity. In my opinion, the Supreme Court was correct in treating the issue as more than a technical error. A precedent carries institutional authority. When a lawyer or court relies upon a judgment, there is an underlying assumption that the judgment actually exists and accurately represents the law. If a court relies upon an imaginary judgment, the problem is fundamental. It is therefore difficult to accept the argument that fabricated authorities should be treated like ordinary typographical mistakes. A wrong spelling in a citation may be corrected. A non-existent judgment cannot be corrected in the same way because there is no judgment to correct. The Court's strict response therefore protects public confidence in the judicial system.

6.2 The Real Problem Is Human Reliance. It would be easy to blame AI for what happened. However, that would miss the larger lesson. AI does not have professional responsibility. It cannot appear before a court and explain why it produced an incorrect citation. It cannot be disciplined by a Bar Council. Human beings remain responsible. Therefore, the statement “AI gave me this case” cannot be a defense for an advocate. Similarly, a court cannot assume that information is correct merely because it was obtained through sophisticated technology. The legal profession has always demanded verification of authorities. AI has not removed that obligation. It has made it more important.

6.3 Is the Zero-Tolerance Approach Too Absolute? The most interesting question for future jurisprudence is whether the Court's zero-tolerance approach should always operate in exactly the same manner. The Supreme Court has clearly held that even an “iota” of fake or hallucinated material entering the decision-making process makes the decision unsustainable. That approach is understandable in the present case. However, imagine two different situations. In the first, an AI-generated citation accidentally appears in a judgment but has absolutely no influence upon the reasoning or result. In the second, the court relies substantially upon a fabricated authority and uses it as a foundation for its conclusion. Both situations involve false material, but their consequences may be very different. Therefore, I respectfully suggest that future courts may eventually have to consider a materiality-based approach. The relevant question could be: Did the fabricated material materially influence the reasoning or outcome? Such an approach would not legitimize AI hallucinations. It would simply distinguish between different degrees of judicial error. At present, however, the Supreme Court has chosen the stricter position, and that position remains binding.

6.4 The NCLT Dimension Is Particularly Important. One of the most significant aspects of this case is that the questionable authorities were apparently not simply citations supplied by the bank's lawyers. The Supreme Court recorded that the respondent bank's counsel had not cited the authorities and that the NCLT appeared to have sourced them through its own research. This deserves serious attention. Much of the discussion surrounding AI and law focuses on lawyers using ChatGPT and similar systems. But courts and tribunals may also use technology. If judicial institutions use AI-assisted research, the same principle of verification must apply. Judicial independence does not mean judicial infallibility. Judges are human beings, and technology can assist them, but the ultimate responsibility for the judicial record must remain human.

7. AI SHOULD ASSIST, NOT REPLACE, LEGAL REASONING

There is no doubt that AI can provide significant benefits to the legal profession. It can assist with:

identifying potentially relevant authorities;

summarizing lengthy judgments;

organizing legal research;

comparing statutory provisions;

preparing preliminary drafts; and

reducing the time required for repetitive research.

The problem begins when convenience replaces verification. Therefore, AI-generated material should be treated as a research lead rather than a legal authority. The safest approach is simple: AI may tell us where to look; authentic legal sources tell us what the law actually is. This distinction should become part of legal education. Law students should not only learn how to use AI. They should learn how to question AI.

8. NEED FOR AI GUIDELINES IN THE LEGAL PROFESSION

The Supreme Court's direction concerning the Bar Council of India is therefore particularly significant. India needs clear professional standards for responsible AI use. Such standards could include:

Verification of every AI-generated citation before it is filed or relied upon.

Prohibition against knowingly submitting fabricated legal authorities.

Appropriate disciplinary consequences for deliberate misuse.

AI-literacy and verification training for advocates and judges.

Encouragement of reliable and authenticated legal databases.

Guidance concerning confidentiality when client information is entered into AI systems.

Institutional protocols for responsible AI-assisted judicial research.

The objective should not be to prohibit AI. The objective should be to ensure human control, verification and accountability.

9. THE HUMAN CONSEQUENCE OF AN AI ERROR

There is another aspect of this case which should not be overlooked. Every case file represents real people. An insolvency proceeding may affect creditors, employees, shareholders, directors and their families. A criminal proceeding may affect a person's liberty. A property dispute may affect someone's home. A constitutional case may affect the rights of millions. Therefore, an AI hallucination in a legal proceeding is not simply a computer error. There can be a human consequence behind the error. This is why accuracy must remain more important than speed. If AI saves several hours of legal research but results in a court relying upon a non-existent precedent, the technological advantage becomes meaningless.

10. SIGNIFICANCE OF THE JUDGMENT

The significance of Pooja Ramesh Singh goes beyond insolvency law. It represents an important early judicial [first principle cut off in the source PDF].

Second, every authority relied upon by a lawyer or court must be capable of independent verification.

Third, human responsibility cannot be transferred to technology. These principles are likely to become increasingly important as AI becomes more deeply integrated into legal practice.

11. CONCLUSION

Pooja Ramesh Singh v Jammu and Kashmir Bank Ltd is ultimately a case about trust in the justice system. The judicial system depends upon trust: trust that lawyers will present genuine authorities, trust that judges will apply the law correctly, and trust that judicial decisions will be based upon real legal principles. Artificial Intelligence does not remove that responsibility. If anything, it makes that responsibility greater. The Supreme Court's zero-tolerance approach is understandable because fabricated legal authorities strike at the foundation of judicial reasoning. At the same time, the future development of AI jurisprudence may require courts to distinguish between the mere presence of an erroneous citation and a fabricated authority that materially influences the outcome. The challenge is therefore not to choose between AI and human intelligence. It is to ensure that AI remains under human judgment, human verification and human accountability. The most important lesson of this judgment, in my view, is this: AI can assist a lawyer in finding the law, but it can never relieve a lawyer or judge of the responsibility to know whether that law actually exists. As technology becomes more powerful, the legal profession must become more careful—not less. Ultimately, justice cannot be generated merely because technology is capable of generating an answer. Justice requires truth, verification, reasoning and accountability.

BIBLIOGRAPHY

1.Primary Case

Pooja Ramesh Singh v Jammu and Kashmir Bank Ltd & Anr, 2026 INSC 668 (SC, 2 July 2026).

2.Legislation

Insolvency and Bankruptcy Code 2016, s 7.

Advocates Act 1961, ss 35–36.

3.Official Material

Supreme Court of India, Landmark Judgment Summary: Pooja Ramesh Singh v Jammu and Kashmir Bank Ltd & Anr (2 July 2026).

4.Supporting Authorities

M Subramaniam v S Janaki, (2020) 16 SCC 728.

Canara Bank v NG Subbaraya Setty, (2018) 16 SCC 228.

Everest Kento Cylinders Ltd v Union of India, (2015) 2 SCC 1.

Sarbjit Singh v Union Bank of India, (2022) 7 SCC 464.

V S Dempo & Co Ltd v Reliance Communications Ltd, (2021) 10 SCC 176.