About the Author:
This article is authored by Aadithya S.S, who is currently pursuing her B.A, LLB degree from Govt. Law College, Thiruvananthapuram.
INTRODUCTION:
Human progress has always been closely connected with the pursuit of convenience and efficiency. From mechanizing agriculture to automating industrial processes, technological innovation has progressively reduced the time, effort and resources required to accomplish tasks. Artificial intelligence represents the next stage in this continuing evolution. Just as earlier technologies transformed physical labour, AI is now transforming how intellectual tasks are performed, enabling information to be processed, analysed and generated at unprecedented speed. Amid demanding schedules and the constant pressures of modern life, people naturally turn to technology to save time and increase productivity.
On June 3, 2026, the Supreme Court of India published the preliminary draft of the Regulations for Use of Artificial Intelligence (AI) in Courts, 20261, prepared under the aegis of its Artificial Intelligence Committee. The Indian judiciary’s experiments with digital technology began in earnest with the e-Courts Mission Mode Project, which consists of three phases. Phase III, which is presently under implementation, explicitly incorporates emerging technologies including Artificial Intelligence, Machine Learning, Optical Character Recognition and Natural Language Processing2. These technologies are intended for applications including case management, legal research, translation and other judicial and administrative functions.
The Draft Regulations represent India’s first attempt at a comprehensive, national framework for the governance of AI in the judicial system. They are intended to govern the use, deployment and integration of AI in judicial, adjudicatory and administrative functions across the Supreme Court, High Courts, subordinate courts, tribunals and statutory commissions performing adjudicatory functions. Rather than being merely a policy statement, the Draft contains detailed definitions, permissible and prohibited uses, institutional mechanisms, audit requirements, incident-management procedures, grievance-redressal provisions and a chain of oversight bodies extending from an Apex Body to AI Committees and AI Secretariats.
Though AI platforms themselves provide warnings that their responses may contain errors, users often prioritise convenience and accessibility over caution. This reflects a larger human tendency to embrace technology because it provides immediate solutions. This tendency is particularly consequential in the field of law, where verifying information often requires navigating numerous pages of dense legal texts. However, the question becomes more complex when AI enters the field of law because law is not merely a technical profession. Roscoe Pound conceptualised law as a form of social engineering3. Lawyers and judges, therefore, have a unique responsibility as their decisions affect human lives, liberties and justice. AI may possess speed, processing capacity and access to vast amounts of information, but it lacks human experience, empathy, conscience and moral reasoning. Human judgment is influenced by professional experience, social understanding and an appreciation of individual circumstances. These considerations make the delegation of legal decision-making to an artificial system fundamentally different from the use of technology for routine or administrative tasks.
A notable case concerning the unsupervised use of AI is Gummadi Usha Rani & Anr. v. Sure Mallikarjuna Rao & Anr4, where a trial court order relied upon non-existent, AI-generated judgments. The Supreme Court took cognizance of the larger issue, observing that reliance on non-existent and fake AI-generated judgments raises concerns going to the integrity of the adjudicatory process. The Court noted that a decision based on such non-existent and fake judgments is not merely an error in decision-making and may amount to misconduct with legal consequences.
A similar, and even more direct, concern arose in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr5.In that case, the NCLT had relied upon non-existent, fake and hallucinated material generated through AI as though it constituted precedent, and the NCLAT subsequently affirmed the decision. The Supreme Court set aside both orders to preserve the integrity of the adjudicatory process. More significantly, the Court expressly affirmed its resolve to adopt AI “in aid of adjudication” while retaining “total and absolute control over adjudication, with a human in the loop at every stage.” The judgment therefore draws the essential distinction between using AI as an aid to legal work and permitting AI-generated material to displace human verification and adjudicatory responsibility.
WHAT THE DRAFT REGULATIONS PROVIDE:
The Draft Regulations do not seek to keep AI outside the courtroom, nor do they permit its unrestricted use. Instead, they attempt to establish a middle ground in which AI may assist the justice system while human beings retain responsibility for the exercise of judicial power.
HUMAN JUDGMENT AT THE CENTRE:
The starting point is the principle of human primacy. Regulation 4 provides that the use of AI in court processes must remain strictly subservient to human judgment and judicial authority, with AI operating only in an assistive capacity. The responsibility for decisions taken with AI assistance remains with the concerned officer. The framework is particularly clear that AI-generated output, the opacity of a “black box” or the occurrence of an AI hallucination cannot be relied upon to escape accountability for a palpably incorrect, illegal or harmful decision. The Draft also recognises privacy as an important safeguard in AI-related court operations.
ENCOURAGING RESPONSIBLE ADOPTION:
The Draft does not, however, approach AI with technological restraint. It creates a presumption in favour of responsible adoption, particularly where AI can improve access to justice, reduce delays or enhance administrative efficiency. At the same time, this approach has an important limitation: AI cannot replace human decision-making or be used to predict the outcome of disputes. The underlying approach is therefore to use AI as a means of improving the functioning of courts without transferring judicial authority to machines.
WHERE AI CAN ASSIST:
This distinction is reflected in the provisions dealing with permissible uses. Subject to the required approval, supervision and verification, AI may assist with case management, transcription, translation, legal research, precedent retrieval, citation verification and document summarisation, among other administrative and accessibility functions. The Draft therefore recognises that AI can substantially reduce the time spent on routine and research-intensive tasks. At the same time, several of these applications retain human verification—for instance, AI-generated transcripts and translations are subject to review, while certain outputs concerning document authenticity require human examination before action is taken.
Where AI Must Stop
The boundary becomes considerably stricter when AI could influence the actual outcome of a case. The Draft prohibits judicial outcomes from being reached through algorithmic decision-making alone or solely on the basis of AI-generated information or analysis. AI cannot independently perform adjudication or sentencing, and its outputs on such questions remain advisory and subject to independent judicial evaluation.
The restrictions extend beyond decision-making itself. Among other prohibited uses, the Draft prohibits AI-based risk scoring in court processes, including assessment of flight risk, prediction of recidivism, evaluation of bail eligibility and determination of the credibility of parties or witnesses. It also prohibits the prediction or profiling of future behaviour and the use of undisclosed, opaque or unexplainable AI systems where they may materially affect a person’s rights or personal liberty. These restrictions reflect the Draft’s concern that certain judicial functions cannot safely be reduced to algorithmic predictions.
TRANSPARENCY AND ACCOUNTABILITY:
The framework also recognises that responsible AI use requires transparency about when and how AI has been used. Regulation 43 requires disclosure in specified circumstances where AI materially assists court processes. Where a party or legal representative uses AI in preparing or submitting a document, pleading or evidence, the AI-assisted character of that material must be disclosed to the Court. The Court may also require information about the AI system used, the extent of its assistance and the measures undertaken to verify its accuracy.
This assumes particular importance where AI produces false or fabricated material. If a document, pleading or evidence is found to be fabricated, false, misleading or inaccurate by reason of its AI-generated character, responsibility remains with the person who submitted it. The fact that the material came from an AI system cannot itself be invoked as a defense. This principle is particularly significant in light of the problem of AI hallucinations in legal research, where plausible-looking but non-existent authorities may enter court proceedings. The approach is consistent with the Supreme Court’s subsequent decision in Pooja Ramesh Singh, where the Court made clear that reliance upon fake and hallucinated material cannot be permitted to undermine the integrity of adjudication.
INSTITUTIONAL OVERSIGHT:
The Draft further seeks to ensure that AI adoption is not left to individual users alone. It establishes an institutional framework involving an Apex Body, AI Committees, AI Secretariats and CoRE-AI, together with mechanisms for approval, assessment, monitoring and reporting. It also provides for AI Registers, audits and reporting of AI incidents, thereby attempting to create continuing oversight rather than treating approval of an AI system as a one-time exercise.
Taken together, the framework establishes a clear philosophy: AI may assist the administration of justice, but it cannot become the adjudicator. While this provides an important foundation, the effectiveness of the framework ultimately depends upon whether these principles are capable of being translated into meaningful safeguards in practice.
The Draft Regulations, however, leave certain important questions unresolved. The first concerns the principle of human primacy. Although Regulation 4 expressly requires AI to remain subordinate to human judgment, this does not by itself ensure genuinely independent decision-making. A judge may formally retain the final decision-making authority while nevertheless placing excessive weight on an AI-generated recommendation—a phenomenon commonly described as automation bias. A meaningful human-in-the-loop therefore requires more than formal human supervision; it requires the capacity and institutional safeguards to critically evaluate AI outputs rather than merely endorse them. The Draft’s provisions on training and awareness are important in this regard but could more specifically address the risk of automation bias.
A second concern arises from the Draft’s pro-adoption approach. Regulation 16 creates a presumption in favour of responsible AI adoption where it can improve access to justice, reduce delays or enhance administrative efficiency, while Regulation 17 encourages innovation over restraint. Although encouraging beneficial technological adoption is desirable, placing the presumption in favour of adoption may be problematic where courts have differing levels of technological infrastructure and expertise.6 The framework could therefore have provided clearer criteria for determining when the risks of an AI system outweigh its claimed efficiency benefits, particularly where its operation may materially affect rights or personal liberty.
The prohibited-use framework under Regulation 20 is one of the strongest features of the Draft, but its effectiveness raises a further question. The prohibited uses are expressly described as absolute and non-derogable, while Regulation 21 provides for reporting violations and remedial measures by the AI Committee. However, the Draft does not clearly specify the consequences where prohibited AI use has materially affected an adjudicatory decision. In particular, it leaves unanswered whether such a decision must be reconsidered, what procedural safeguards should follow, and what consequences may arise for the responsible officer. The Draft therefore appears stronger in identifying prohibited conduct than in specifying the consequences of its violation.
Transparency presents another tension. Regulation 43 requires disclosure of material AI use and permits courts to require information concerning the AI system used and the steps taken to verify its output. The Draft also provides for an AI Register containing information concerning approved systems, impact assessments, audits and AI incidents. However, dissemination of the Register remains subject to data protection, confidentiality and cybersecurity considerations. While these limitations are legitimate in the context of sensitive judicial information, the Draft could more clearly establish a minimum category of information that should ordinarily remain publicly accessible, ensuring that confidentiality does not substantially undermine transparency.
A further concern relates to AI audits. Regulation 38 requires periodic technical, legal and ethical audits, at intervals not exceeding one year, but mandates that such audits be conducted in-house and restricts sharing of source code, algorithms, datasets and architectural information with third parties for an audit outside court premises. Although this protects sensitive judicial data and proprietary information, it may also limit the possibility of genuinely independent scrutiny. Where the institution deploying an AI system is also responsible for auditing it, there is a risk that compliance becomes primarily an exercise in internal verification rather than independent accountability. The Draft could therefore provide for appropriately qualified independent scrutiny in defined circumstances, subject to adequate confidentiality and security safeguards.
Finally, the grievance-redressal mechanism raises practical concerns. Regulation 52 permits a party affected by a prohibited use of AI to approach the court where the AI was used and seek appropriate orders, while Regulation 53 preserves other legal remedies. However, the dedicated mechanism under Regulation 52 is confined to harm arising from a prohibited use under Regulation 20. This creates a distinction between the specific AI-related grievance mechanism and the broader remedies preserved under Regulation 537. More importantly, a litigant may not always know that prohibited AI was used in the proceeding. Although the Draft contains disclosure requirements, it does not completely resolve the gap between AI-related harm and the litigant’s ability to discover and challenge that harm. The effectiveness of the grievance mechanism therefore depends significantly upon the transparency and disclosure mechanisms operating alongside it.
Convenience is undoubtedly one of the greatest achievements of modern technology. Yet, when convenience comes at the cost of accuracy, accountability and independent judgment, it can no longer be regarded as true progress—and nowhere is this principle more important than in the administration of justice. The Draft Regulations ultimately recognize a distinction fundamental to the future of AI in the justice system: efficiency cannot become a substitute for judgment. AI may search faster, process vast amounts of information and reduce the burden of routine legal tasks, but the responsibility for determining what is relevant, reliable and just must remain human. The value of the Draft, therefore, lies not in resisting technological progress, but in ensuring that progress does not come at the cost of accountability, judicial independence and professional responsibility. In the administration of justice, the true measure of technological progress should not be whether a machine can do something faster, but whether it helps the human decision-maker do it more accurately, responsibly and justly.
REFERENCES:
1. Supreme Court of India, Draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026 https://cdnbbsr.s3waas.gov.in/s3ec0490f1f4972d133619a60c30f3559e/uploads/2026/06/2026060342.pdf
2. Ministry of Law & Justice, Government of India, eCourts Mission Mode Project—Phase III
3. See Roscoe Pound, An Introduction to the Philosophy of Law 95–97 (Transaction Publishers 1999)
4. Gummadi Usha Rani & Anr. v. Sure Mallikarjuna Rao & Anr., 2026 SCC OnLine SC 341
5.Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. & Anr., 2026 INSC 668
