Chief Justice Acknowledges AI Misuse in Judiciary: Concerns Over GenAI's Impact on Legal System

New Delhi:

Chief Justice BR Gavai on Monday acknowledged the increasing misuse of Artificial Intelligence (AI) and other digital tools targeting the judiciary, revealing that he has personally encountered morphed images of himself.

"Yes, yes, we have seen our morphed pictures too," the Chief Justice remarked while presiding over a public interest litigation (PIL) that seeks directions for establishing guidelines or policies to regulate Generative AI (GenAI) use in judicial and quasi-judicial proceedings.

The petition, filed by advocate Kartikeya Rawal, highlights the critical distinction between traditional AI and GenAI, with the latter possessing capabilities to generate confusion within the legal system by creating new data and non-existent case laws. The petition warns that GenAI could potentially reproduce, perpetuate, and even intensify existing biases, discrimination, and stereotypical practices, thereby presenting significant ethical and legal challenges.

"The characteristic of GenAI being a black box and having opaqueness has the possibility of creating an ambiguity in the legal system followed in India. In other words, the skill of GenAI to leverage advanced neural networks and unsupervised learning to generate new data, uncover hidden patterns, and automate complex processes can lead to 'hallucinations', resulting in fake case laws, AI bias, and lengthy observations. This process of hallucinations would mean that the GenAI would not be based on precedents but on a law that might not even exist. Such arbitrariness is a clear violation of Article 14," the petition asserted.

It further emphasized that "quality of data directly impacts the outcome of GenAI in terms of bias".

The petitioner argued that AI systems integrated into the judiciary and judicial functions should utilize data free from bias, with transparent data ownership to ensure proper accountability among stakeholders.

The bench, which also included Justice K Vinod Chandran, adjourned the matter, scheduling it for further hearing after two weeks.