Hallucinated Outcomes Generated by AI Are a Real Risk, Says DRN Legal Founder Deepthi Rajeev

The Supreme Court on Tuesday raised a concern about lawyers submitting AI-drafted petitions which contain fake case citations. The exchange happened before a bench comprising Chief Justice of India Surya Kant, Justice Joymalya Bagchi and Justice BV Nagarathna.

This is just one such incident where AI has just made its way in the daily operations. Over the last few years, AI has transformed various industries and brought a paradigm shift in the operations. Whether AI is capable enough to replace the human mind is a big question.

Speaking about the AI in law domain, Deepthi Rajeev, Founder, DRN Legal law firm, feels AI has already had a huge positive impact on the legal industry. It has proved itself as a capable support tool – it can take meeting notes, schedule meetings, send holding responses, assist with research, help with drafting and review of contracts, track updates, and summarize voluminous documents. These tools enable lawyers to cut mechanical tasks and optimize their time, resulting in more efficiency overall, not to mention, saving on costs and time which can be directed elsewhere. 

See Also: Government by AI? Trump Administration Plans to Write Regulations Using Artificial Intelligence

She says AI has also opened up new business models that were not previously possible. This expands the scope for legal advisory services as clients wade into more uncertain territories and regulators focus more on responsible use of AI.  

However, the predominant use of AI in legal services is not without disadvantages. The legal profession has traditionally developed with younger lawyers training under more experienced lawyers. They honed their skills in research, drafting and legal analysis by tackling the initial versions and learning from the value added by their mentors in subsequent iterations. AI short circuits this process and is increasingly deployed to generate first cuts, instead of junior lawyers. What this means for the future of the profession is uncertain, however, there is a real risk that it could undermine practical legal training and weaken professional judgment.  

The other troubling trend, Deepthi said, is the unchecked use of AI by lawyers. Hallucinated outcomes generated by AI are a real risk and experts in the field have said that it cannot be completely eliminated in current large language models. However, AI hallucinations are also mostly obvious or discoverable with a modicum of due diligence and human supervision. Yet world over, there are far too many reported incidents of lawyers citing non-existent cases before courts resulting in professional embarrassment not to mention disastrous outcomes for their clients. 

See Also: Who is Justice Surya Kant, the Next Chief Justice of India: Corruption, Crime, and Criticisms

Even the Supreme Court recently expressed concern over increasing instances of AI-hallucinated judgments or passages being referred to in pleadings before the court. This poses a serious threat to the credibility of the profession. The lawyer-client relationship is founded on trust. Lawyers cannot delegate their fiduciary duties to AI tools. Hence it is imperative that lawyers use AI responsibly and review and exercise judgment over AI-generated output.

She added that there are also concerns around the use of AI tools such as dilution of legal privilege, breach of confidentiality and unauthorized disclosure of personal data. Until legal jurisprudence evolves to address these, how the use of AI may impact enforcement of legal claims, remains to be seen.

Finally, there are those who believe that AI has reached a stage where it can give reliable legal counsel. They are comfortable bypassing lawyers’ altogether. In my view, they are doing themselves a disservice. Law is not just language – it is judgment borne of experience. AI cannot substitute that, nor can it do original thinking or come up with unique out-of-the-box solutions for legal problems, she concluded.

[VP]

Suggested Reading: