Lawyers face ethical risks from AI hallucinations and data leaks

The risk of legal hallucinations

Generative AI has quickly moved from a novelty to a daily tool, but it brings significant risks for the legal profession. One of the most prominent issues is 'hallucinations,' where an AI generates convincing but completely fabricated information. In a legal context, this means a tool might invent a case opinion or a rule of law to support a specific argument.

The consequences of these errors can be severe for legal professionals. As JD Supra reported, the case of Mata v. Avianca serves as a primary example of this danger. In that instance, attorneys submitted a response that included fake quotes and citations created by ChatGPT, which did not exist in any official reporter.

The court took swift action by imposing sanctions on the lawyers involved. The court noted that the attorneys abandoned their responsibilities by submitting non-existent judicial opinions. This serves as a warning that while the technology may produce a result, the human lawyer remains responsible for verifying every fact.

Core ethical duties for lawyers

Bar associations across the country are providing guidance based on three core ethical themes: competence, confidentiality, and supervision. Technical competence requires lawyers to understand the risks and benefits of the tools they use. This means they must know how to navigate the technology to provide effective representation while avoiding common pitfalls like unverified AI output.

Confidentiality is another major pillar of legal ethics that AI complicates. When a lawyer inputs client information into an AI tool, that data may be processed on external servers. Many AI companies use this data to train their models, which could potentially expose privileged information if not handled with extreme care.

Supervision is the final key duty, requiring lawyers to oversee the work of colleagues, employees, and outside vendors. If a junior associate or a non-attorney uses AI to complete a task, the managing lawyer is still responsible for the final product. They must ensure that the AI-generated work complies with all professional conduct rules and maintains client privacy.

Data privacy and security hurdles

Maintaining client confidentiality is a fundamental responsibility that predates the digital age. However, the complexity of modern AI systems creates new challenges for transparency and accountability. It is often difficult for a law firm to verify exactly how a vendor processes, stores, or uses submitted prompts for model improvement.

Legal professionals must ask specific questions before adopting any AI tool for practice. They need to know if data is encrypted both in transit and at rest, and who holds the encryption keys. They also need to understand the vendor's data retention policies and whether any employees can view the submitted prompts.

Because the technology can be opaque, some experts suggest a cautious approach. If a lawyer has any hesitation regarding how a tool handles data, they may choose to demur until more transparent information is available. The goal is to balance the efficiency of AI with the strict requirements of attorney-client privilege.

State-level guidance and consent

While the three main themes of competence, confidentiality, and supervision are standard, some states have added specific requirements. For example, the State Bar of California has issued guidance recommending that lawyers disclose to clients if they intend to use generative AI tools during representation.

Other jurisdictions are moving toward even stricter requirements regarding client permission. The Florida Bar Board of Governors suggested that lawyers should not only inform clients about AI use but also obtain informed consent. This ensures that the client is fully aware of how their information is being handled by third-party software.

A safe approach for legal professionals today is to be completely transparent with their clients. Lawyers should explain exactly how they intend to use AI and, more importantly, how they will ensure the results are accurate. This maintains the trust between the lawyer and the client while utilizing modern tools for efficiency.

The human element in AI

Despite the excitement surrounding automation, AI is not a replacement for human judgment. The legal profession has seen waves of technological change before, and the growth of the field has consistently outpaced the fears of AI-driven job loss. The focus remains on using AI as a tool to enhance human capabilities rather than a substitute for them.

The overarching requirement for lawyers is to maintain oversight of all work products. Even if a tool provides a positive result that matches what a lawyer was seeking, the lawyer must perform due diligence. They cannot simply accept an AI's output as fact without independent verification of the underlying legal authorities.

Ultimately, the responsibility for the work remains with the human professional. By treating AI as a tool that requires constant supervision, lawyers can leverage its benefits while protecting their clients. The goal is to ensure that technology serves the law, rather than allowing the technology to bypass the rules of the profession.