California enacts first state law regulating attorney AI use
The new law sets strict rules for how legal teams must verify AI outputs and protect client data starting in 2027.
The short version
- The law takes effect on January 1, 2027, and applies to all attorneys licensed in California, including in-house counsel.
- Attorneys must personally verify every citation in a court filing, regardless of whether it was generated by AI or a human.
- Lawyers are prohibited from putting confidential or personal identifying information into AI systems unless access is strictly limited.
- Arbitrators are banned from delegating any part of their decision-making process to artificial intelligence.
- Firms are generally held responsible for violations committed by their lawyers and staff.
Why it matters
This sets a legal precedent for how AI 'assistance' differs from 'delegation' in professional services. It forces legal teams to build rigorous human-in-the-loop verification processes to avoid court sanctions.
A new legal standard for AI
California has officially become the first state to regulate the use of artificial intelligence by attorneys through state statute. Senate Bill 574 passed both chambers and was signed by Governor Gavin Newsom in late 2026. The law establishes a framework for how generative AI can be used to assist in the practice of law without being used to replace human judgment.
While the State Bar had issued guidance in 2023 and 2025, this new law moves those expectations into formal state requirements. It specifically addresses the use of generative AI and agentic AI, which refers to systems capable of carrying out tasks with very little human direction. The core principle is that while AI can be a tool, attorneys cannot delegate the actual practice of law to it.
The legislation also introduces specific rules for arbitrators and tightens requirements for court filings. As KPVI reported, the law aims to ensure that while technology improves efficiency, the responsibility for the work remains firmly with the human professional. It codifies many practices that careful lawyers were already following, such as protecting client privacy and double-checking research.
Mandatory disclosures and verification
One of the most significant requirements of the new law is the mandatory disclosure of AI use in court documents. Attorneys must tell the court when they have used AI for documents submitted for filing. For content that is shared with the public, the law only requires that attorneys consider whether a disclosure is necessary.
The law also places a heavy emphasis on the accuracy of legal citations. Every citation in a court filing must be personally verified by the attorney responsible for the document. This rule applies even if the citation came from a traditional research database, a colleague's old brief, or an AI tool. This is intended to prevent the submission of "hallucinations," which are instances where AI generates false or fabricated information.
Failure to comply with these citation rules can lead to court sanctions against both the individual attorney and their firm. Because firms are generally on the hook for the actions of their staff, companies will need to create formal internal processes to manage these checks. The law provides a 21-day window for attorneys to fix or withdraw a challenged filing before such sanctions are applied.
Data privacy and human oversight
The law sets strict boundaries on how sensitive information can be handled by AI systems. Attorneys are prohibited from inputting confidential, personal identifying, or nonpublic information into an AI unless access is restricted to authorized individuals. The definition of personal identifying information includes specific details like medical records, financial data, and witness addresses.
Beyond the courtroom, the law impacts the day-to-day work of in-house legal teams. These teams often use AI to review contracts or answer internal business questions, and the duties to protect information and verify output still apply. Even if a non-lawyer handles some of the contracting work, the attorney who uses that output remains legally responsible for its accuracy.
Arbitrators are also subject to specific restrictions under the new statute. They are not allowed to hand off any part of their decision-making process to an AI system. If an arbitrator chooses to rely on AI-generated information that is not part of the official record, they must notify the parties involved and allow them to provide input where possible.
The path toward national rules
While California is the first to enact a state law, other states are already moving toward similar regulations. New York adopted a statewide rule on attorney AI use in 2026, while Florida updated its practice rules to focus on citation accuracy. Illinois has taken a different approach, allowing lawyers to use AI without disclosure as long as they review the output.
The fact that California is the first state to write these rules into law does not mean other states will follow the exact same path. In many states, the supreme court regulates the practice of law rather than the legislature. This means future rules in other jurisdictions may come from judicial decisions rather than new state bills.
For legal teams operating across state lines, this creates a complex patchwork of requirements. Some firms may choose to adopt California's rules as their baseline for all operations to ensure consistency. However, the lack of a clear line between "assisting" and "delegating" means that firms will likely have to develop their own internal policies to manage the risks of agentic tools.
Preparing for 2027
Legal teams have until January 1, 2027, to adjust their workflows to meet these new requirements. Organizations should begin by auditing every AI tool currently in use, including features built into standard research and contracting software. It is important to identify who has access to the data entered into these tools, including the third-party vendors.
Firms should also establish clear guidelines on who is responsible for verifying citations on large filings with multiple contributors. Because the law does not specify a single responsible party for group projects, firms must decide on an internal accountability structure. This includes settling on specific disclosure language to ensure that the firm is meeting the court's requirements consistently.
Finally, companies should look for where risk accumulates during the handoff of AI-generated work. A flawed clause in an AI-drafted template can end up in every contract built from that same template. Establishing a clear AI accountability policy now can help ensure that no one assumes someone else has already checked the output for errors.
What to watch
- Whether other states adopt California's disclosure requirements as a model.
- How courts define what constitutes a 'useful' AI disclosure.
- How the distinction between 'assistance' and 'delegation' is interpreted as AI agents become more autonomous.
Try it
Ask your outside counsel for a written summary of how they currently verify citations and handle AI disclosures in court filings.
Sources and links
- GovernmentSenate Bill 574 leginfo.legislature.ca.gov
- Governmentsimilar bills nysenate.gov
- GovernmentJudicial Council courts.ca.gov
- GovernmentAI policy illinoiscourts.gov
- Governmentproposed adding AI guidance calbar.ca.gov
- ReportingKPVI: the article we sourced
First reported by KPVI. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.