Wikimedia asks judge to rule AI web retrieval is not fair use

A middle-ground legal stance

The Wikimedia Foundation has asked a federal judge in New York for permission to file a brief in the ongoing copyright litigation against OpenAI. This filing is unique because it does not side entirely with the AI companies or the news publishers. Instead, it proposes a nuanced distinction between how AI models learn and how they interact with the live web.

The non-profit argues that the pre-training of large language models is likely a fair use of copyrighted material. This aligns them with the positions held by OpenAI and Microsoft. However, Wikimedia takes a different stance on retrieval-augmented generation, which is the process where a chatbot searches the internet to find specific passages to build a reply.

The foundation wants the court to rule that while learning from data is permissible, grabbing live web pages to generate answers should follow the rules of the original site. As reported by PPC Land, this position seeks to protect the specific licensing terms of open-knowledge projects like Wikipedia.

The mechanics of retrieval

To understand the legal distinction, it is important to look at how chatbots function. In the retrieval stage, a system searches a database or the web, copies relevant text into a prompt, and then constructs a response based on those snippets. Wikimedia argues this specific step is where the copyright infringement occurs.

The foundation highlights that most Wikimedia content is released under a Creative Commons CC BY-SA 4.0 license. This is a 'copyleft' model, meaning that any remixes or derivatives must be shared under the same free license terms and provide proper credit. When a chatbot pulls this content without following those rules, it effectively bypasses the licensing agreement.

Wikimedia's brief suggests that if these actions are not considered fair use, it could have significant consequences for the open-knowledge movement. They argue that if humans are directed away from the original sources, it disincentivizes people from contributing to free-knowledge projects. This creates a situation where the 'flow of information' between authors and audiences is disrupted.

Scraping vs. Copyright use

One of the most novel parts of the Wikimedia filing is its take on the act of scraping itself. The foundation argues that scraping is merely a means of acquiring copies of works, rather than a copyright 'use' in its own right. They suggest that the legal focus should be on what a developer does with those copies after they are acquired.

This position differs from the publishers in the case, who are trying to hold OpenAI liable at the very point of acquisition. Wikimedia argues that if a later use of a copy is fair, the fact that it was acquired without authorization should not automatically make it unfair. They believe this distinction preserves the purpose of fair use laws while still allowing for other legal remedies.

The brief notes that while scraping is not a copyright use, it is not necessarily benign. It acknowledges that aggressive crawling can violate a website's terms of use, state contract laws, or the Digital Millennium Copyright Act. Wikimedia suggests that these actions should be handled through contract or computer-misuse statutes rather than copyright law.

Impact on the open internet

Wikimedia expresses concern that a ruling in favor of the publishers could lead to a more closed internet. They argue that if web publishers are not protected against AI, they will be pushed to put up more walls and paywalls against all automated actors. This could result in a less accessible web for human users who rely on free information.

The foundation also points out the physical infrastructure costs of AI activity. They note that high-volume non-human traffic from scraping and crawling can impact a site's ability to stay online for human readers. They have already had to implement technical safeguards and direct high-volume users to a specific enterprise service to manage this load.

The case is still in the early stages, and the judge has not yet decided whether to accept Wikimedia's brief. However, the filing introduces a third perspective to a high-stakes legal battle. It seeks to balance the technological advancement of AI with the preservation of the licensing systems that keep information free and accessible to the public.