UK watchdog may probe Home Office use of AI in asylum cases

The scope of the investigation

The Independent Chief Inspector of Borders and Immigration (ICIBI) is considering a formal investigation into how the UK Home Office uses generative AI. This watchdog monitors the management of borders, immigration, and customs, and it is currently planning its inspection topics for next year. The potential probe follows significant public pressure from lawmakers and researchers regarding the secrecy surrounding these tools.

The watchdog has reached out to the nonprofit Open Rights Group (ORG) to gather more information on the specific software being deployed. ORG spent six months investigating the technology by filing 11 parliamentary questions, two ministerial letters, and 10 public records requests. The Home Office has largely withheld details about these tools during this period, leading to calls for greater accountability.

While the ICIBI has not confirmed its final inspection priorities, it acknowledged being aware of concerns regarding AI reliability. The inspectorate previously noted that the effectiveness and reliability of one of the tools were varied in a report last year. This move signals a shift toward scrutinizing how automated systems impact the legal rights of individuals seeking protection.

How the AI tools function

The Home Office utilizes two specific systems: Asylum Case Summarisation (ACS) and Asylum Policy Search (APS). These tools were built using Microsoft Foundry and OpenAI's 2023 GPT-4 large language model. They are designed to help caseworkers navigate through massive amounts of information, including interview transcripts and country-specific policy data.

Caseworkers use the ACS to quickly review transcripts where asylum seekers describe the persecution they faced in their home countries. The APS acts as a search assistant to find and summarize information about a country's policies to determine if an applicant faces a genuine risk of harm. According to government estimates, using these tools together can save roughly one hour of work per case.

The government maintains that these tools are only intended to assist human staff and cannot make final decisions in isolation. A spokesperson for the Home Office stated that the tools are used to help navigate records and policy information rather than replace human judgment. However, the speed of the process is a priority as the government works to clear a backlog of approximately 100,000 asylum cases.

Accuracy and bias concerns

Researchers are deeply concerned about the accuracy of these tools because they are used in high-stakes decisions about whether to deport individuals. ORG's investigation found that the percentage of summaries deemed unfit for use due to factual errors was significantly higher than the 9 percent error rate published by the government. Furthermore, the study found that there is currently no quality control for the summaries produced by the APS tool.

The risk of "hallucinations"—where an AI generates false information—has already had real-world consequences in the UK court system. In July, a judge accused the Home Office of citing an AI-hallucinated document to refuse a Moroccan woman and her child's asylum claim. The court found the government had used false information to claim her home country was safe for her to return to.

Beyond factual errors, advocates worry about "cultural bias" inherent in GPT models. They argue that these models might struggle with language barriers or fail to capture cultural nuances in summaries. Such failures could lead to discriminatory outcomes where the specific context of an individual's experience is lost or misrepresented by the machine.

Transparency and oversight

The UK's "AI Playbook" requires government departments to be transparent about the algorithmic tools they use in decision-making processes. This includes listing those tools on a public repository so that the public and researchers can understand how they work. However, neither the ACS nor the APS tools are currently listed on the government's public repository.

Asylum seekers are currently not informed when these AI tools are used during the processing of their cases. This lack of notice makes it difficult for legal practitioners and migrant organizations to challenge incorrect summaries. Open Rights Group argues that not being able to contest an incorrect AI summary of personal information may breach existing UK data rights laws.

The Public Law Project has identified these two tools as just two of 11 high-risk AI systems they found being used across the government without being publicized. Caroline Selman, a senior research fellow at the project, warned that other tools may be deployed without any public awareness. This secrecy makes it nearly impossible for researchers or the public to ask questions about how these technologies are affecting lives.