ChatGPT-generated cancer education matches clinician quality in Nigeria

A first look at AI in Africa

Researchers have conducted a first-of-its-kind study evaluating how artificial intelligence can be used to create cancer prevention materials in Nigeria. This study specifically looks at how large language models (LLMs), such as ChatGPT, perform in generating patient education materials (PEMs). PEMs are documents or resources designed to help patients understand their health conditions and prevention strategies.

The study was prompted by the fact that research on AI-generated cancer education in low- and middle-income countries (LMICs) is still very scarce. While digital tools have already helped personalize health information for patients in underserved settings, the specific role of AI in these regions is still emerging. The researchers aimed to close this knowledge gap by comparing machine-generated content against human-authored content.

The team hypothesized that ChatGPT-generated materials would match the quality of those written by actual clinicians. They focused on three specific metrics to determine success: usefulness, understandability, and clarity. By testing these variables, the researchers sought to see if AI could serve as a reliable tool for healthcare providers who may lack the time to create extensive educational resources for every patient.

Comparing AI and human output

The study compared materials produced by ChatGPT-using GPT-4o against those developed by human clinicians. The results showed that the AI-generated PEMs were comparable to the clinician-developed materials across all three primary metrics. There were no overall significant differences found in how useful, understandable, or clear the patients found the information provided by the AI versus the humans.

However, there were specific nuances where human clinicians still held an edge in clarity. The study found that clinician statements regarding a healthy diet and the importance of limiting alcohol were rated as clearer than the AI's equivalents. These specific differences were noted as statistically significant, suggesting that while AI is highly capable, human experts may still provide more direct instructions on certain lifestyle habits.

These findings are significant because they suggest that AI can produce high-quality content that meets patient needs. For healthcare systems in Nigeria and across Africa, this could mean a way to generate dynamic responses to patient queries quickly. It provides a foundation for using AI as a supportive tool rather than a total replacement for medical professionals in educational roles.

Patient trust and technology use

The research also looked at how people in Nigeria currently interact with technology and health information. Interestingly, only 14% of the participants reported actually using AI tools like ChatGPT for health information. Despite this low current usage, a majority of the participants—58%—expressed trust in the technology. This gap between current behavior and expressed trust suggests a growing readiness for AI integration in the healthcare space.

When looking for health information, the majority of participants still rely on traditional and social sources. The study found that 73% of people relied on doctors, while 52% turned to Google. Other sources included social media at 36% and a specific Nigerian physician influencer, Aproko Doctor, at 22%. AI tools currently sit at the bottom of these primary information-seeking channels at 14%.

The study also tracked how people access the internet and use health-related gadgets. Most participants, 95%, accessed the internet via smartphones, while 49% used laptops. In terms of wearables, 30% used fitness trackers and 36% used health-related mobile apps. However, only 4% of the participants were currently using apps specifically designed for cancer care.

The path toward AI adoption

Despite the current low usage of cancer-specific apps, there is a high level of interest in adopting them. The study found that 90% of participants expressed interest in using cancer apps for both prevention and care. This indicates a significant opportunity for developers to create tools that meet a high demand for accessible, digital health resources in the region.

The researchers argued that AI adoption across the African continent is on an upward trend. Because of this growth, they emphasized an urgent need to incorporate more African evidence-based and culturally relevant information into AI outputs. Without this specific data, there is a risk that AI models may provide information that is not tailored to the local context, potentially leading to misinformation.

Moving forward, the goal is to ensure that AI becomes a tool that respects local nuances while providing high-quality medical education. By focusing on culturally relevant content, developers can help ensure that AI-generated PEMs are not only clear and useful but also accurate for the specific populations they serve. This study serves as a starting point for understanding how to bridge the gap between advanced technology and local healthcare needs.