AI models show different strengths in dental implantology exams
A study found that ChatGPT-4 excels at multiple-choice recall while Gemini-2.0 performs better on open-ended questions in dental education.
The short version
- ChatGPT-4 achieved the highest accuracy of 84.67 percent on multiple-choice questions.
- Gemini-2.0 outperformed other models on open-ended questions, scoring an average of 1.63 on a three-point scale.
- Gemini-2.0 showed a statistically significant weekly score increase during open-ended testing over a three-week period.
- Researchers found that no model was reliable enough to function independently in dental curricula without expert supervision.
Why it matters
This research highlights that different AI models have distinct capabilities depending on the type of academic task. For students and educators, it underscores that AI is a tool for assistance rather than a replacement for expert clinical guidance.
Testing AI in dental education
Researchers wanted to see if large language models could help undergraduate dental students learn the basics of implantology. This field involves placing artificial teeth into the jawbone, and the study looked at how well AI handles core textbook concepts. Until now, most research on AI in dentistry focused on complex clinical cases for advanced professionals.
The study aimed to fill a gap by testing how these systems handle foundational training. The team designed an experimental framework to compare different models on questions drawn from academic curricula and professional literature. They specifically wanted to see if these conversational systems could be reliable enough for early-stage students.
To conduct the test, the authors constructed an examination dataset comprising multiple-choice questions across four distinct knowledge sections alongside open-ended questions tailored to undergraduate dental implantology learning objectives. They tested three specific models: ChatGPT-4, DeepSeek-R1, and Gemini-2.0. The researchers ran these tests over three separate sessions conducted over three consecutive weeks.
How the models were graded
The researchers used different methods to grade the different types of questions. For the multiple-choice items, they calculated precise accuracy rates across all knowledge sections. This provided a quantitative look at how well the models could recall specific facts from the dental curriculum.
The open-ended questions required a more complex grading system. Because these questions involve exploration and explanation, the researchers used a blind evaluation by human assessors. These assessors scored the AI responses based on three specific criteria: accuracy, completeness, and relevance.
Each of those three criteria was graded using a three-point Likert scale. This scale allowed the team to gauge the quality of the responses more deeply than a simple right-or-wrong score. To ensure the results were statistically sound, the team used several mathematical tests, including the Kruskal-Wallis H test and Bonferroni correction.
ChatGPT-4 vs Gemini-2.0
The results showed that the choice of model significantly changed the outcome based on the question format. ChatGPT-4 was the leader in structured testing, reaching an 84.67 percent accuracy rate on the multiple-choice questions. It was followed closely by DeepSeek-R1 at 83.78 percent, while Gemini-2.0 trailed at 78.44 percent.
The ranking completely flipped when the models were asked open-ended questions. Gemini-2.0 took the lead with an average score of 1.63 plus or minus 0.29 on the Likert scale. It outperformed DeepSeek-R1, which scored 1.57 plus or minus 0.36, and ChatGPT-4, which scored 1.53 plus or minus 0.30.
The study also found that testing time influenced the results for open-ended questions. While the three-week interval did not change the accuracy of multiple-choice answers, Gemini-2.0 showed a consistent weekly score increase. This suggests that the model's performance on complex, descriptive tasks may fluctuate or improve over different sessions.
The need for human oversight
Despite the high scores in some categories, the researchers reached a cautious conclusion about the role of AI in schools. They found that none of the three models were reliable enough to work on their own in undergraduate dental education. The lack of consistent accuracy across all subjects remains a primary concern for educators.
The study emphasizes that different models have different strengths that must be understood by those using them. ChatGPT-4 is currently better for multiple-choice recall, while Gemini-2.0 excels at open-ended responses. However, neither can replace the nuanced understanding required for medical training.
As reported by Bioengineer.org, the authors stressed that expert academic supervision is indispensable. Because the models can still produce inconsistent results, they should be treated as aids rather than primary sources. Human experts must remain in the loop to ensure students receive accurate and safe information.
What to watch
- How AI performance changes as dental curricula become more complex.
- Whether specific models become more reliable as they are updated with more medical data.
Try it
If you use AI for studying, try comparing how different models (like ChatGPT vs Gemini) answer a fact-based question versus a 'how-to' explanation to see if you notice the difference in style.
Sources and links
First reported by Bioengineer.org. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.