By 
The AI response looks convincing. It identifies a likely diagnosis and explains its reasoning clearly.
But when you compare that diagnosis with the patient’s test results and the scientific reference ranges supplied by your lecturer, something does not add up.
What do you do next?
That is the decision facing first-year anatomy and physiology students in an assessment designed by Dr Emma Hargreaves. Rather than asking students to avoid AI, Emma gives them an opportunity to use it—and to find out why they cannot rely on it unquestioningly.
AI is reshaping biomedical science and healthcare
AI is becoming increasingly important in biomedical science and healthcare. Today’s students are likely to encounter it in their future careers, from analysing data to supporting research and clinical decision-making.
However, AI can produce medical information that is inaccurate, incomplete or inappropriate for a particular context. It can also present that information with enough confidence to appear trustworthy.
Emma wanted students to encounter this problem early in their studies, within a structured environment where they could learn from mistakes. Her aim was not simply to test whether they could reach the correct answer. She wanted them to practise combining AI support with scientific knowledge, evidence and professional judgement.
Test an AI diagnosis against evidence
Emma created a low-stakes summative assessment based on a fictional patient case. Using AI, she generated a personalised dataset for each of the 191 students taking the module. These datasets contained clinical-style information such as cardiovascular measurements and test results.
Students selected a likely diagnosis from a defined list and explained their reasoning. They were explicitly permitted to use generative AI to help them:
- interpret unfamiliar results;
- explore possible explanations;
- reason through the relationship between different measurements; and
- organise their response.
However, Emma deliberately designed the task so that entering the patient information into a general AI tool could lead to the wrong answer. Students also had to apply the specific reference ranges supplied in the module. These ranges formed part of the scientific context that the AI would not necessarily know unless the student provided and used them correctly.
This meant that copying an AI-generated diagnosis was not enough. Students had to compare its suggestion with the evidence, apply what they had learned in their cardiovascular lectures and decide whether the answer remained scientifically defensible.
Before starting the assessment, students attended two lectures on the cardiovascular system and a workshop on using generative AI for clinical-style reasoning. Emma demonstrated how prompting could affect the advice an AI system produced and reinforced an important professional boundary: the students were training as biomedical scientists, not clinicians.
Students also completed a reflection on their learning process. This was intended to help them consider not only what answer they reached, but how they used the tools and information available to them.
Emma used AI in the assessment design too. It helped her produce individual datasets and calculate the expected diagnosis for each one, giving her an accessible set of answers when marking.
Results: students learn from AI errors
The task made the consequences of uncritical AI use visible.
Fifty of the 191 students selected the wrong diagnosis, even though the module materials contained the information needed to challenge the AI response. Some students also lost marks after failing to verify calculations, and two incorporated data that AI had invented.
The average mark was 63.1%, broadly in line with Emma’s expectations for the assessment. More importantly, the submissions showed her exactly where students needed further support.
Emma gave students personalised feedback on their analysis and apparent use of AI. She also produced a detailed general feedback document explaining the common pitfalls, how students could check AI-generated material more effectively and where they could find further University guidance.
The assessment uncovered another challenge: some students appeared reluctant to discuss their AI use, even though it was permitted. In future versions, Emma plans to make this a clearer requirement within the reflective element and provide a workshop specifically focused on ChatGPT Edu.
At Kent, students are developing the confidence to work with emerging technology, and the knowledge to recognise when it gets something wrong.
Why we loved Emma’s approach to using AI at Kent
AI is increasingly shaping professional practice across higher education disciplines, and educators need to consider how it will affect their field. Students need opportunities to use it and to understand its limits.
Emma’s assessment brings that challenge to life. Students can ask AI to help interpret a fictional patient case, then compare its suggestions with scientific reference ranges, course knowledge and the evidence in front of them. They see how an AI-generated diagnosis can go wrong when it is not tested against the information available.
That is what makes Emma’s approach so effective. It moves beyond teaching students how to write a prompt or how to replace their work with AI completely. Students learn to use AI as a starting point for investigation and question its answers, check their reasoning and make a defensible decision of their own.
Curious about AI in assessments?
Our team supports Kent students and staff with training materials, advice, and regular information. Get in touch with us by emailing AITeam@Kent.ac.uk or sign up for updates.
This post was drafted based on a webinar transcript with with help from generative AI, then edited, reviewed and developed by a member of the AI@Kent team.