Over the last few months, one of the phrases I have heard more and more in conversations about AI is:
“We need to make sure there is a human in the loop when we incorporate AI into a process.”
I can see why this has become the default response.
Agentic AI is now being incorporated into systems and processes across workplaces and institutions. These systems can sort information, recommend actions, draft responses and move work from one stage to another.
Keeping a person involved can be an important safeguard. The difficulty is that the phrase human in the loop tells us very little about what that person can actually see, decide or change.
A person may be named as the reviewer while having only a narrow view of what the system has done. They might be asked to approve a recommendation without seeing how it was reached, or have permission to disagree without the time, information or opportunity to do so properly.
In a busy workflow, “review” can easily become a quick click to accept what the system has suggested.
That leaves a more practical question: can the person in the loop recognise when the system has got something wrong, and is there a clear way for them to put it right?
When review becomes approval
Imagine a university introducing an AI system to help manage student support enquiries.
It reads a student’s message, assigns it to a category, recommends a priority, drafts a response and suggests which service should deal with it.
A member of staff then sees a summary and three options: approve, edit or refer. The workflow diagram would show a human decision point.
But what if they cannot see why the system chose that category? What if important information has been left out? What if they have dozens of cases waiting and approval is the quickest route through the queue?
They may have the option to disagree, but does the process make it realistic for them to do so?
Staff may struggle to review every recommendation if the workflow withholds context, encourages speed or makes disagreement difficult.
By the time the case reaches them, the system may already have interpreted the situation, selected what it considers relevant and presented one action as the default. The decision has been framed before the reviewer begins.
What meaningful human judgement requires
Meaningful human involvement requires more than placing an approval button at the end of a process.
The reviewer needs to understand what role the AI has played, see the relevant evidence and know where the system may be unreliable. They also need to be able to reach a different conclusion and act on it.
Can they reject the recommendation, pause the process, ask for more information or refer the case to a colleague? Can they reverse an action that has already taken place?
If they spot a problem, the workflow needs to give them a straightforward way to respond. That may mean correcting the recommendation themselves, returning the case for further review, escalating it or stopping the process altogether.
Even when AI only makes a recommendation, it can shape which evidence is noticed, which option appears normal and how much effort is needed to choose a different path.
The person reviewing the outcome needs a genuine opportunity to apply their own judgement.
Putting review where it matters
Requiring staff to check every action taken by an AI system may sound like the safest approach. In practice, blanket review can create its own problems.
If staff are expected to approve hundreds of low-risk recommendations, review soon becomes another routine task. The volume makes careful scrutiny less likely, while the presence of a human gives everyone a sense that the risk has been managed.
We could add considerable work and gain very little protection.
The questions become quite practical. What happens if the system gets something wrong? How easy would it be to spot? How easy would it be to correct? And who is affected if it isn’t?
A poor chatbot response is often easy to spot. A flawed decision inside an internal workflow may be much harder to identify.
Questions I keep coming back to
The more I thought about this, the more I found myself returning to the same three questions.
What role is the AI actually playing?
Finding information, producing a draft, making a recommendation and taking an action carry different levels of responsibility.
A system described as providing “support” might determine which information a member of staff sees or which cases receive attention first. Even without making the final decision, it may significantly influence the outcome.
We need to describe its role accurately and be clear about where its involvement begins and ends.
Can people meaningfully challenge its recommendations?
A short summary may make a process faster, but it can also remove important context.
The reviewer may need access to the original information, the sources used by the AI and some indication of uncertainty or missing evidence.
The practical design of a workflow matters too. If approval takes one click while rejection requires several steps or a written justification, the system is encouraging a particular outcome.
People need enough information, confidence and opportunity to exercise their own judgement.
What happens when it gets something wrong?
From a student’s perspective, it may offer little reassurance to know that a human was involved somewhere internally.
What matters is whether someone can identify the problem, understand what happened and take action to put it right.
That includes clear ownership of the system, straightforward routes for escalation and a visible way for students to challenge an outcome when necessary.
Keeping students at the centre
These systems could provide real benefits to universities.
They may help students find information more quickly, reduce unnecessary handoffs between teams and remove repetitive administration that takes colleagues away from more valuable work. Used well, they could create more space for conversation, care and professional judgement.
Much depends on the purpose behind the workflow.
When the main aim is to process more students at greater speed, human contact can gradually become something reserved for exceptional cases. Students may be directed towards automated support even when their circumstances are complex, or when they simply need to speak to someone.
Efficiency matters, particularly when services are under pressure. But students rarely judge a service by how efficient the workflow was behind the scenes. They judge it by whether they got the help they needed, whether they felt listened to and whether the outcome felt fair.
A fast response may be incomplete, and a smooth workflow can still send a student in the wrong direction.
The real test is whether the technology improves the experience of the people the service exists to support.
So, is a human in the loop enough?
In some settings, yes.
A clearly defined human role, supported by the right information and sufficient time, can provide valuable judgement and accountability.
The phrase on its own gives us no assurance that those conditions exist.
Before introducing an AI-enabled workflow, institutions should be able to explain what the system does, where human judgement enters the process, what the reviewer can change and how the person affected can challenge the outcome.
For me, the practical test remains:
Can the person in the loop recognise when the system has got something wrong, and is there a clear way for them to put it right?
These systems may help universities reduce friction and provide more responsive services. But the quality of those services will still depend on the decisions we make about where human judgement belongs, and what role people are expected to play when something goes wrong.
A human in the loop can be an important safeguard. We need to make sure that the person has the information, time and means to act when something does not look right.