Human in the loop AI
Human in the loop describes the steps where a person remains in the path of the work. There are two: assist, where the work is prepared in advance so that a person decides faster and more consistently, and person, where the step is left to a human because it cannot yet be specified precisely enough to be anything else. Both are designed before anything is built, not added afterwards as a safety net.
What does human in the loop actually mean?
It means a named person, at a named step, with something specific to decide. A general instruction to review outputs is not human in the loop. It produces a rubber stamp, because a reviewer with no threshold to apply and no consequence attached to the decision will approve.
The useful version specifies what the person sees, what they are deciding, what evidence is in front of them, and what happens to each answer they can give.
Which steps need a person, and which do not?
Every step receives one of four assessments. Retire removes it. Automate runs it without anyone present. Assist prepares the work so that a person decides faster and more consistently. Person leaves the step with a human, because it cannot yet be specified precisely enough to be anything else.
Assist and person are the two human-in-the-loop outcomes, and they differ in kind. Assist changes the preparation and keeps the judgement. Person changes nothing about who decides, and records why.
What has to be specified before anything is built?
- what it must support;
- what it must be gated from supporting;
- the quality threshold it must meet;
- the evidence and research it relies on;
- how often that evidence must be refreshed;
- where human review is required; and
- what assurance gates and audit trails are needed.
Human review appears on that list alongside the rest. It is a design parameter with a cost and a measurable effect, not a residual placed at the end to absorb whatever the system gets wrong.
Where does human review usually belong?
- at the point where an output leaves the organisation;
- at the point where a determination is recorded that a client or a regulator may later examine;
- wherever the system's confidence sits below the stated threshold;
- wherever the evidence the system relied on is stale, missing or contested; and
- on a sample of everything else, sized so that a drift in quality is detected before someone outside reports it.
Placing review anywhere else tends to be habit rather than design. Placing it everywhere is the same as placing it nowhere, because the reviewer's attention is finite and volume decides where it goes.
Each of these is a gate with a threshold, an owner and a defined outcome on failure. A gate with no failure path is not a gate. It is a notification.
How do assurance gates and audit trails work?
An assurance gate is a condition that must hold before work passes a point. It names the check, the threshold and what happens on failure. An audit trail records what was produced, from which sources, under which version of the specification, and who reviewed it.
Together they make the human contribution inspectable. Without them, a claim that a person reviewed the output cannot be tested after the fact, which is exactly when it is asked about.
How do you know the review is doing anything?
By measuring it. If a reviewer's decisions never diverge from the system's proposal, either the system is performing at the threshold and the gate can be narrowed, or the review has stopped being a review. Both conclusions are actionable, and neither is available unless the divergence was recorded from the start.
Human review is also a cost to serve. It should be justified against the commercial value it protects, in the same way as any other component.
Common questions
-
What does human in the loop mean?
That a named person sits at a named step, with something specific to decide and the evidence in front of them. It is an architectural choice about where a person is placed, not a general instruction to check the output.
-
What is the difference between human in the loop and human on the loop?
In the loop, the work does not proceed until the person acts. On the loop, the work proceeds and the person monitors it, intervening when something is flagged. The first is a gate. The second is supervision. Which is correct depends on the consequence of a wrong output passing unreviewed, and it is decided step by step.
-
Why is it important to have a human in the loop?
Because some steps cannot yet be specified precisely enough to be determined, and pretending otherwise does not remove the judgement. It hides it. A person also gives the process an accountable point, which is what is asked for after the fact.
-
Does agentic AI have a human in the loop?
Only where one has been designed in. A system that plans and executes several steps on its own has more places where a gate could sit and fewer places where one is obvious, which makes specifying the gates in advance more important rather than less. The same list applies: what it must support, what it must be gated from supporting, the threshold, the evidence, the review points and the audit trail.
-
Is human in the loop the same as human oversight?
Related but not identical. Oversight is a governance posture. Human in the loop is an architectural choice about which steps a person sits in, and it is decided step by step.
-
Does human review slow the process down?
At the steps where it is applied, yes. That is why it is applied at named steps against a stated threshold rather than everywhere. Steps removed elsewhere usually more than pay for it.
-
Can a person step become an assist or automate step later?
Often. What cannot be specified at the outset can sometimes be specified once the evidence exists. The list of steps that could not be specified is part of what a Discovery returns, and it is the natural input to a later phase.
-
Who writes the quality threshold?
You do, with us. It has to be a threshold the business will stand behind, because it is the threshold the output will be judged against.