Article
Seven is the number, not eighty-nine
What Enveda’s AI study on batch-record review means for quality professionals: is the AI the better reviewer, and who decides what matters?
Article
What Enveda’s AI study on batch-record review means for quality professionals: is the AI the better reviewer, and who decides what matters?
The evidence for AI in life-science quality work is starting to arrive with study designs attached. The clearest example I have seen so far was published on 27 October, and it is worth reading properly rather than taking the headline figure off it.
Enveda, a drug developer, built an AI tool on Anthropic’s Claude models to pre-screen executed GMP batch records for its reviewers, and published a technical report on how it performed. Anthropic, which makes Claude, wrote about it the same day. The evaluation covered four batch records from one Phase 2 campaign, 1,061 pages in all. Across its runs, the AI tool raised all eleven issues the human reviewers had found, and one run raised seven more that none of them had. Review time fell from almost 26 hours to under three, including the time people spent checking the tool’s comments.
Everyone is quoting the 89%. I think seven is the more interesting number, because it raises two questions every quality professional should ask.
The seven stand up. A Quality adjudicator who had not reviewed the records scored every comment, human or AI, against one test: would an auditor reading the same record raise it? The seven passed, a second person confirmed the labels, and Enveda’s Quality team resolved all seven with the manufacturer before the batch was released.
But the report does not show a better reviewer. About four in five comments needed no action, from the people and from the AI alike. The AI wrote far fewer comments, which matters to whoever has to read them, and a similar share of them needed action. The authors’ own conclusion is that the reviewers found different things, and that coverage grew most when an AI run was added to a human review. A further run on a newer Claude model raised issues that neither the people nor the earlier runs had raised.
So: not a better reviewer. A different one, and a fast one. The gain came from the combination. Which leaves a question many organisations cannot answer: what does our current review miss, and how would we know?
The second question is the one I care most about. The AI did not only raise the seven. It graded them, two Minor and five Informational, after its own reconciliation pass had lowered two of them from Major. Among the seven, Enveda’s announcement lists a process step that exceeded its agreed limit and a test marked FAIL with no recorded result. In each run the AI also misread one existing entry, and posted that misread at Major.
That is not a flaw in the design. The grades are guidance, and Enveda’s reviewers decide every comment. But it shows where the quality work actually sits. The point is not to raise more issues. It is to see every potential issue and then decide, actively, which ones matter: a named person, with a reason. AI can widen what gets seen. It should support the decisions quality professionals make, not make them on their behalf.
And the decision has to leave a trace. Review records show what was raised; they rarely show what was considered and set aside. An observation a reviewer saw and judged not worth raising looks, afterwards, exactly like one nobody saw. The difference between a miss and a decision is the written reason.
The difference between a miss and a decision is the written reason.
There is a cost. When four comments in five need no action, every extra comment is another decision. Volume is not coverage either: the people in this study wrote many more comments than the AI and still did not raise seven issues. Push volume up far enough and deciding turns into clicking. Surface broadly, and keep every decision real.
The most useful thing in the report is not a number. It is the design, and the candour about it. An adjudicator who had not reviewed the records, and a second person confirming every label. One stated standard. The AI’s own grades and downgrades reported, and its misreads disclosed. Time measured with the human checking included. The tool kept outside the regulated record. And the limits stated plainly: four records, one campaign, Enveda’s own people, a tool not validated as a GMP system, an analysis the authors call exploratory. That is what an evaluation looks like when it is written for a quality function rather than for a press release.
SAGA is a different tool for a different document — validation deliverables, not batch records — and none of these results transfer to it. What it shares is the boundary. SAGA prepares the deliverables and hands them to a named reviewer. Every review finding stays open until a named person fixes it or accepts it in writing, with a reason, against the exact version they reviewed. It does not write to your DMS or QMS, and it does not sign. A written reason makes a decision visible. It does not make it a good one.
Nobody in quality is paid to be impressed by speed. They are paid to show, afterwards and under questioning, that the work was done correctly. So take the numbers — they are published, and they are not ours — and ask what your function asks of everything else that comes through the door. What was the baseline, and who measured it? Who decided what matters, and where is the reason written down? What happened when the output reached Quality? Anyone selling you AI for GxP work should expect those three questions — us included.
Klavs Esbjerg is co-founder of Verheim.
