What AI Review Tools Can and Cannot Do── Finding the right distance between over-trust and underestimation

01Types of tools ── don't lump "AI review support" into one thing

First, let's unpack the words. "AI review-support tool" sounds like one thing, but the insides differ. Different mechanisms mean different strengths and different limits. Asking "is AI usable?" while mixing these together is why the conversation goes nowhere. Broadly, there are four kinds.

Real products combine these four. Banned words are screened out by the rule base, delicate expressions are caught by the classifier, comments are drafted by the LLM, and the basis is pulled by the search type ── and so on. So when you evaluate a tool, don't ask "is it AI?"; look at "which type is used for which part." How much you can trust it differs by type. A rule base that catches set words works almost with certainty. A classifier that guesses by probability, or an LLM that writes prose, must be treated on the assumption that it will get things wrong.

02What they can do ── tireless, thorough, fast

Before turning to limits, let's give fair credit to what they can do, because underestimation is also a mistake. AI support tools have strengths that human reviewers structurally cannot match.

First, coverage and consistency. People lose focus in the back half of a long document, and the same banned word may be judged differently depending on the day. Machines don't tire. They check hundreds of pages against the same yardstick to the very end, and their conclusion doesn't change between morning and evening. The "variation between reviewers" we saw in Vol. 5 can be erased in the machine-matching portion.

Second, speed and search. How was a similar expression judged in the past? Where might it touch the approved information? Work that takes a person tens of minutes of flipping through documents, a search-type tool surfaces as candidates in seconds. Faster to "retrieve" than a reviewer is to "recall."

Third, draft generation. Writing out the reasoning for a comment from scratch is hard labor. The LLM type can produce that first draft. In many situations, fixing the draft that comes out is faster than writing from zero.

03What they cannot do ── context, figures, and final responsibility

So what lies outside the tool? Getting this wrong leads to over-trust, so let's lay it out concretely. The wall of "it's decided by context," which we touched on in Vol. 4, is the very core.

Especially dangerous is the second: figures and layout. Exaggeration does not reside in words alone. Shift the origin of the vertical axis on an efficacy graph and a slight difference looks dramatic. Shrink a safety note to an unreadable size and the risk stops standing out. Even when the machine returns "pass" on the wording, the impression the whole material gives may be going too far ── this is a territory current tools can barely touch.

And the fourth. Final responsibility can only be borne by people. This is not a problem of immature technology but of how regulation and ethics are built. Both the Pharmaceutical and Medical Device Act and the Guidelines for Sales Information Provision Activities (= the Sales-Info Guidelines, discussed later) place the subject of responsibility on people and organizations. "The AI passed it" holds up in no review. This one point won't move no matter how smart the tools get.

04The risk of over-trust ── the trap of "the tool passed it, so it's safe"

Once you grasp what they can and cannot do, the failure mode to guard against most comes into view. It is automation bias (= the tendency to assume, without checking, that the answer a machine produces is correct). When a tool displays "no problem," people ease off looking with their own eyes. This is not weakness of will; it has long been known as a property of human attention.

What happens when this occurs on the review floor? The exaggeration the tool missed, the person waves through too. What should be a double fence opens at the same spot on both. And the tricky part is that the more you use the tool, the duller the reviewer's eye becomes. Keep leaning on the machine and the muscle of judging for yourself weakens ── a phenomenon long noted as the classic irony of automation. The reviewer who usually leaves it to the tool is the one who fails to notice when the machine gets it wrong.

The reverse ── underestimation ── also exists. One off-target comment from the tool and you write the whole thing off as "useless." Then you let go of even the coverage and consistency the machine is good at. Over-trust and underestimation are two sides of the same "failure of distance." The right distance lies in dividing the labor part by part: leave the parts it's good at, doubt the parts it's weak at.

05The adoption decision ── decide first what, and how much, to delegate

So how do you actually decide whether to bring a tool in? Deciding by the flashiness of the pitch or the lowness of the price is the way to avoid most. Set the axes of judgment in this order.

• Decide the purpose first ── Do you want thorough screening, a draft of the comment, or a search of past cases? Narrow the desired function to one, and the type you need (Section 1) is determined
• Limit the use to screening ── Don't delegate the pass decision. Position the tool as a device for narrowing down "candidates worth a person's look." Leave the pass/fail decision with people
• Try it small ── Don't roll it out company-wide at once; test it on materials whose judgment is already known. Check the actual missed violations and false alarms with your own eyes before widening
• Look at maintenance and cost ── Every time the approved information changes, the dictionary or model needs updating. The cost and effort of continued use bite harder than the cost of adoption

There is one more axis you can't skip on the pharma floor: handling of confidential data. Materials sent to review contain unpublished product information and internal judgments. If the mechanism sends them to an outside cloud, check where the information is stored, whether it won't be used for training, and how the contract protects it. Leak confidentiality in exchange for convenience and you're nowhere near helping the review.

The right side of this table ── the column people keep holding ── must not be surrendered when a tool is adopted. As long as you hold this, whichever product you choose you won't go far wrong. Conversely, a product premised on handing even the right side to the machine should be passed over, however high its spec.

06The discipline of validation ── measure missed violations before you use it

Even once adoption is decided, don't put it straight into production. A tool is validated before use. And where you place the weight of validation is crucial. To put the conclusion first: measure missed violations as the top priority.

First, prepare a bundle of materials whose correct answers are known ── ones people have already settled the judgment on. Mix in both appropriate and problematic ones. Run these through the tool and count two kinds of error separately. Missed violations (= returning "pass" on a problematic material) and over-detection (= stopping an appropriate material as "fail"). These two are utterly different in character.

Validation isn't finished in one pass. When approved information is revised and new forms of violation appear, the tool's real capability changes too. So re-measure periodically. In the same spirit as the version control of rules described in Vol. 4, keep a record of "when, on which correct-answer set, and with what result." Do this and you can later explain that "review in this period was supported by a tool of this capability." When there's an audit or a query, this speaks.

One more point. A tool's score has meaning compared against human review. Set the rate at which people miss and the rate at which the tool misses side by side, and see how far the overall miss rate drops when the two are combined. Not the tool's standalone score, but how firm the fence becomes with the "human + tool" combination ── that is the number you really want to know.

07Connections to other chapters ── place the tool inside the design of review

"How to keep distance from the tool," seen in this volume, connects to the series' other volumes as follows. A tool doesn't work on its own; it works only when placed inside the design of the review as a whole.

• AI Material Review Vol. 4 ── Rule Design ── What this volume's rule-based matcher uses is the banned words and approved-information dictionary built in Vol. 4. The insides of the tool are the very rules you designed
• AI Material Review Vol. 5 ── Standardizing Review and AI ── The tool's coverage and consistency are a means to erase variation between reviewers. Read this volume together as the equipment of standardization
• AI Programming Vol. 1 ── Foundations of Code Generation ── The validation idea that "it ran" is not "it's correct" has exactly the same skeleton as the discipline of measuring a tool's score against a correct-answer set
• AI Marketing Vol. 1 ── Marketing Redefined ── The equipment for absorbing the mass-generated promotional materials. The side that makes and the side that reviews use the same tool, AI, from opposite faces

An AI review-support tool is neither magic nor useless. In the work of handling volume ── coverage, consistency, search ── it delivers force people can't match, and in the work of deciding quality ── drawing the line by context, the implications of figures, the final pass/fail and responsibility ── it stays outside people. Get this boundary wrong and two failures await. The over-trust that delegates too much and misses violations, and the underestimation that writes it off and discards the benefits. Both are the shape of getting the distance from the tool wrong.

The right distance lies in dividing the labor part by part. Leave the screening it's good at; doubt the judgment it's weak at. A "pass" is not an indulgence but the starting point for a person to check. Before use, measure missed violations as the top priority, and re-measure every time the approved information changes. And never entrust the final responsibility to the tool. As long as you don't break this stance, AI makes the reviewer faster and surer. Next time we take up how to preserve "when, by whom, and how it was judged" ── the review record and audit trail, and on through corrective action.