Teacher’s check
Can the student explain their claims plainly, without the draft?
Then change one fact. What would they claim now?
Both students in the figure explain the claims clearly without the draft. The second line separates them. It comes from The Irreducible Officer, an essay written for military education, which looks for the person who "revises under challenge rather than retreating to the artifact's language".
The study behind this example
Researchers led by David Autor ran a three-month experiment with 133 patent lawyers at eleven U.S. law firms. Some got a custom AI tool for drafting patents and some didn't, and 91 lawyers finished. Expert lawyers, who didn't know who had the tool, graded the work. No students took part. The headline: tested without AI at the end, the experienced lawyers got better, and the newer lawyers split, some better and some worse.
With the AI tool, the lawyers' drafts scored higher after 10 days and again after 90 days, and the newer lawyers gained the most.
At the end, the 91 lawyers who finished each fixed a flawed patent application without AI. Those who had used the tool did better overall, but only because the experienced lawyers did better. The newer lawyers, with fewer than seven years on the job, didn't improve on average: fewer of them scored in the middle, and more scored low or high. The authors suggest AI was a springboard for some newer lawyers and a cushion for others.
Fifteen of the 91 final tests showed signs of AI use. Leave them out, and the overall gain is too small to be sure of, while the experienced lawyers' gain still holds by a looser test.
The authors say plainly that the experiment doesn't explain why AI helped some lawyers and not others. It covers one job, the group of newer lawyers was small, and the AI tool was custom-built. Google ran the study, paid for it, and works with all eleven firms.
The figure's two students show one way such a split could happen in a class. The study measured scores, so it didn't test the figure's explanation: writing down what's new before drafting, and reworking the claims from it when a fact changes. The authors suggest one possible reason for the split: newer lawyers using AI worked faster but didn't get better without it.
Source: Autor, Rodchenko, Martin, Iscenko, Strand, Pearl & Ferere, "Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting," NBER Working Paper 35720 (revised October 2026). Quotes are from the abstract unless a page is given. The students, the class, the invention, the claims, the older patent and the check are constructed.
From the original PME essay
Fluency substitution is easier to miss. AI produces the tone of analytic maturity (balanced paragraphs, caveats in the right places, the measured voice of a considered judgment) and the student mistakes well-ordered language for well-owned reasoning. Researchers studying AI's effects on student cognition have introduced the concept of epistemic confinement to describe this condition: an illusion of competence while operating entirely within AI-constructed analytical boundaries, where the student believes they are thinking independently while the frame doing the work was never theirs (Chow et al., 2026). In strategy, fluency substitutes for deciding. A paragraph that balances every consideration may never identify which risk actually matters most.
The Irreducible Officer, Section II, What the Problem Looks Like
Go further
- The study: Autor et al. (2026), NBER Working Paper 35720
- The original essay, written for military education: The Irreducible Officer, Section II
