Appropriate reliance
You have time to check one of two AI answers. Decide where the check goes, then see what the study found.
Read the question and results
This reading version presents the figure’s questions, results and reasoning. It does not record your choices.

An interactive figure with two tasks from a 2023 field experiment with BCG consultants. Task A: segment the footwear industry market based on users. Task B: use a spreadsheet and insider interviews to pick the brand a clothing firm should back. An AI answers both in the same form, each marked ready, and both answers will go into your team's first draft. You have time to check one task; a check means working it yourself, then comparing with the AI's answer. You choose which task gets the check and rely on the AI for the other. The rest of this description, and the rest of the beat list, appear after you answer or skip each question.
Two tasks from one study.
two task cards. Task A: "Segment the footwear industry market based on users." Task B: "Use a spreadsheet and insider interviews to pick the brand a clothing firm should back." At top left: "Dell'Acqua et al. · Field experiment, 2023 · 758 BCG consultants · some with GPT-4".
An AI answers both. The two answers look equally finished.
under each task, "AI's answer" and two grey lines of writing on a beige band. Each gets the same grey "Ready" tag at the same moment.
You have time to check one. Which one?
along the bottom, "Time for one check. A check: work it yourself, then compare." Each card gets the line "If it's your team's first draft" and a "Check this one" button. The figure waits for your choice. Your choice shows as a red dot on that card, and "You rely on it" in red on the other. If you skip, both cards say "No call made".
After the first question: results and the next question
After the first question: a jagged line, the frontier, is drawn between the tasks: Task A inside, Task B outside. On tasks inside the frontier, consultants with AI finished 12.2% more tasks, 25.1% faster, at more than 40% higher quality. On Task B, consultants without AI were right 84.5% of the time, with GPT-4 70.6%, and with GPT-4 plus a prompt-engineering overview 60% (the gap between the two AI groups is uncertain), and graders rated the AI users' work higher in quality, right or wrong. Task B's deciding details were in the interviews; the spreadsheet only looked complete. For Task A, the figure reasons that there are no hidden facts, so a weak answer shows when read. The suggested call, reasoned from the results, is to rely on Task A, since a check there leaves Task B unchecked, and to work Task B from the interviews yourself, then compare. Then your segmentation goes straight into the launch plan, and you choose again for Task A: rely on it after a careful read, move or put your check there, or ask for a week to check both.
A jagged frontier marks where an AI stops being reliable. Inside it, AI helped.
an uneven line is drawn down between the cards, labelled "Inside the frontier" over Task A and "Outside the frontier" over Task B. On Task A's card: with AI, across tasks inside the frontier, 12.2% more tasks, 25.1% faster, more than 40% higher quality.
On Task B, consultants with AI were right less often. Graders still rated their work higher.
the picture dims and a panel rises: right answers on Task B. Without AI, 84.5%. With GPT-4, 70.6%. With GPT-4 plus tips (a prompt-engineering overview), 60%. A bracket under the two GPT-4 bars: "Rated higher in quality, right or wrong." The panel settles onto Task B's card.
Task B's deciding details were in the interviews. Task A, the figure reasons, has none.
in Task B, "insider interviews" is underlined, with "The deciding details were in the interviews. The spreadsheet only looked complete." Then under Task A, labelled "Reasoning: reading is enough": "No hidden facts: a weak answer shows when read."
AI helped on tasks like A. The suggested call: rely on A, work B yourself.
a key at bottom right: "Suggested call · reasoned from the results". The options you didn't choose fade. Task A gets a navy "Rely on it" tag, with the note "A check here leaves Task B unchecked". Task B gets "Work it from the interviews, then compare". Your choice stays in red.
Now your segmentation goes straight into the launch plan. What about Task A?
a second line appears on Task A: "If it goes straight into the launch plan", with three buttons: "Rely on it, after a careful read", "Move my check here" and "Ask for a week to check both". The figure waits for your choice. If you move your check from Task B, Task B's mark turns to a ring that says "Check moved to Task A".
After the second question: results and reasoning
After the second question: the suggested call is to rely on Task A and read it first: it is still inside the frontier and a weak answer shows on a careful read. Details AI tends to miss would move the check. Task B needs the check. Moving the check leaves Task B to a read-through only; asking for a week to check both is defensible and slips the launch a week. At the end your calls are set against the suggested calls, and the figure ends on the reviewer’s check.
Inside the frontier: rely on it, read it first. Details AI tends to miss would move the check.
the options you didn't choose fade. Beside "Move my check here": "Task B: read-through only". Beside "Ask for a week to check both": "Defensible · a week late". Task A's second line gets a navy tag: "Rely on it · read it first".
Reviewer’s check: What evidence justified relying on the AI here, and what would have changed the call? Change one condition, like details AI tends to miss, and ask again.
both lines under the picture at the same size. Both tasks, both answers, the frontier, the evidence and every suggested call stay in view. At bottom left, your calls are set against the suggested calls in words: for the first draft and for the launch plan, each matches, differs, is defensible, or has no call. Under Task B's bars: "Tips: a short prompting overview" and "The gap between the AI groups is uncertain."
What a good answer says
A good answer says where the check went and why. This is the figure's reasoning from the study's results; the study itself didn't test checking, a time budget, or what the work was for. Task B's deciding details were in the interviews, and the spreadsheet only looked complete. Graders rated the AI users' recommendations higher whether or not they were right, so a read-through wouldn't show a miss. That's where the one check belongs, and it starts from the interviews. Task A is the kind of work the study found inside the frontier, and the figure reads it as having no hidden facts: a weak segmentation shows when read. That covers quality, with one known blind spot: the study also found AI users' ideas less varied, and a careful read of one segmentation won't show that. As a first draft it can go without a check. When the segmentation goes straight into the launch plan, it is still inside the frontier, so the call is still to rely on it, but now someone has to read it carefully first. What would move the check is a detail the AI tends to miss: if the segmentation turned on something buried in the source material, as Task B's answer did, Task A would need working through too. Moving the one check would leave Task B, the risky task, to a read-through; asking for a week to check both is defensible, and it slips the launch a week. Changing one condition and asking again shows whether the analyst has a reason for the call.
Evidence
Dell'Acqua et al., Navigating the Jagged Technological Frontier, HBS working paper 24-013 (2023). In the field experiment, 758 BCG consultants, about 7% of the firm's individual-contributor consultants, were split between two sets of tasks: 385 worked the tasks inside the frontier and 373 the task outside it. Within each set they were randomly assigned to work without AI, with GPT-4, or with GPT-4 plus a prompt-engineering overview. On the tasks inside the frontier, consultants with AI finished 12.2% more tasks, 25.1% faster and at more than 40% higher quality. There was a cost in variety: on the first inside task, ideas for a new shoe, AI users' ideas were better but showed "a marked reduction in the variability of these ideas compared to those not using AI" (p. 12). On the task chosen to fall outside it, consultants without AI were right about 84.5% of the time, with GPT-4 70.6%, and with GPT-4 plus the overview 60%: in the working paper's words, with the two AI groups pooled, "19 percentage points less likely to produce correct solutions" (abstract). Graders still rated the AI users' recommendations higher in quality "regardless of the correctness of their answer" (p. 15). Two limits: the gap between the two AI groups is significant only at the 10% level, and so is the higher rating among wrong answers alone. The study examined GPT-4 as it was in 2023.
About the tasks
The tasks inside the frontier were 18 product-development questions for a fictional footwear company. Consultants generated ideas for a new shoe, segmented the market by users, listed the steps to launch it, and wrote a press release and a persuasive memo. On the task built to fall outside the frontier, consultants told a fictional clothing company's CEO which of its brands to invest in, working from a spreadsheet of financial data and interviews with company insiders. The spreadsheet was designed to look complete, but the deciding details were in the interviews, and the authors designed the task so that GPT-4 would get it wrong. In the study each consultant worked one set of tasks; the choice of where to spend one check, and the launch-plan condition, are this figure's. The task wording on screen is paraphrased. The frontier moves as models change; this is where it sat in 2023, with GPT-4.

From the original PME essay
AI improved performance on some tasks and degraded it on others, and the boundary was not obvious in advance. Tasks that looked similar from the outside differed in whether AI helped or hurt, a finding Dell'Acqua et al. describe as the jagged technological frontier (Dell'Acqua et al., 2023). Future leaders will operate along that frontier in every AI-enabled workflow, facing systems capable enough to invite reliance and uneven enough to make reliance dangerous. The educational response is pattern recognition: learning to identify what kind of task is in front of you, where models tend to be strong, where they tend to fail, and what independent checks are required before the output governs the work.
The Irreducible Officer, Section V, Appropriate Reliance as a Teachable Competency
Go further
- The study: Dell'Acqua et al. (2023), HBS working paper 24-013
- See the failure: Uncalibrated reliance
- The original essay, written for military education: The Irreducible Officer, Section V
The skill and the reviewer’s check come from an essay on judgment with AI. This page shows them with the consulting tasks from the study it cites.