Judgment Lab

Appropriate reliance

You have time to check one of two AI answers. Decide where the check goes, then see what the study found.

Appropriate reliance, an interactive figure in ten beats

An interactive figure with two tasks from a 2023 field experiment with BCG consultants. Task A: segment the footwear industry market based on users. Task B: use a spreadsheet and insider interviews to pick the brand a clothing firm should back. An AI answers both in the same form, each marked ready, and both answers will go into your team's first draft. You have time to check one task; a check means working it yourself, then comparing with the AI's answer. You choose which task gets the check and rely on the AI for the other. The rest of this description, and the rest of the beat list, appear after you answer or skip each question.

  1. Appropriate reliance · The skill

    On screen: the name, "Appropriate reliance", appears large over the stage with "The skill" above it, then moves into the top-left corner, where it stays.

  2. Two tasks from one study.

    On screen: two task cards. Task A: "Segment the footwear industry market based on users." Task B: "Use a spreadsheet and insider interviews to pick the brand a clothing firm should back." At top left: "Dell'Acqua et al. · Field experiment, 2023 · 758 BCG consultants · some with GPT-4".

  3. An AI answers both. The two answers look equally finished.

    On screen: under each task, "AI's answer" and two grey lines of writing on a beige band. Each gets the same grey "Ready" tag at the same moment.

  4. You have time to check one. Which one?

    On screen: along the bottom, "Time for one check. A check: work it yourself, then compare." Each card gets the line "If it's your team's first draft" and a "Check this one" button. The figure waits for your choice. Your choice shows as a red dot on that card, and "You rely on it" in red on the other. If you skip, both cards say "No call made".

  5. Make your call to see this.
  6. Make your call to see this.
  7. Make your call to see this.
  8. Make your call to see this.
  9. Make your call to see this.
  10. Make your call to see this.
  11. Make your call to see this.

About 2 minutes of playback, plus time for your calls. Read the question and results.

Faculty check

What evidence justified relying on the system here, and what would have changed the call?

Change one condition and ask for the call again.

How this figure works

Appropriate reliance means accepting AI support where it's warranted, checking or refusing it where it isn't, and being able to say why. Checking takes time, so the figure gives you two tasks from a 2023 field experiment and time to check only one. You decide where the check goes before you see what happened, then make one more call after a condition changes. You can skip either question, and Start over plays the whole figure from the beginning.

PME means professional military education. Its example comes from The Irreducible Officer, the National War College essay.

After you make your call: make or skip both calls to open this.

From the essay

The distinction between trust and reliance matters here. Trust is a subjective disposition, a feeling of confidence that a system will perform well. Reliance is the observable act of accepting its output and acting on it. Appropriate reliance is harder and more discriminating: accepting support when it is warranted, refusing or verifying when it is not, and being able to give an account of the difference (Raees & Papangelis, 2026).

The Irreducible Officer, Section V, Appropriate Reliance as a Teachable Competency

The educational response is pattern recognition: learning to identify what kind of task is in front of you, where models tend to be strong, where they tend to fail, and what independent checks are required before the output governs the work.

The Irreducible Officer, Section V, Appropriate Reliance as a Teachable Competency

Go further

Explore the seven failure modes