THE DIMENSION
Checking the relevance, quality, and limits of information used in a decision.
The dimension in context
Checking the relevance, quality, and limits of information used in a decision.
You may distinguish a useful observation from an unsupported assertion.
You may rely on familiar sources or have limited access to evidence.
An illustrative work situation
A participant hears a vivid customer story and wants to redesign a process around it. The story reveals a real experience, but the team does not yet know how common the problem is or which conditions produced it. The decision requires attention to both the example and the broader evidence.
Evidence review involves relevance, quality, and limits, not simply collecting more data. A large dataset can miss an important experience, while one compelling account may not describe the typical case. Reflection should ask what claim the evidence supports and what further information would distinguish competing explanations.
A small practice to examine
State the claim suggested by the example. Identify the population or situation to which you are tempted to extend it, then look for an appropriate comparison. Record missing information and possible selection effects. Decide what can reasonably be concluded now and what should remain an open question.
Endless information gathering can postpone a proportionate choice.
A question for closer study
Can participants distinguish evidence that a problem exists from evidence about its frequency, cause, or best solution?
Read the example alongside the research
The following editorial passages come from the linked framework chapters. They provide context for the original teaching example above; the example is fictional and is not a reported study finding.
A proposed MDP study could give reviewers the same decision record with different eventual outcomes and examine whether their process judgments change. That would test the review procedure as well as the participant's self-description. Researchers should distinguish a justified update from a retrospective rewriting of the original decision rule. They should also limit the record to information needed for the study, rather than collecting confidential details merely to make an example vivid. An informative result might be that participants report frequent review but mostly explain outcomes after the fact without revisiting their earlier assumptions. Such a finding would support revising the review items and recommendations, not claiming that the questionnaire has already improved decision quality.
Continue: A good outcome does not prove a good decision process →
A responsible use might involve privately selecting one decision habit to examine and sharing only a chosen process improvement. The participant can state what remains uncertain and what would trigger revision. The profile should not be used to determine who is smart enough to lead a project, manage money, or make safety-critical judgments. Its development status and narrow construct boundary rule out those inferences. Future research should examine whether users understand that boundary and whether feedback encourages proportionate inquiry. Trust will come from that evidence and from transparent revision, not from claims that a new questionnaire reveals the quality of a person’s mind.
Continue: Accountability should improve reasoning, not reward a performance of certainty →
SOURCES AND FURTHER READING
Kahneman & Klein - Conditions for intuitive expertise ↗Mellers et al. - The psychology of intelligence analysis: Drivers of prediction accuracy in world politics ↗Lerner & Tetlock - Accounting for the effects of accountability ↗Bruine de Bruin, Parker & Fischhoff - Individual differences in adult decision-making competence ↗