CN-SIGMA Research Guide
Compare explanations before choosing one
Analysis of Competing Hypotheses (ACH) is a structured way to compare plausible explanations against the same body of evidence. It helps make assumptions, disagreements, and gaps easier to examine; it does not turn judgment into certainty.
This guide is educational material for a general professional audience. It is not legal advice and is not a substitute for professional judgment, subject-matter expertise, or appropriate review.
What ACH does
Rather than building a case for one early explanation, ACH lays out several reasonable hypotheses and checks each significant piece of evidence against all of them. Its central question is often: which explanation does this evidence challenge?Evidence that fits every hypothesis may be less useful for distinguishing among them than evidence that fits one and conflicts with another.
The CIA-hosted U.S. Government tradecraft primer describes ACH as identifying alternative explanations and evaluating evidence to disconfirm rather than confirm hypotheses. The method below adapts that comparison process for a short, general-purpose working note.
A practical comparison process
- 01
Frame one answerable question
Set the issue and time period. Separate what happened from why it happened, and make clear what the evidence can and cannot address.
- 02
Develop plausible alternatives
List distinct explanations before settling on a favorite. Include ordinary explanations and state important assumptions. If a new explanation emerges, add it and revisit the evidence.
- 03
Gather significant evidence
Record relevant facts and arguments, including contrary information and important gaps. Note source quality and whether apparently separate reports depend on the same underlying source.
- 04
Compare every item with every hypothesis
Use a matrix. For each evidence item, record whether it is consistent (C), inconsistent (I), or neutral (N) for each explanation. Explain consequential ratings so another reader can review them.
- 05
Look for diagnostic evidence and disconfirmation
Ask which evidence actually distinguishes the explanations. Focus on meaningful inconsistencies, not just the amount of evidence that appears to support a favored account. Consider what should be observable if each hypothesis were true, and whether it is missing.
- 06
Check sensitivity and report limits
Revisit the hypothesis set and key assumptions. Ask how the judgment would change if a critical item were wrong, dependent on another source, or misleading. State what is best supported, what remains unresolved, and what new indicator would change the assessment.
A small hypothetical example
Suppose a company team cannot reach its scheduling portal from the office. The following explanations and observations are invented for illustration; they describe no real incident.
Hypotheses
- H1: The portal provider has an interruption.
- H2: An organization access-policy change is blocking the portal.
- H3: A problem in the office network route is blocking it.
In this simplified matrix, C means consistent, I means inconsistent, and N means neutral or not useful for separating the hypotheses. These are comparisons, not numeric scores.
| Observation | H1 Provider | H2 Policy | H3 Network |
|---|---|---|---|
| An outside monitor reaches the portal during the problem. | I | N | N |
| An access-policy update was logged shortly before the problem. | N | C | N |
| Staff using mobile data can reach it; office users cannot. | I | C | C |
The observations make a broad provider interruption less consistent and leave both the policy and network explanations plausible. The timing of a logged policy update points toward H2, but timing alone does not establish cause. A useful next check would compare policy-denial records with connection tests from affected and unaffected networks. If those records are unavailable, the cause remains unresolved.
Common pitfalls
- A narrow hypothesis set: if the likely alternatives were never considered, a tidy matrix can still point in the wrong direction.
- Counting instead of diagnosing: a raw tally treats weak, dependent, and highly informative evidence as if they were interchangeable. Ask what actually discriminates; do not treat a score as a probability.
- Mistaking repetition for corroboration: multiple reports may trace back to one source or record.
- Treating missing evidence as proof: absence matters only when the evidence would reasonably be expected and observable under that hypothesis.
- Letting ratings hide judgment: explain disputed assessments, source limitations, assumptions, and what could change the result.
Limits and uncertainty
ACH organizes comparison; it cannot repair incomplete collection, poor source quality, an omitted hypothesis, or a mistaken assumption. Ratings involve judgment, and an apparent inconsistency may reflect faulty information, a condition not considered, or a more complicated explanation. Recheck critical items and state how dependent the conclusion is on them.
Do not claim certainty because one hypothesis has fewer inconsistencies in a matrix. Report unresolved alternatives, the quality and gaps in the evidence, the assumptions that matter, and indicators that would prompt an update. ODNI's standards make similar requirements for Intelligence Community products: assess plausible alternatives, acknowledge material contrary information, and explain the basis and causes of uncertainty. Those standards govern their stated U.S. Intelligence Community context; they are cited here as public references, not as a CN-SIGMA affiliation or endorsement.
Public references
- U.S. Government, A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis (March 2009), ACH section, pp. 14–15. It describes generating alternatives, assessing each item against each hypothesis, revising the matrix, checking sensitivity, and focusing on disconfirmation. The primer also cautions that structured techniques alone do not guarantee analytic precision or accuracy. Read the primer.
- Office of the Director of National Intelligence, Intelligence Community Directive 203, Analytic Standards, standards on analysis of alternatives, uncertainty, assumptions, and clear argumentation. These requirements apply to designated U.S. Intelligence Community analytic products. Read ICD 203.
CN-SIGMA International is independent of the U.S. Government. These public sources are cited for their methodological content; their inclusion does not imply ODNI or CIA endorsement, affiliation, certification, or review of this guide.