Verifying Claims

---
name: verifying-claims
description: Verifies claims and assesses source credibility. Use when fact-checking claims, verifying accuracy, validating sources, detecting hallucinations, triangulating evidence, or performing SIFT or RADAR assessment.
---

# Verifying Claims

## Purpose

Systematically verify factual claims in both AI-generated and human-generated content using established journalism standards, academic source evaluation frameworks, and AI-specific hallucination detection methods.

This skill enables rigorous fact-checking through:
- Claim categorisation (factual/interpretive/opinion)
- Multi-source triangulation and cross-referencing
- Source quality assessment using established criteria
- AI hallucination detection (semantic entropy, extrinsic checks)
- Conflicting source resolution protocols
- Transparent confidence calibration

## When to Use This Skill

**Primary triggers:**
- User requests fact-checking, claim verification, or accuracy assessment
- Reviewing research outputs or AI-generated content for reliability
- Validating sources cited in documents or reports
- Detecting potential hallucinations in AI responses
- Resolving conflicting information across sources
- Assessing information quality before relying on it

**Integration contexts:**
- Used by researcher-agent to validate research findings
- Used by critic-agent to assess factual accuracy in artefacts
- Used by producer-agent to ensure reliable sourcing
- Used as verification gate before finalising deliverables

## Inputs Required

**Always required:**
- [ ] **Claim(s) to verify** - Specific statements requiring fact-checking
- [ ] **Verification mode** - Retrospective (sources provided) OR Proactive (find sources)

**Context-dependent:**
- [ ] **Provided sources** (for retrospective verification) - Documents, research, context materials
- [ ] **Domain context** - Field-specific considerations (medical, legal, technical, etc.)
- [ ] **Risk level** - Significance of accuracy (high-stakes decisions vs casual inquiry)
- [ ] **Time constraints** - Available time affects verification depth

## Outputs Produced

**Core verification report:**
1. **Claims extracted** - Specific factual assertions identified
2. **Categorisation** - Each claim classified (factual/interpretive/opinion/uncheckable)
3. **Sources evaluated** - Quality assessment using RADAR criteria for each source
4. **Verification findings** - Evidence for/against each claim with source attribution
5. **Confidence assessment** - Calibrated rating (confirmed/likely/possible/unverified/disputed)
6. **Audit trail** - Complete documentation of verification process

**Optional outputs:**
- Recommendations for addressing unverified or disputed claims
- Flags for claims requiring domain expert review
- Suggested corrections for identified inaccuracies

## Operating Principles

**Medium freedom with preferred patterns:**
- Follow the core workflow sequentially (claim categorisation → source mode → verification → confidence assignment)
- Apply SIFT method and triangulation protocols as standard practice
- Use RADAR criteria for source quality assessment (see `references/source-evaluation-radar.md`)
- Adapt verification depth based on claim significance and risk
- Exercise judgement on when to escalate to domain experts

**Non-negotiable standards:**
- Apply identical verification standards to all claims (non-partisanship)
- Document all sources consulted with transparent audit trail
- Use calibrated confidence language (never present uncertain claims as certain)
- Surface disagreements between authoritative sources rather than hiding them
- Prioritise primary sources over secondary sources over tertiary sources

## Core Workflow

### Phase 1: Claim Extraction and Categorisation

**Step 1: Extract specific factual claims**

Identify discrete, verifiable assertions within the content.

**Questions to guide extraction:**
- What specific statements are presented as facts?
- Which claims involve names, dates, numbers, statistics, or events?
- What causal relationships or predictions are asserted?
- Are there embedded factual claims within interpretive passages?

**Example:**
- Content: "The UK's ageing population, which reached 67.3 million in 2021, will strain healthcare resources."
- Extracted claims:
  - Claim 1: "UK population reached 67.3 million in 2021" (factual)
  - Claim 2: "UK population is ageing" (factual, requires definition)
  - Claim 3: "Ageing population will strain healthcare resources" (interpretive/predictive)

**Step 2: Categorise each claim**

Use the three-category taxonomy to determine verification approach.

**Category 1: Factual/Verifiable Claims**
- Definition: Statements provable or disprovable using objective evidence
- Characteristics: Specific, testable, has objective truth value
- Examples: Statistics, dates, historical events, scientific findings
- Action: **Proceed with full verification**

**Category 2: Interpretive/Analytical Claims**
- Definition: Combines facts with interpretation, analysis, or prediction
- Characteristics: Requires reasoning assessment, involves judgement
- Examples: Causal claims, predictions, impact assessments
- Action: **Verify factual basis, assess reasoning quality**

**Category 3: Opinion/Non-Verifiable Statements**
- Definition: Values, beliefs, preferences, or personal experiences
- Characteristics: Subjective, no objective standard for verification
- Examples: Aesthetic judgements, moral prescriptions, preferences
- Action: **Do not attempt to fact-check; flag as opinion**

**Step 3: Filter uncheckable claims**

Identify claims that cannot be verified even if factual in nature.

**Uncheckable categories:**
- **Predictions about the future** - Cannot verify until time passes
- **Personal experiences** - No external verification possible
- **Vague claims** - "Many people believe..." (who? how many?)
- **Hypotheticals** - "If X had happened, Y would have occurred"

**Action:** Flag as uncheckable and note why verification is impossible.

**Step 4: Prioritise verification efforts**

For multiple claims, assess priority based on:
- **Significance** - Claims affecting key conclusions or decisions (highest priority)
- **Specificity** - Concrete, checkable assertions (easier to verify efficiently)
- **Risk** - High-stakes contexts require thorough verification
- **Feasibility** - Available sources and expertise

Focus resources on high-priority factual claims.

---

### Phase 2: Source Mode Selection and Discovery

**Choose verification approach based on context.**

#### Mode A: Retrospective Verification (Sources Provided)

**When to use:** Checking claims against provided context, documents, or source material. Common for detecting extrinsic hallucinations in AI-generated content.

**Protocol:**
1. **Extract claims from AI output or document**
2. **Compare systematically against provided sources**
3. **Flag contradictions** (claim conflicts with sources - severity high)
4. **Flag unsupported claims** (claim unaddressed by sources when sources should cover it - severity medium)
5. **Verify supported claims** (claim explicitly backed by provided sources - severity low/none)

**Extrinsic hallucination checklist:**
- [ ] Are statistics and numbers present in provided sources?
- [ ] Are quoted statements actually from cited sources?
- [ ] Are causal claims supported by provided research?
- [ ] Are dates, names, and specific details consistent with sources?
- [ ] Does output acknowledge limitations mentioned in sources?
- [ ] Are confidence levels calibrated to source certainty?

**Key focus:** Detecting when AI ignores or contradicts provided ground truth.

For detailed AI hallucination detection methods, see `references/hallucination-detection.md`.

#### Mode B: Proactive Validation (Find Sources)

**When to use:** No sources provided, or provided sources insufficient. Requires web search and source discovery.

**Protocol - Apply SIFT Method:**

**S - Stop**
- Pause before accepting or sharing claim
- Check your emotional reaction (strong emotions = increased checking needed)
- Assess claim plausibility (extraordinary claims require extraordinary evidence)

**I - Investigate the Source**
- Practice lateral reading: leave the source and open new tabs
- Search what trusted sources say about the original source
- Check fact-checking sites (Snopes, FactCheck.org, PolitiFact)
- Look for Wikipedia entries on organisations or publications
- Search "[source name] + bias" or "[source name] + credibility"

**F - Find Better Coverage**
- Search for other trusted sources on the same topic
- Look for consensus across multiple reputable sources
- Prioritise primary sources (original research, official data)
- Check if major news organisations (Reuters, AP, BBC) covered the story
- Seek peer-reviewed research for scientific claims

**T - Trace Claims, Quotes, and Media**
- Follow quotes back to original context
- Verify images haven't been taken out of context
- Check if statistics are cited correctly
- Look for original research papers or official documents
- Confirm claim-makers are quoted accurately

---

### Phase 3: Source Quality Assessment

**Evaluate every source using RADAR criteria.**

Apply systematically to each source before weighting its evidence. For detailed RADAR framework with red flags and scoring guidance, see `references/source-evaluation-radar.md`.

**Quick RADAR summary:**

- **R - Rationale (Purpose and Bias)**: Why was this created? Are important facts omitted? Is language neutral or emotionally charged?
- **A - Authority (Credibility)**: What are the author's credentials? Is the author affiliated with reputable institutions?
- **D - Date (Currency)**: When was this published? Is this information still current for the field?
- **A - Accuracy (Verification)**: Does this cite reliable sources? Can you verify key claims elsewhere?
- **R - Relevance (Applicability)**: Does this directly address your research question?

**Priority:** Always prefer official/primary sources over third-party interpretations.

---

### Phase 4: Evidence Hierarchy and Weighting

**Apply systematic source weighting using the primary/secondary/tertiary framework.**

#### Tier 1: Primary Sources (Highest Weight)

**Definition:** Original documents of events, discoveries, or research.

**Examples:** Original research papers, official statistics (ONS, census), historical documents, legislation, patents, official organisational statements

**Weight:** 90-100% confidence when multiple primary sources agree

**Standard:** "Always prefer primary sources over secondary sources." When primary sources exist, cite them directly.

#### Tier 2: Secondary Sources (High Weight)

**Definition:** Analysis, reviews, or summaries of primary sources providing context and interpretation.

**Examples:** Literature reviews and meta-analyses, academic textbooks, reputable news reporting (Reuters, AP, BBC), systematic reviews, expert analysis citing primary evidence

**Weight:** 70-89% confidence when multiple quality secondary sources agree

**Use cases:** When primary sources are inaccessible or require expert interpretation.

#### Tier 3: Tertiary Sources (Low Weight)

**Definition:** Indexes or consolidations of primary and secondary sources without new analysis.

**Examples:** Encyclopaedias (including Wikipedia), dictionaries, handbooks, fact books and almanacs

**Weight:** Useful for orientation only; insufficient for citation

**Standard:** "Tertiary sources are usually not acceptable as cited sources in research because they are so far from firsthand information."

**Appropriate use:** Initial orientation, finding primary/secondary sources, quick fact checks requiring verification.

For context-dependent adjustments and detailed weighting guidance, see `references/evidence-hierarchy.md`.

---

### Phase 5: Multi-Source Triangulation

**Verify significant factual claims with at least three independent, high-quality sources.**

#### Triangulation Protocol

**Step 1: Ensure source independence**

Verify sources are truly independent:
- Not citing each other directly
- Different organisations/institutions
- Different methodologies or data sources
- Different perspectives or contexts

**Red flag:** Three sources all citing the same original claim without independent verification provides weak triangulation.

**Step 2: Apply data triangulation**

Check claims across:
- Different time periods (temporal consistency)
- Different geographic locations (spatial consistency)
- Different groups or populations (demographic consistency)

**Step 3: Seek consensus**

**Strong consensus (3+ high-quality sources agree):**
- Confidence: Confirmed
- Action: Accept as established fact with attribution

**Weak consensus (2 sources agree, 1 disagrees):**
- Confidence: Likely
- Action: Investigate outlier, weight by source quality, note disagreement

**No consensus (sources contradict):**
- Confidence: Disputed
- Action: Proceed to conflicting source resolution protocol (Phase 6)

**Step 4: Document triangulation**

Record for audit trail:
- Which sources consulted
- Points of agreement and disagreement
- Quality assessment for each source
- Rationale for final confidence rating

---

### Phase 6: Handling Conflicting Sources

**Five-step resolution protocol:**

**1. Identify disagreement:** Pinpoint exactly what sources disagree about (facts vs interpretation? same question? different contexts? Example: Different unemployment figures may reflect different dates or methodologies)

**2. Analyse methodologies:** Examine how each source arrived at their conclusion (methods used, assumptions, data access, limitations, could methodology explain discrepancy?)

**3. Assess source quality:** Apply RADAR to each. Weight by: primary vs secondary, expertise in domain, track record, recency (context-dependent), independence

**4. Seek additional sources:** Find third/fourth sources to break tie (search for primary sources, consult experts, check authoritative bodies, systematic reviews, fact-checking organisations). When multiple high-quality sources agree, outliers receive less weight.

**5. Transparent attribution:** Present disagreement with caveats. **Core principle:** "Surface the conflict rather than quietly averaging it away." Use patterns like "Source A argues X, while Source B maintains Y" or "Most experts agree X, though [Source Y] argues Z" or "Evidence insufficient; experts divided." Assign "Disputed" when authoritative sources disagree and resolution unclear.

---

### Phase 7: Confidence Calibration and Documentation

**Assign calibrated confidence levels and create transparent audit trail.**

#### Confidence Level Definitions

**Confirmed (90-100% confidence)**
- Multiple high-quality independent sources agree
- Primary sources available and consistent
- No credible contradictory evidence
- Language: "Confirmed," "Established fact," "Verified"

**Likely (70-89% confidence)**
- Strong support from quality sources
- Majority of sources agree
- Minor uncertainties or gaps remain
- Language: "Likely," "Probably," "Strong evidence suggests"

**Possible (50-69% confidence)**
- Some credible support
- Limited independent verification
- Significant uncertainties remain
- Language: "Possible," "May be," "Some evidence suggests"

**Unverified (30-49% confidence)**
- Insufficient evidence to confirm
- Single-source claims without corroboration
- Unable to find authoritative sources
- Language: "Unverified," "Cannot confirm," "Insufficient evidence"

**Disputed (varies)**
- Authoritative sources explicitly disagree
- Methodological conflicts unresolved
- Expert opinion divided
- Language: "Disputed," "Experts disagree," "Conflicting evidence"

**Note:** Confidence percentages are indicative ranges based on source quality and triangulation strength, not calculated scores. Use the qualitative criteria (multiple sources agree, etc.) as primary guidance.

#### Documentation Requirements

**Create complete audit trail:** Claims extracted, categorisation rationale, sources consulted (with RADAR), search strategies, verification findings, triangulation results, conflict resolution, confidence assignment rationale, limitations acknowledged.

**Transparency standard:** Enable readers to reproduce your verification process.

#### Correction Protocols

**When verification reveals errors:**

1. **Document clearly:** Specify incorrect claim vs corrected information, cite sources, note severity
2. **Communicate transparently:** Make corrections prominent, explain what was wrong, acknowledge openly
3. **Maintain audit trail:** Record date, who identified error, verification process, preserve original context
4. **Proportionality:** Minor errors (brief note), significant errors (detailed correction), critical errors (flag for review/retraction)

**Example:** "Correction [Date]: Original claim stated 'Marie Curie won three Nobel Prizes in Physics.' Verification confirms two Nobel Prizes: Physics (1903) and Chemistry (1911). Claim incorrect in number and field(s)."

#### Final Verification Report Format

```
## Claim: [Specific factual assertion]
**Categorisation:** Factual/Interpretive/Opinion/Uncheckable
**Mode:** Retrospective/Proactive
**Sources:** [List with tier + RADAR summary]
**Findings:** [Evidence for/against with attribution + triangulation]
**Confidence:** Confirmed/Likely/Possible/Unverified/Disputed
**Rationale:** [Why this confidence level]
**Limitations:** [Uncertainties/scope boundaries]
```

---

## Edge Cases and Constraints

**Mixed Factual-Opinion:** Separate factual components (verify) from opinions (flag as subjective, don't verify). Example: "terrible Prime Minister" (opinion) vs "resigned in 2022" (factual).

**Scientific Consensus:** Verify as statement of expert consensus (check IPCC, scientific bodies for level of agreement), not as absolute truth. Distinguish consensus from individual expert opinions.

**Temporal Changes:** When sources disagree on facts, check if facts changed over time. Note publication dates; present: "As of [date], X; previously it was Y"

**Definitional Disagreements:** Sources may use different definitions (e.g., "unemployment": ILO vs claimant count). Clarify definitions; present both: "By definition A, X; by definition B, Y"

**Expert Review Escalation:** Flag for domain experts when encountering: technical claims requiring specialised knowledge, medical/legal high-stakes claims, conflicting expert opinions within specialist field, novel research, complex methodological disputes.

For additional edge cases and common pitfalls, see `references/edge-cases-and-pitfalls.md`.

---

## Integration with Other Skills

### Conducting Research (`conducting-research`)

**Integration point:** Phase 2 (Source Identification and Evaluation)

**Enhancement:** This skill's RADAR framework and evidence hierarchy can strengthen source evaluation in research workflows.

**Workflow:** Use `conducting-research` for comprehensive topic investigation; use `verifying-claims` for targeted fact-checking of specific assertions.

### Assessing Quality (`assessing-quality`)

**Integration point:** Factual accuracy is one dimension of quality assessment.

**Workflow:** Quality assessment may invoke claim verification for factual claims in artefacts under review.

### Workflow (`skills/designing-workflow/SKILL.md`, File Naming and Versioning section)

**Integration point:** Verification serves as quality gate before finalising deliverables.

**Workflow:** Producer should fact-check claims before submitting artefacts; critic should verify factual accuracy during review.

---

## Evaluation Scenarios

### Scenario 1: Verifying Factual Claim with Primary Source

**Task:** Verify "The IFCN Code of Principles requires signatories to meet 31 criteria"

**Expected process:** Categorise as factual → Proactive mode → Apply SIFT (find IFCN.org official documentation) → RADAR assessment (primary source, high authority) → Count criteria in official code → Triangulate with secondary sources if available → Assign confidence (Confirmed if matches, or issue correction if differs)

**Success criteria:** Locates primary source, counts correctly, assigns appropriate confidence, provides attribution, issues correction if needed.

### Scenario 2: Detecting AI Hallucination (Extrinsic)

**Task:** AI summarises provided research paper. Check if claims match source.

**Expected process:** Extract claims from AI summary → Compare against original paper (retrospective mode) → Apply extrinsic hallucination checklist → Flag contradictions (high severity) and unsupported claims (medium severity) → Document with source references

**Success criteria:** Identifies contradictions vs unsupported claims, provides specific source attribution, recommends corrections.

### Scenario 3: Resolving Conflicting Authoritative Sources

**Task:** Three reputable sources provide different dates for the same event

**Expected process:** Identify disagreement point → Analyse methodologies (timezone/definition differences?) → RADAR assessment → Seek primary source → Determine if definitional or factual → Present transparently if unresolvable → Assign appropriate confidence

**Success criteria:** Investigates root cause, seeks primary source, presents disagreement transparently, does not arbitrarily choose, uses appropriate confidence language.

For additional worked examples, see `examples.md`.

---

## Quality Checklist

Before finalising verification report, confirm:

- [ ] All factual claims extracted and categorised (factual/interpretive/opinion)
- [ ] Uncheckable claims filtered and flagged appropriately
- [ ] Appropriate verification mode selected (retrospective/proactive)
- [ ] SIFT method applied for proactive verification
- [ ] RADAR criteria assessed for all significant sources
- [ ] Evidence hierarchy applied (primary > secondary > tertiary)
- [ ] Minimum three independent sources for significant claims (triangulation)
- [ ] AI-specific checks applied where relevant (semantic entropy, extrinsic hallucinations)
- [ ] Conflicting sources resolved or disagreement surfaced transparently
- [ ] Confidence levels calibrated and language appropriate
- [ ] Complete audit trail documented (sources, methods, reasoning)
- [ ] Limitations acknowledged explicitly
- [ ] Expert review flagged where needed
- [ ] Non-partisan approach maintained (identical standards for all claims)
- [ ] Corrections documented and communicated transparently if errors discovered

---

## Supporting Files

This skill uses progressive disclosure. Core workflow is in this file; detailed reference material is in supporting files:

- **`references/source-evaluation-radar.md`** - Detailed RADAR criteria with red flags, scoring guidance, and context-dependent standards
- **`references/hallucination-detection.md`** - AI-specific detection methods (semantic entropy, HaluGate, extrinsic hallucination checks, confidence vs correctness)
- **`references/evidence-hierarchy.md`** - Detailed evidence tier definitions and context-dependent weighting adjustments
- **`references/edge-cases-and-pitfalls.md`** - Additional edge cases, time-constrained verification protocols, and common pitfalls to avoid
- **`examples.md`** - Extended worked examples demonstrating complete verification workflows