Sourceplease

---
name: sourceplease
description: Use when a claim, number, statistic, source or document needs a credibility weight before it is relied on or quoted — "how much can I trust this?", "is this true?", "check the sources", "what is this data worth?". Also use before embedding research figures into reports, decks, briefs or knowledge bases, and when another workflow hands over numbers it wants to cite.
---

# sourceplease — a credibility ruler

## Overview

Two ratings, never one. **Who says it** (source reliability) and **what is said** (credibility of the specific claim) are rated separately and never blended. A reliable source can be wrong about one fact; a dubious source can happen to be right.

Second principle: **"no data" is a valid result.** The ruler exists to tell weighed from unweighed — not to give everything a weight. Filling a gap with a plausible-looking number is forbidden.

## When to use

- A figure or claim is about to go into a report, deck, brief, article or knowledge base.
- Someone asks "is this true?", "how much can I trust this?", "what is this source worth?".
- A document or research section needs its claims triaged: keep, flag, remove, verify.
- Another workflow (research, sales prep, content writing) produced numbers and wants to cite them.

**Not for:** judging people or conversations (this is not a lie detector), professional advice in medicine, law or finance, or rating every casual sentence.

## The scale (Admiralty Code, adapted)

**Source — A to F**

| Class | Who this is |
|---|---|
| A | peer-reviewed research, meta-analyses, official government statistics |
| B | large named studies with a stated sample and method (e.g. Gartner, McKinsey, Pew) |
| C | industry reports with a described methodology; serious media with fact-checking |
| D | interested parties — a vendor reporting on the market it sells into, press releases |
| E | practitioner experience, case studies, podcasts, expert posts |
| F | anonymous, "everyone knows", source cannot be established |

A conflict of interest caps the class at D, however polished the presentation.

**Claim — 1 to 6**

| Score | Meaning |
|---|---|
| 1 | confirmed by independent sources |
| 2 | probable: consistent with other data, no contradictions found |
| 3 | possible: logical, but little confirmation |
| 4 | doubtful: contradicting data exists |
| 5 | improbable: contradicted by stronger data |
| 6 | cannot be judged: single source, nothing to check against |

**Rounding rule:** torn between two levels → always the *less credible* one (the later letter, the higher number).

**What is rated:** the claim exactly as received, attribution included. "Gartner says X" fails if Gartner did not say X, even if some X-like number exists — a rescued version is a new claim, rated separately. A famous name with no traceable original ("Einstein said…") is class F. A secondary summary of an A source (press office, association page, textbook) takes the class of what produced *the number*: a summary that reports a study's figure keeps the study's class; a summary that estimates its own figure is C.

## Checklist — run for every claim

1. **Who is the source**, and do they gain from the claim looking good? → class A–F.
2. **Are sample, method and year stated?** "646 buyers surveyed in 2025" ≠ "studies show".
3. **Independent confirmation.** Ten reprints of one press release = one source. Two studies by different organisations with different methods = two. Count works *within* a class, never across: many independent practitioners can confirm that something happens (E1), not how much; one unconfirmed study (A3) can give a number, not proof that it holds in practice. Fifty practitioners repeating one guru, or only the ones it worked for, are not independent.
4. **Criticism.** Has anyone tried to refute it? For anything contested, and for anything claiming A–B, a web search is required: `<claim> criticism`, `debunked`, `replication`, `methodology`. If search is unavailable, rate from what you know, mark the rating **provisional** and name what was not checked. Psychology and behavioural economics need extra care: several textbook effects failed replication — check the current status of an effect, do not take it from the textbook.
5. **Freshness.** Decay depends on the field: trend figures live 1–2 years, fundamentals of human decision-making live decades. Always state the year; if the claim carries none, write "undated" — that is itself a finding.
6. **Effect size.** "Statistically significant" ≠ "matters in practice". An effect can be real and tiny.

## Reading the rating

Read top to bottom; the first matching row gives the verdict. The source notes from the D and E rows also apply when a D or E source lands in an earlier row (a D6 is "a single source claims" *and* "vendor data").

| Rating | Verdict | What to do with the claim |
|---|---|---|
| any 4–5 | contradicts data | do not use; if the claim matters, write why it was rejected |
| any 6 | nothing to check against | do not present as knowledge; at most "a single source claims" — plus the source note for D/E |
| F1–F3 | origin unknown | find the origin and rate it; report both (`F → C3`); until found, treat as 6 |
| A1–B2 | can be relied on | use it; cite source and year |
| A3, B3, C1–C3 | working hypothesis | use with the note "probable, not proven" |
| E1 | confirmed in practice | working hypothesis that the effect exists; the size is unknown — "many independent practitioners report it, no measured figure" |
| D1–D3 | interested data | only with the note "vendor data"; several vendors agreeing is shared interest, not independence |
| E2–E3 | practitioner experience | "practitioner observation, no hard numbers" — an honest and useful category |

## Modes

**Single claim** ("how much can I trust that X?") → five labelled items, as in the example below: **Claim** (verbatim) · **Source** (who, what, year; "undated" if the claim as received carries no year) · **Rating** (e.g. `B2`), two lines of why, and "provisional — not checked: …" when verification was not possible · **To reach the next level** — what would raise the claim score by one; add what would raise the source class if that is the real bottleneck; for a 4–5, what would have to overturn the contradicting data, or "n/a" · **Verdict** from the table, with how to phrase the claim if it is used at all. Several unrelated claims → single-claim mode, repeated.

**Document or section** → a table: claim | source | rating | verdict | action (keep / flag / remove / needs verification). Close with a summary: what can be relied on, what hangs on someone's word.

**Called from another workflow** → run every number through the checklist before it is written into a report, base or deck. Next to each figure in the text: source, year, and for anything below B2 the verdict note. Section headings must not promise more than the weight of the facts inside them.

## Example

> **Claim:** "Remote workers are 13% more productive."
> **Source:** Bloom et al., 2015, *Quarterly Journal of Economics* — randomised trial at a Chinese travel agency's call centre.
> **Rating: A3.** Source is a peer-reviewed randomised experiment (A). The 13% held for call-centre staff in one company; later studies in other settings found smaller or negative effects, so as a general statement about "remote workers" it is possible but unconfirmed (3). Rounding rule keeps it at 3, not 2.
> **To reach A2:** two independent trials in different industries reproducing a positive effect of similar size.
> **Verdict:** working hypothesis. Quote as "in a 2015 randomised trial, call-centre staff working from home were 13% more productive", not as "remote workers are 13% more productive".

## Common mistakes

| Mistake | Fix |
|---|---|
| Counting reprints and re-tweets as confirmations | Trace to the original; one origin = one source |
| Upgrading the class because the report looks professional | Class follows interest and method, not design |
| Downgrading practitioners out of snobbery | A class-E practitioner with a verifiable case beats anonymous "statistics"; fifty independent ones outrank one unconfirmed study on *whether* it works |
| Counting heads instead of origins | Four hundred people repeating one claim are one source; volume is not confirmation |
| Turning a narrow finding into a general law | Quote the population, setting and year the finding actually covers |
| Quoting an upper bound as a point estimate ("as much as 40%" → "40%") | Keep the qualifier; a ceiling is not a measurement |
| Giving a rating to fill the table | `6` and "no data" are legitimate cells |
| Defending a previous rating when new data arrives | Your earlier ratings are data with a weight too — re-rate |

## Boundaries

- A rating is a reasoned judgement, not a sentence. New data → re-rate.
- Facts, numbers and sources only — not people, motives or private messages.
- Medicine, law, money: weighing the evidence ≠ professional advice.
- Web verification is required for contested claims, important decisions and anything claiming A–B — not for every sentence. No search available → rate, mark provisional, name what was not checked.