Treatment Guideline Appraisal

---
name: treatment-guideline-appraisal
description: Critically appraising clinical practice guidelines — AGREE II, evidence grading, and detecting bias.
category: scientific
---

## Overview

treatment-guideline-appraisal teaches structured critical appraisal of clinical practice
guidelines: assessing methodological quality (AGREE II), understanding evidence-grading systems
(GRADE), spotting conflicts of interest and panel bias, and deciding whether a guideline deserves
to change practice. Guidelines carry authority; this skill is about checking whether they've
earned it.

## When to use

- Deciding whether to adopt a new guideline in practice or policy.
- Appraising guidelines with AGREE II: scope, stakeholder involvement, rigor, clarity,
  applicability, editorial independence.
- Understanding GRADE: quality of evidence vs strength of recommendation.
- Detecting industry influence, panel stacking, and evidence cherry-picking.
- Comparing conflicting guidelines on the same topic.
- Developing trustworthy local protocols from guideline evidence.

## Core concepts

- **AGREE II domains.** (1) Scope and purpose: is the clinical question clear? (2) Stakeholder
  involvement: were patients and relevant disciplines included? (3) Rigor of development:
  systematic search, evidence selection criteria, explicit link between evidence and
  recommendations. (4) Clarity: are recommendations specific and unambiguous? (5) Applicability:
  implementation tools, resource implications, audit criteria. (6) Editorial independence:
  funding and conflicts disclosed and managed. Score each; rigor and independence weigh heaviest.
- **GRADE.** Separates quality of evidence (high/moderate/low/very low — based on study design,
  risk of bias, inconsistency, indirectness, imprecision, publication bias) from strength of
  recommendation (strong/weak — based on balance of benefits/harms, values, resources). A strong
  recommendation can rest on low-quality evidence (and vice versa) — read both, not just the
  recommendation.
- **Conflicts of interest.** Financial ties to manufacturers of recommended products are common
  on guideline panels. Check: disclosed COIs, proportion of panel with ties, whether conflicted
  members voted on related recommendations, and funding of the guideline itself. A guideline
  recommending a sponsor's drug on weak evidence deserves skepticism proportional to the ties.
- **Panel composition bias.** Specialty-dominated panels recommend their specialty's
  interventions (surgeons recommend surgery). Trustworthy guidelines include methodologists,
  generalists, and patient representatives — and use formal consensus methods (Delphi, nominal
  group) rather than eminence.
- **Evidence selection.** Was the search systematic and current? Are key trials missing or
  mischaracterized? Check whether the guideline cites the trials you'd expect and whether effect
  sizes are reported as absolute (not just relative) benefits.
- **Recommendation wording.** "We recommend" (strong) vs "we suggest" (weak/conditional) in GRADE
  language. Vague verbs ("consider," "may be used") often signal weak evidence dressed as
  guidance — treat them as such.
- **Conflicting guidelines.** When guidelines disagree, compare: recency of evidence review,
  panel independence, GRADE ratings, and whether disagreement is about evidence or about values
  (e.g. different harm tolerance). Disagreement about values is legitimate; disagreement from
  cherry-picked evidence is not.
- **Living vs static.** Evidence moves; guidelines decay. Check the update plan and the date of
  the last evidence search — a 2019 guideline applied in 2026 needs a delta review.

## Practical workflow

1. **Scope check.** Does this guideline address your clinical question and population? (AGREE II
   domain 1.)
2. **Independence screen.** Funding source, panel COIs, methodologist involvement. Fail here =
   read with maximum skepticism.
3. **Rigor appraisal.** Systematic review methods, evidence tables, explicit evidence-to-decision
   process. (AGREE II domain 3.)
4. **GRADE extraction.** For each key recommendation: evidence quality, recommendation strength,
   and the stated rationale. Note where strong recommendations rest on low-quality evidence.
5. **Effect-size reality check.** Convert relative effects to absolute benefits/harms and NNT/NNH
   in your population's baseline risk.
6. **Applicability.** Resources, workflow fit, patient values in your setting. (AGREE II domain 5.)
7. **Decision.** Adopt, adapt (document modifications), or reject — with reasons recorded. Set a
   review date.

## Common pitfalls

- Treating all guidelines as equally authoritative (eminence-based vs evidence-based).
- Ignoring COI disclosures — or assuming disclosure alone manages bias.
- Confusing strong wording with strong evidence.
- Relative-risk framing hiding trivial absolute benefits.
- Outdated guidelines applied as current standard.
- Specialty-panel recommendations adopted without considering alternatives.
- Implementing without adaptation to local resources and patient values.