Build Judgment List

---
name: build-judgment-list
description: "Build graded relevance judgments (explicit or click-derived) and an offline harness before tuning. Reach for this when there's no eval."
---

# Skill: Build judgment list

Tuning without a judgment list is guessing dressed as engineering (§3 #3).

## Step 1 — Sample the query mix
Representative queries weighted by real traffic (§3 #7).

## Step 2 — Grade relevance
Explicit graded labels or click-derived judgments, with position-bias caution (§3 #3 #6).

## Step 3 — Build the offline harness
Reusable NDCG/MRR/precision@k harness over the judgment list (§3 #3).

## Step 4 — Set the baseline
The current ranking's metrics — the bar every change must beat (§3 #1).

## Output
A graded judgment list and an offline harness with a recorded baseline.