Confirm an offline relevance win with an online A/B on CTR/conversion before declaring victory. Reach for this after an offline gain.
Scanned 9/23/2026
Skills Directorynpx -y skills add mcorbett51090/RavenClaude --skill validate-online --agent claude-codeInstalls into .claude/skills of the current project.
Are you the author of Validate Online?
Add the live security badge to your README — it updates automatically with every re-scan.
[](https://www.skillsdirectory.com/skills/mcorbett51090-validate-online)More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.
---
name: validate-online
description: "Confirm an offline relevance win with an online A/B on CTR/conversion before declaring victory. Reach for this after an offline gain."
---
# Skill: Validate online
Offline gains don't always transfer to user behavior (§3 #6).
## Step 1 — Confirm the offline win
A clear NDCG/MRR improvement on the judgment list first (§3 #1).
## Step 2 — Design the A/B
Control vs variant on a real metric (CTR/conversion), powered and bias-aware (§3 #6).
## Step 3 — Read the online result
Did the offline gain move the user metric? It may not (§3 #6).
## Step 4 — Ship only on confirmation
Promote the variant only if online confirms; otherwise keep digging (§3 #6).
## Output
An online A/B result confirming (or refuting) the offline relevance gain. Traverse Tree 1 in the decision-trees file.
Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.
No comments yet. Be the first to comment!
