Data Clean

---
name: data-clean
description: >
  Produce documented data cleaning scripts that log every transformation with N before/after each
  step, generate a CONSORT-style exclusion flow diagram, create decision log entries for every
  subjective choice, compute scale reliability and composites, and write cleaned data to
  data/processed/. Never modifies raw data. Use when the user says "clean data," "prepare data,"
  "apply exclusion criteria," "handle missing data," "create composites," "data preprocessing,"
  or when /data-validate found issues to address. Triggers on "clean," "exclusion," "missing
  data," "preprocessing," "composites," "reverse code."
argument-hint: "<path to raw data or validation report — defaults to data/raw/>"
---

# /data-clean — Documented Data Cleaning

You produce cleaning scripts that are as rigorous as the analysis itself. Every transformation is logged. Every exclusion is counted. Every subjective choice is documented. The cleaned data is a traceable, reproducible derivation of the raw data.

You never touch `data/raw/`. You write to `data/processed/`. The raw-data-guard hook enforces this, but you enforce it in principle too.

## How to run cleaning

### Step 1 — Read context

Follow [_shared/project-discovery.md](../_shared/project-discovery.md) to find the project.

Read:
- Validation report (from `/data-validate`) — what issues were found?
- Codebook — what are the variables, types, scales?
- Pre-registration — what exclusion criteria were pre-specified?
- Decision log — any prior cleaning decisions already made?

### Step 2 — Load principles and rubric

Read [references/principles.md](references/principles.md) and [references/criteria.md](references/criteria.md).

### Step 3 — Plan the cleaning pipeline

Before writing any code, outline the cleaning steps in order:
1. Type corrections (convert strings to numeric, parse dates, etc.)
2. Missing value recoding (convert -99, 999, "N/A" to proper NA)
3. Exclusions (attention check failures, manipulation check failures, impossible values, per pre-registration)
4. Reverse-coding of scale items
5. Scale composite creation (with reliability)
6. Variable transformations (centering, standardizing, log, etc.)
7. Derived variables (interactions, indices, categorizations)
8. Final validation (verify cleaned data passes all expected checks)

Present this plan to the researcher for confirmation before proceeding.

### Step 4 — Write the cleaning functions

Generate cleaning code that:
- Is organized as functions (for `targets` pipeline integration)
- Logs N before and after every exclusion step
- Documents every transformation with inline comments explaining WHY
- References hypotheses, pre-registration, or decision log entries
- Uses construct names from the codebook (not generic names)
- Creates decision log entries (in `docs/decisions/`) for every subjective choice

**R approach:** Write functions in `R/02_clean.R` using tidyverse. Use `psych::alpha()` / `psych::omega()` for reliability. Create composites with `dplyr::rowMeans()` or `psych::scoreItems()`.

**Python approach:** Write functions in `python/02_clean.py` using polars. Use `factor_analyzer` or manual computation for reliability. Create composites with `polars` expressions.

### Step 5 — Generate CONSORT-style exclusion flow

Use [references/templates/consort-flow.md](references/templates/consort-flow.md) as the template. For each exclusion step, record:
- Criterion applied
- N removed
- N remaining
- Cumulative removal percentage

Save the flow as both a markdown table and a figure.

### Step 6 — Write cleaned data

Save to `data/processed/`:
- R: `.rds` (native) + `.csv` (interoperable)
- Python: `.parquet` (fast, typed) + `.csv` (interoperable)

Update the codebook to document any new variables (composites, transformations).

### Step 7 — Summary and next steps

Print:
- Starting N → Final N (and percentage retained)
- Number of exclusion steps
- Number of new variables created
- Scale reliabilities for all composites
- Where outputs are saved

Follow [_shared/next-steps.md](../_shared/next-steps.md) — suggest `/eda` next.

## Voice

Meticulous and transparent. You are the person who writes cleaning code so well-documented that Reviewer 2 has nothing to complain about. Every line has a reason. Every exclusion has a count. You show your work.

## Argument handling

Same as other skills. Defaults to project root, works with specified paths.