Fine Tuning Dataset Curator

---
license: Apache-2.0
name: fine-tuning-dataset-curator
description: "Prepare high-quality datasets for LLM fine-tuning with filtering, deduplication, augmentation, and RLHF data formatting. Activate on: fine-tuning data, training data curation, RLHF dataset, data quality filtering, SFT dataset. NOT for: model training infrastructure (ai-engineer), prompt engineering without fine-tuning (prompt-engineer)."
allowed-tools: Read,Write,Edit,Bash(python:*,pip:*,npm:*,npx:*)
category: AI & Machine Learning
tags:
  - fine-tuning
  - dataset-curation
  - rlhf
  - data-quality
  - sft
pairs-with:
  - skill: ai-engineer
    reason: Fine-tuned models deploy through AI engineer pipelines
  - skill: data-pipeline-engineer
    reason: Large-scale data extraction and transformation before curation
  - skill: llm-evaluation-harness
    reason: Evaluate fine-tuned model quality on curated test sets
---

# Fine-Tuning Dataset Curator

Prepare, filter, deduplicate, and format high-quality datasets for supervised fine-tuning (SFT), RLHF, and DPO training of language models.

## Activation Triggers

**Activate on**: "fine-tuning dataset", "training data preparation", "RLHF data", "DPO pairs", "SFT data", "data quality filtering", "dedup training data", "curate instruction dataset", "preference data"

**NOT for**: Model training loop implementation (ai-engineer), prompt optimization without fine-tuning (prompt-engineer), or general ETL pipelines (data-pipeline-engineer)

## Quick Start

1. **Define the task** — What behavior should the fine-tuned model exhibit? Write 10 gold-standard examples by hand first.
2. **Collect raw data** — Scrape, export from logs, use existing datasets, or generate synthetic examples with a stronger model.
3. **Filter and clean** — Remove duplicates, low-quality entries, PII, and off-topic examples. Target quality over quantity.
4. **Format for training** — Convert to the target format: chat-ml for SFT, chosen/rejected pairs for DPO, reward signals for RLHF.
5. **Validate** — Hold out 10-15% for evaluation, verify distribution balance, run a small training test before full fine-tune.

## Core Capabilities

| Domain | Technologies | Notes |
|--------|-------------|-------|
| **Quality Filtering** | fasttext classifiers, perplexity scoring, regex rules | Remove noise before it poisons the model |
| **Deduplication** | MinHash (datasketch), exact hash, SimHash | Near-dedup critical for training stability |
| **Augmentation** | LLM-generated paraphrases, backtranslation, persona variation | 3-5x dataset size with diversity |
| **Format Conversion** | chat-ml, Alpaca, ShareGPT, OpenAI JSONL | Match target training framework |
| **PII Removal** | presidio, regex, spaCy NER | Legal requirement for most training data |
| **RLHF/DPO Prep** | Preference pair generation, reward model labeling | Chosen/rejected pairs with margin scoring |

## Architecture Patterns

### Pattern 1: SFT Data Curation Pipeline

```
Raw Sources ──→ [Extract] ──→ [Filter] ──→ [Dedup] ──→ [Augment] ──→ [Format] ──→ [Validate]
    │               │            │            │            │              │            │
  logs, docs    parse to     quality       MinHash      paraphrase    chat-ml     hold-out
  APIs, CSVs    instruction/ scoring       near-dedup   via LLM       or JSONL    eval set
                response     remove <                   persona                   distribution
                pairs        threshold                  variation                 check
```

```python
# Quality filtering pipeline
import hashlib
from datasketch import MinHash, MinHashLSH

def curate_sft_dataset(raw_examples: list[dict]) -> list[dict]:
    # Step 1: Basic quality filters
    filtered = []
    for ex in raw_examples:
        instruction, response = ex["instruction"], ex["response"]
        if len(response.split()) < 10:          # Too short
            continue
        if len(response.split()) > 2000:         # Too long (likely garbage)
            continue
        if instruction.strip() == "":            # Empty instruction
            continue
        if response.count("\n") > 50:            # Excessive formatting
            continue
        filtered.append(ex)

    # Step 2: Near-deduplication with MinHash
    lsh = MinHashLSH(threshold=0.8, num_perm=128)
    deduped = []
    for i, ex in enumerate(filtered):
        mh = MinHash(num_perm=128)
        for word in ex["response"].split():
            mh.update(word.encode("utf-8"))
        if not lsh.query(mh):  # No near-duplicate found
            lsh.insert(f"doc_{i}", mh)
            deduped.append(ex)

    # Step 3: Format for training
    formatted = []
    for ex in deduped:
        formatted.append({
            "messages": [
                {"role": "user", "content": ex["instruction"]},
                {"role": "assistant", "content": ex["response"]}
            ]
        })
    return formatted
```

### Pattern 2: DPO Preference Pair Generation

```
Instruction ──→ [Generate N responses] ──→ [Score/Rank] ──→ [Select Pairs]
                      │                         │                  │
                  temperature=0.8            human eval         chosen: best
                  N=4 responses per          or LLM judge       rejected: worst
                  instruction                or heuristic        margin > threshold

Output format (DPO):
{
  "prompt": "Explain quantum computing",
  "chosen": "Quantum computing uses qubits...",    # High-quality response
  "rejected": "Quantum computing is computers..."  # Lower-quality response
}
```

### Pattern 3: Synthetic Data Augmentation

```
10 Gold Examples ──→ [LLM Generator] ──→ [Quality Filter] ──→ 500 Examples
                          │                      │
                     "Generate 50            remove duplicates,
                      variations of          score perplexity,
                      this instruction       human spot-check
                      with different          10% sample
                      personas and
                      complexity levels"
```

## Anti-Patterns

1. **Quantity over quality** — 1,000 high-quality examples outperform 50,000 noisy ones. Filter aggressively; a model trained on garbage outputs garbage.
2. **No deduplication** — Duplicate or near-duplicate examples cause the model to memorize specific phrasings instead of learning patterns. Always dedup.
3. **Ignoring distribution balance** — If 80% of your data is one category, the model will be biased. Track category distribution and balance or weight accordingly.
4. **Training on the eval set** — Accidentally including test examples in training data produces inflated metrics. Deduplicate between train/eval splits.
5. **Skipping PII removal** — Training on data with real names, emails, and phone numbers bakes PII into model weights. Use presidio or spaCy NER to scrub before training.

## Quality Checklist

- [ ] Gold-standard examples written by hand (minimum 10) before scaling
- [ ] Raw data filtered: minimum response length, maximum length, non-empty instructions
- [ ] Near-deduplication applied (MinHash threshold 0.8 or tighter)
- [ ] Train/eval split with deduplication across splits (no leakage)
- [ ] PII removed from all examples (names, emails, phone numbers, addresses)
- [ ] Distribution balanced across categories/topics (or intentionally weighted)
- [ ] Format matches target training framework (chat-ml, Alpaca, OpenAI JSONL)
- [ ] Augmented data quality-checked via spot-sampling (10% human review)
- [ ] DPO pairs have sufficient quality margin between chosen and rejected
- [ ] Final dataset stats documented: size, categories, avg tokens, quality score distribution