Synthetic Data Workflow

---
name: synthetic-data-workflow
description: >-
  Privacy-preserving workflow for building synthetic datasets and data source skills when raw data must never enter the DAAF container. The user profiles their sensitive data locally with a disclosure-controlled script; only a summary profile report crosses the boundary; DAAF builds a synthetic dataset and skill from the report alone. Use whenever data is sensitive, proprietary, PII-bearing, HIPAA/FERPA-governed, held in a secure enclave, or the user says the data cannot leave their environment, they cannot upload it, or asks to profile it locally. Covers a four-tier disclosure ladder (T1 schema, T2 marginals, T3 relationships, T4 local high-fidelity synthesis), profile-only generation with simstudy (R) / NumPy-SciPy copulas (Python), and three-part QA (disclosure-safety, report consistency, synthetic-vs-profile validation). Not for synthetic control, the causal-inference method; for that see data-scientist. Synthetic data here is a code-development scaffold, not an analytic substitute.
metadata:
  audience: any-agent
  domain: research-methodology
---

# Synthetic Data Workflow

Privacy-preserving methodology for the case where raw data cannot enter the DAAF container. The user runs a disclosure-controlled profiling script on their own machine (inside their secure enclave, their laptop, wherever the sensitive data lives), reviews the human-readable summary it produces, and hands DAAF only that summary — a JSON profile report plus a plain-text review copy. DAAF then constructs a seeded synthetic dataset whose structure and (tier-permitting) marginals and relationships match the report, and authors a data source skill carrying explicit synthetic-provenance metadata. This skill provides the doctrine, the disclosure ladder, the profiling-report specification, the generation patterns (R-first, Python second), the local high-fidelity synthesis option, and the validation model. It is domain-agnostic infrastructure, not tied to any dataset. Triggers include sensitive/proprietary/PII data, secure enclaves, "data can't leave my environment," local profiling, and disclosure control. It is emphatically **not** the synthetic *control* method (a causal-inference estimator) — that lives in `data-scientist` (`references/causal-synth.md`).

## The Cardinal Doctrine — Read First

**Synthetic data is a code-development scaffold, not an analytic substitute.** A synthetic dataset built from a profile is *structurally* valid (right columns, right types, plausible-looking values, approximately right marginals) but *statistically* invalid (the joint distribution, conditional relationships, tail behavior, and missingness mechanism are approximations at best and fabrications at worst). Its purpose is to let DAAF and the user develop, debug, and dry-run analysis code against something shaped like the real data — never to produce findings.

This is settled practice at statistical agencies, not a DAAF invention:

- **CMS DE-SynPUF** (the Medicare synthetic claims file) is distributed explicitly "for development and testing," with the standing warning that "univariate statistics and regression coefficients ... will be biased" — it is for building pipelines, not drawing conclusions (research: `synthetic-data-research.md` §3).
- **OpenSAFELY dummy data** runs the *same analysis command* against fake data locally and real data inside the enclave; its dummy data is deliberately "cleaner than real" — it will not reproduce impossible values, real missingness, or true comorbidity structure (same source).

The operational consequence, which must be stated in every synthetic-data skill and every report built this way: **all findings must be finalized by re-running the vetted analysis code against the real data, inside the environment where the real data lives.** Results computed on synthetic data are provisional scaffolding and nothing more. When a user starts treating synthetic numbers as answers, stop and re-anchor them to this doctrine.

## When This Workflow Applies

```
Does raw data need to stay out of the container?
├─ No — user can bring the data in
│   └─ Use standard Data Onboarding (data-ingest profiles the real file in-container)
├─ Yes — sensitive / proprietary / PII / enclave / "can't leave my environment"
│   └─ THIS workflow:
│       1. DAAF prepares a disclosure-controlled profiling script (assets/ templates)
│       2. User runs it locally, reviews the .txt summary, returns the JSON report
│       3. DAAF validates the report, interprets it, generates synthetic data,
│          validates synthetic-vs-profile, and authors a synthetic-provenance skill
└─ User wants higher fidelity than a profile can carry
    └─ T4: user runs a local synthesizer (synthpop / SDV) inside their environment;
       only synthetic ROWS cross the boundary — never the real data or the fitted model
```

## Workflow Stages

This skill is the knowledge layer for a sub-workflow of Data Onboarding (stage wiring and orchestration live in the mode reference — dispatch 2). The stages the knowledge supports:

| Stage | Name | Who acts | This skill's role |
|-------|------|----------|-------------------|
| DS-1 | Script Preparation | DAAF (research-executor) | Configure a profiling-script template for the user's file/tier — see `profiling-script-spec.md`, `assets/profile_data_template.*` |
| DS-2 | User Local Run | **User (outside container)** | User edits the Config block, runs the script, reviews the `.txt` summary, returns the JSON |
| DS-3 | Report Intake & Validation | DAAF (code-reviewer) | Internal-consistency checks on the returned report — see `validation-checks.md` QA(b) |
| DS-4 | Interpretation | DAAF (data-ingest) | Read the report as a data dictionary; draft skill sections |
| DS-5 | Synthetic Generation & Validation | DAAF (research-executor + code-reviewer) | Generate seeded synthetic parquet, then validate synthetic-vs-profile — `generation-patterns-r.md` / `-python.md`, `validation-checks.md` QA(c) |
| — | rejoin DI-7 / DI-8 | DAAF | Author the skill with synthetic-provenance metadata (template wiring is dispatch 3) |

## The Four-Tier Disclosure Ladder

Each tier is a strict superset of the one below. The user picks the *lowest* tier that still lets the intended code development proceed — the Five Safes principle that "safe data" is a residual: remove only as much protection as the task requires you to keep (research §3). Full tier definitions, exactly what each emits and forbids, and the suppression rules are in `disclosure-tiers.md`.

| Tier | Name | What crosses the boundary | Never crosses |
|------|------|---------------------------|---------------|
| **T1** | Schema | Column names, dtypes, row count | Any values, any statistics |
| **T2** | Marginals (default) | Per-column: categorical levels with small-cell suppression (threshold default 5) + rare-category binning (a sub-threshold `__OTHER__` is folded further); numeric percentiles (p1..p99) + mean/SD — degraded to quartiles-only for small-n and value-withheld for near-constant columns; missingness rates; string-length stats + pattern flags | Raw min/max, example string values, small-cell counts, identifier values |
| **T3** | Relationships | Everything in T2 + Pearson/Spearman correlation matrices (numeric), Cramér's V (categorical pairs), named numeric~numeric summaries (OLS slope/intercept/R² + correlations), cross-tabs with primary + complementary cell suppression (suppressed cells emitted as `null`) | Same forbiddens as T2, plus unsuppressed cross-tab cells |
| **T4** | Local high-fidelity synthesis | Only synthetic *rows*, generated locally by synthpop (R) / SDV (Python) fit on the real data inside the user's environment | The real data AND the fitted model artifacts — both stay local |

**Suppression is ON by default at T2 and above.** Small cells (below the threshold) are suppressed; rare categorical levels are binned to `__OTHER__`; only percentiles are emitted (never raw min/max, an outlier-disclosure risk per the UK Data Service SDC handbook); example string values are never emitted; columns flagged as likely identifiers get structure-only treatment — dtype, uniqueness, length stats, never values. Identifier flagging is deliberately asymmetric: a high-uniqueness *string* column (a key like `client_id` or an email) is flagged, but a high-uniqueness *continuous numeric* is not flagged by uniqueness alone — it is a measurement, not a key, and percentiles-not-min/max already protect it; numerics are flagged only by an identifier-shaped name (`*_id`, `account`, `ssn`). Values matching email/phone/ID patterns flag a column regardless of type.

## Generation: Profile-Only vs Data-Fitted

The boundary that makes this workflow safe maps onto a clean library split (research §1-2):

- **Profile-only generators** build FROM declarations alone — marginal parameters plus a correlation matrix — with no microdata. These run *inside* DAAF on the returned report. **R (flagship): `simstudy`** (Gaussian copula via `genCorGen`/`addCorGen` from marginals + a correlation matrix; `fabricatr` noted for hierarchical/nested structure). **Python: hand-written NumPy/SciPy Gaussian-copula code** plus `Faker` for identifier-shaped columns. **SDV is deliberately NOT used for profile-only generation** — its synthesizers `fit()` on real rows, which DAAF never has.
- **Data-fitted synthesizers** learn from real microdata and therefore only ever run *locally, inside the user's environment* (T4). **R: `synthpop`** (CART synthesis, agency-grade, light dependencies — the flagship local option). **Python: SDV `GaussianCopulaSynthesizer`.** Only their synthetic output crosses the boundary.

Routing detail, worked examples, and caveats (e.g. simstudy recovers Poisson correlation more faithfully than binary; synthcity/CTGAN are heavier and reserved for users who explicitly want them) live in `generation-patterns-r.md`, `generation-patterns-python.md`, and `local-synthesis-t4.md`. **All generation is seeded for reproducibility**, and synthetic parquet is written to `data/synthetic/` within the research project.

## QA Model — Three Independent Checks

Because the real data never enters the container, the usual Data Onboarding QA move (recompute statistics against the source file) is impossible here. QA is restructured into three checks, specified in full in `validation-checks.md`:

1. **Disclosure-safety review of the OUTBOUND script (before the user runs it).** Does the configured profiling script emit anything the chosen tier forbids? This is the highest-stakes check — a leak here is irreversible once the report is shared. Reviewed by code-reviewer against the tier's forbidden-emissions list.
2. **Internal-consistency validation of the RETURNED report.** Percentiles monotone non-decreasing; category counts consistent with the row count and the suppression rule; correlation matrices symmetric with unit diagonal and PSD-tolerant; missingness rates in [0,1]; embedded validation-check results present and passing.
3. **Synthetic-vs-profile validation of GENERATED data.** Marginals within tolerance of the profile; correlations reproduced within tolerance; suppressed categories absent; row count matched; identifier columns structurally shaped but value-free.

## Reference Files

Load on demand — these are thorough by design (Level 3 loading; token cost only when read).

| File | Read when |
|------|-----------|
| `references/disclosure-tiers.md` | Choosing or explaining a tier; deciding what a tier may emit; setting the suppression threshold |
| `references/profiling-script-spec.md` | Preparing a profiling script (DS-1); the canonical JSON report schema (`report_version`, tier, suppression settings, per-column blocks, relationships block, embedded validation results) |
| `references/generation-patterns-r.md` | Writing R generation code (DS-5) — simstudy copula from a profile, fabricatr for hierarchy |
| `references/generation-patterns-python.md` | Writing Python generation code (DS-5) — NumPy/SciPy copula, Faker identifiers |
| `references/local-synthesis-t4.md` | Preparing the T4 local synthesizer templates (synthpop / SDV) the user runs in their environment |
| `references/validation-checks.md` | Any of the three QA checks — disclosure-safety, report consistency, synthetic-vs-profile |
| `references/synthetic-data-research.md` | Grounding evidence — the library split (§1-2), disclosure-control practice (§3), and failure modes (§4) the workflow rests on; cited throughout for source URLs |

## Asset Templates

Self-contained, zero-DAAF-dependency scripts the user runs where DAAF cannot reach. Copy and configure — never execute the shipped asset in place (execution logs would pollute the pristine template).

| Asset | Purpose |
|-------|---------|
| `assets/profile_data_template.R` | Disclosure-controlled profiler (R, flagship) — base R for CSV, optional arrow/haven; tier-parameterized; emits JSON + `.txt` review summary |
| `assets/profile_data_template.py` | Disclosure-controlled profiler (Python) — stdlib + pandas, optional pyarrow; same tier/config/dual-output design |
| `assets/synthesize_local_template.R` | T4 local high-fidelity synthesis (R) — synthpop CART, emits only synthetic rows + a generation log |
| `assets/synthesize_local_template.py` | T4 local high-fidelity synthesis (Python) — SDV GaussianCopula, same discipline |

## Boundaries

- **Never** ask the user to send raw data, raw extracts, unsuppressed cross-tabs, example values, or the fitted synthesis model. If a request would require any of these, stop and re-scope to a tier that does not.
- **Never** present synthetic-data results as findings. Every deliverable built on synthetic data carries the scaffold-not-substitute caveat and the "finalize against real data" requirement.
- **Always** have the user review the `.txt` summary before they share the JSON report — the human disclosure review is a required gate, not a formality.
- **Always** seed generation and record the seed for reproducibility.
- The disclosure-safety review of the outbound script is the single most important control in this workflow — treat a possible leak as a BLOCKER, never a WARNING.