Joe Data Analysis

---
name: joe-data-analysis
description: Use when designing the Monte Carlo study and empirical illustration that demonstrate a Journal of Econometrics (JoE) method works in finite samples. Covers size/power simulation design, DGP stress tests, and the role of the applied illustration relative to the theory.
---

# Monte Carlo & Empirical Illustration (joe-data-analysis)

## When to trigger

- The theorems are settled but the finite-sample evidence is thin or one-off
- A simulation reports point estimates but no size/power, or never stresses the assumptions
- You are unsure how large or how diverse the Monte Carlo design must be
- You have an empirical illustration but it is doing the wrong job (over- or under-claiming)

## What "data analysis" means at a methodology journal

At the *Journal of Econometrics* the empirical work serves the **method**, not the other way around. A theorem describes behavior as $n\to\infty$; the **Monte Carlo** shows the asymptotics bite at realistic sample sizes, and the **empirical illustration** shows the method is usable and yields a sensible answer on real economic data. The applied illustration is a demonstration, **not** the paper's primary contribution — purely applied work without a methodological advance is out of scope here. Build both as evidence that the formal claims hold.

## Monte Carlo design

### Report the right quantities
- **Estimators:** bias, RMSE, coverage of confidence intervals.
- **Tests:** empirical **size at nominal 5%/10%**, then **size-adjusted power** curves. Over-rejection that vanishes only at huge $n$ is a finding, not a footnote.
- Compare against the **nearest existing method** on identical DGPs (ties back to `joe-literature-positioning`).

### Stress the assumptions, do not flatter them
- Vary **sample size** (including small $n$ where asymptotics may fail).
- Vary the **DGP**: error distributions (heavy tails, heteroskedasticity), dependence (serial/cluster/spatial), degree of endogeneity or identification strength, dimension.
- Vary **tuning parameters** (bandwidth, lag length, penalty, number of moments) and show sensitivity.
- Include designs **near the boundary** of your conditions — that is where referees look.

### Computational hygiene
- Fix and **report seeds**; report **replication count** and Monte Carlo standard errors so differences are not noise.
- Parallelize heavy designs; record runtime/hardware for the replication archive.

## Empirical illustration

- Pick a dataset where the method's advantage is **visible** (the problem it solves actually occurs).
- Show the **method changes a conclusion** or sharpens inference relative to standard practice — that is the payoff.
- Keep claims proportionate: this is an illustration of the tool, not a causal study. Do not oversell the applied finding.
- Cite the data with the Elsevier **`[dataset]`** tag and prepare materials for the archive (see `joe-replication-and-data-policy`).

## Anti-patterns

- A single DGP at one sample size "confirming" the theory
- Reporting raw power without empirical size (size-distorted power is meaningless)
- Hiding tuning-parameter sensitivity or boundary cases
- An empirical section that drifts into an applied paper the method only decorates
- Unreported seeds / replication counts, so results are not reproducible

## Output format

```
【MC estimators】bias / RMSE / coverage reported? [Y/N]
【MC tests】size at 5%/10% + size-adjusted power? [Y/N]
【DGP stress】distributions / dependence / tuning / boundary? [list]
【Benchmark】compared to nearest method on same DGP? [Y/N]
【Reproducibility】seeds + reps + MCSE reported? [Y/N]
【Illustration】method changes/sharpens a real conclusion? [Y/N]
【Next step】joe-tables-figures
```