Jop Data Analysis

---
name: jop-data-analysis
description: Use for analysis-stage decisions on a The Journal of Politics (JOP) manuscript — uncertainty, robustness, and reporting norms — written so the work is reproducible from line one. JOP makes acceptance contingent on replicability and a JOP replication analyst re-runs the code, so every reported number must come from a script. Guides analysis; it does not fabricate results.
---

# Data Analysis (jop-data-analysis)

At JOP, analysis and **reproducibility are the same task**: acceptance is **contingent on
replicability**, and a **JOP replication analyst** re-runs your code at conditional acceptance. Write the
analysis so that every number in the paper is **regenerated by a script** — and reported with honest
uncertainty within the **page budget**.

## When to trigger

- Setting up the estimation/analysis pipeline
- Deciding which robustness checks belong in the main text vs the Online Appendix
- A reviewer asked for additional specifications, uncertainty, or sensitivity
- Preparing numbers that must match the deposited replication package exactly

## Analysis norms

- **Report uncertainty**, not just point estimates: CIs, SEs (clustered appropriately), and
  substantive effect sizes a general reader can interpret.
- **Specification transparency**: show the primary specification clearly; relegate the grid of
  alternatives to the Online Appendix, but reference it.
- **Robustness that targets the threat**: each check should answer a specific objection (confounding,
  functional form, sample, measurement), not pad the count.
- **Multiple comparisons**: adjust or pre-specify when testing many implications.
- **Substantive interpretation**: translate coefficients into quantities of interest (predicted
  probabilities, marginal effects) — general-interest readers want magnitudes, not just stars.

## Reproducible-from-line-one (the JOP analyst will re-run this)

- One **master script** runs everything in order and sets the working directory once.
- **Set a seed** for every stochastic step (bootstrap, simulation, MCMC, jitter, sampling).
- **Record software and package versions** for the readme (e.g., "R 4.3.1", "Stata/MP 18.0").
- Build a **codebook** naming and defining every variable used in the analysis.
- Tables and figures are **generated by code**, never hand-edited — numbers in the text must match.

## Fit the analysis to the page budget

- Lead with the result that carries the argument; do not narrate every regression.
- Move the robustness grid, balance tables, and diagnostics to the **Online Appendix (≤ 25 pp)**.
- A Short Article (≤ 10 pp) should show one clean, decisive analysis, not a buffet.

## Anti-patterns

- Numbers in the manuscript that the deposited code cannot reproduce (fails the analyst check)
- Unseeded randomness or unpinned versions ("works on my machine")
- Star-gazing with no effect sizes or uncertainty a general reader can use
- Robustness checks chosen to inflate the count rather than rebut a threat
- Cramming every specification into the main text and blowing the page budget

## Output format

```
【Primary result】estimand + magnitude + uncertainty
【Robustness】each check ↔ the threat it answers (main vs appendix)
【Reproducible】master script + seeds + pinned versions + codebook? [Y/N]
【Numbers match】text == deposited output? [Y/N]
【Page discipline】main text lean, overflow in appendix? [Y/N]
【Next】jop-tables-figures
```

## Supplementary resources

- [`../../resources/external_tools.md`](../../resources/external_tools.md) — estimation packages and reproducibility tooling (renv, seeds, version pinning)
- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — JOP replicability-contingent acceptance and replication-analyst check