Ajps Replication And Verification

---
name: ajps-replication-and-verification
description: "Use when building the replication package for an American Journal of Political Science (AJPS) manuscript. AJPS is famous for MANDATORY third-party PRE-publication verification — an independent verifier re-runs your deposited code and confirms it reproduces the numerical results in the main text before the article is published, with materials deposited to the AJPS Dataverse on Harvard Dataverse. Prepares the package; it does not waive requirements."
category: backend-and-data
source_repo: brycewang-stanford/Awesome-Journal-Skills
source_path: "American-Journal-of-Political-Science-Skills/skills/ajps-replication-and-verification/SKILL.md"
source_url: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/American-Journal-of-Political-Science-Skills/skills/ajps-replication-and-verification/SKILL.md
---


# Replication & Verification (ajps-replication-and-verification)

This is AJPS's signature differentiator. Unlike journals that check reproducibility in-house (or not at
all), **AJPS sends your deposited materials to an independent third party that actually re-runs the
code** and confirms it reproduces the numerical results reported in the **main text** — *before*
publication. If it does not reproduce, the article does not publish until it does. Build the package as
you analyze (`ajps-data-analysis`); it cannot be improvised at acceptance.

## When to trigger

- Building the replication / verification package
- A manuscript is accepted and you have received upload instructions (after final draft, during the
  technical check, before copyediting)
- Data cannot be fully shared (privacy, ethics, provider restriction) and you need a path
- A **qualitative** or **multi-method** paper whose evidence needs a documented verification route

## How AJPS verification actually works (official baseline checked 2026-06-20)

1. **Deposit to the AJPS Dataverse.** Materials go into a Dataset within the **AJPS Dataverse on the
   Harvard Dataverse Network** — not a personal site or generic cloud link. Other copies may exist
   elsewhere as long as everything needed is in the AJPS Dataverse Dataset.
2. **Independent third-party verification.** A verifier re-runs the code and confirms it reproduces the
   **numerical results in the main text**. The published article carries a statement: *"The Cornell
   Center for Social Sciences verified that the data and replication code submitted to the AJPS
   Dataverse replicates the numerical results reported in the main text of this article."* MPSA also
   identifies Cornell Center for Social Sciences as the contracted replication-and-verification provider.
3. **Qualitative path.** Qualitative / multi-method materials follow the AJPS verification guidelines
   and qualitative checklist: document collection procedures, instruments, analytic operations, source
   fragments or access restrictions, and any exemption route needed for human-subjects, copyright, or
   legal constraints.
4. **Timing.** It happens **after the final draft is submitted, during the technical check, before
   copyediting** — late enough that fixing an unscripted analysis under deadline is painful.

## Build-as-you-go checklist (so the re-run matches)

- [ ] **readme.txt** lists and describes every file (group as "Data files", "Stata .do files", "Files
      to Reproduce Table 1", ...)
- [ ] One **master script** runs the modular scripts **in order** and sets the working directory once
- [ ] **set seed** for every stochastic step (Monte Carlo, bootstrap, randomization inference, jitter)
- [ ] Explicit **software-version** statements (e.g., "R 4.3.2", "Stata/MP 18.0"); packages pinned
- [ ] Every **main-text** number, table, and figure is regenerated by the code, names matched
- [ ] Restricted data: explain why, give exact access instructions, provide synthetic data where feasible
- [ ] Sharing terms and any access restrictions documented in the Dataverse materials
- [ ] Preregistration / pre-analysis plan linked (anonymized) where applicable

## When data cannot be shared

- **Explain why** (ethics, privacy, legal/provider restriction).
- Provide **README instructions on exactly how others can obtain the data** (process, application, contact).
- Where possible, **provide synthetic data** resembling the real data so the code can run end to end.

## Anti-patterns

- Treating the deposit as a post-publication afterthought (it **gates** publication)
- Depositing code that does not actually regenerate the printed main-text numbers
- A personal URL instead of the AJPS Dataverse
- Unseeded, unpinned, hard-coded-path code that "works on my machine" but not on the verifier's
- Claiming data are restricted with no access path or synthetic substitute

## Output format

```
【Repository】AJPS Dataverse (Harvard) — Dataset staged? [Y/N]
【Reproduces main-text numbers?】master script re-run locally, matches? [Y/N]
【Files】readme.txt + master script + seeds + pinned versions? [Y/N]
【Verifier】Cornell Center for Social Sciences / AJPS qualitative checklist path
【Restricted data?】why + access path + synthetic data?
【Sharing terms】Dataverse/exemption notes complete?
【Next】ajps-review-process
```

## Supplementary resources

- [`../../resources/external_tools.md`](../../resources/external_tools.md) — replication-package tooling and qualitative-transparency options
- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — AJPS Replication & Verification Policy, verifier transition, and AJPS Dataverse

---

**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `American-Journal-of-Political-Science-Skills/skills/ajps-replication-and-verification/SKILL.md`