Micro Reproducibility

---
name: micro-reproducibility
description: Use when hardening a MICRO paper's results for re-derivation — pinning simulator commits and configs, recording workload trace provenance and SimPoint recipes, versioning power/area models, capturing RTL toolchain state, and writing the run manifests that later survive MICRO's post-acceptance artifact evaluation.
---

# MICRO Reproducibility

In microarchitecture research the "experiment" is usually a software artifact
pretending to be hardware. That makes results *perfectly* reproducible in
principle — and silently unreproducible in practice, because a simulator commit, a
config diff, or a trace-generation flag changed between the headline run and the
camera-ready. This skill installs the record-keeping that prevents that.

## What must be pinned, by evidence type

| Evidence | Pin these or the number is unrecoverable |
|---|---|
| Cycle-level simulation | Simulator name + exact commit hash; full config files (not "8-wide OoO" prose); patch set applied; host compiler for the simulator build |
| Workload traces | Source binary + compiler flags; trace tool + version; SimPoint interval size, K, and seed; warmup instructions per region |
| Power / area estimates | McPAT/CACTI/Accelergy version; technology node; the XML/cfg input actually fed in; any manual scaling factors |
| RTL results | HDL commit; synthesis tool + version; target library/PDK; constraints file; which corners were run |
| FPGA / silicon | Board/chip stepping; bitstream hash or firmware; frequency; measurement method for power (rail vs counter model) |
| DRAM behavior | Ramulator/DRAMsim3 version + timing config (device datasheet parameters) |

The rule of thumb: **a result is reproducible when a stranger can regenerate the
exact bar in Fig. 3 from the manifest alone**, without asking you anything.

## The run manifest

Emit one manifest per experiment batch, generated by the launch script — never
written by hand after the fact:

```yaml
# results/2026-04-02_headline/manifest.yaml
simulator: {name: champsim, commit: 3fa9c21, patches: [prefetch-fix.diff]}
config:    {file: configs/8w_ooo_baseline.json, sha256: 9be1...}
mechanism: {file: configs/ours_32kb.json, knobs: {table_entries: 2048, tag_bits: 12}}
workloads: {suite: spec2017_rate1, traces: traces/v3/, simpoints: k30_int200M_seed7,
            warmup_M: 200, detailed_M: 1000}
power:     {tool: mcpat, version: "1.3", node_nm: 22, input: mcpat/ours.xml}
host:      {machine: lab-node-14, compiler: gcc-13.2}
outputs:   {raw: stats/, figures: fig3_headline.pdf, script: plot_headline.py}
```

Commit the manifest with the raw stats files. Figures are regenerated from stats by
a script named in the manifest — a figure that cannot be regenerated does not go in
the paper.

## Disclosure lines the paper itself owes

The methodology section (see `micro-writing-style`, ~1 page) must state, in text:

- Simulator and version, with your modifications summarized honestly ("we added a
  cycle-accurate model of X; the interconnect remains the default model").
- The full baseline configuration in a table (pipeline width, ROB, caches with
  latencies, predictor, prefetchers, DRAM timing).
- Sampling recipe: how regions were chosen, warmup, and measured instruction counts.
- Power/area model, node, and the sentence "these are model estimates" — the
  community's tools (CACTI, McPAT — both MICRO papers themselves) are respected but
  known to be approximations, and pretending otherwise reads as naivety.

## Determinism and drift checks

- Two identical launches must produce bit-identical stats; if the simulator has any
  nondeterminism (thread interleaving in parallel sims), pin it or report run count
  and spread.
- Re-run the baseline after every simulator patch — a patch that changes baseline
  IPC by 0.5% invalidates every relative number already collected.
- Keep a `results-ledger.md`: date, manifest path, headline geomean, what changed.
  Rebuttal season (June) will ask "what config produced Fig. 7?" — from three
  months earlier; the ledger answers in minutes, memory does not.

## Validating the instrument itself

Reproducibility of a wrong number is worthless; record the evidence that the
simulator models what the paper says it models:

- **Cross-check the baseline against published behavior** (see
  `micro-experiments`): store the comparison table — suite, metric, published
  range, your value — in the repo, not just in someone's memory.
- **Unit-test the mechanism model.** Hand-construct access sequences with known
  correct outcomes (a burst that must train the table, a pattern that must not)
  and assert the simulator's counters match. These micro-tests are the first
  thing an AE evaluator can run and the best evidence the model is not a stats
  bug.
- **Record known fidelity gaps** in a `LIMITS.md`: default interconnect model,
  simplified TLB, unmodeled prefetcher interactions. The paper's claims must
  stay inside these limits; writing them down keeps deadline-week edits honest.

## Anonymity interaction (submission phase)

MICRO 2026 requires artifact links to be fully anonymized or removed. Before the
April upload, sweep the repo you intend to link: git author fields, hostnames in
manifests (`lab-node-14` is fine; `patt-lab-utexas-14` is not), cluster paths,
funding strings in READMEs. The manifest discipline above makes this sweep
mechanical rather than archaeological.

## Storage economics of keeping everything

Raw cycle-level stats for a full paper (dozens of configs × dozens of workloads ×
sweeps) usually fit in single-digit gigabytes — trivially archivable, and the
cheapest insurance in research. Policy: keep **all** raw stats files, forever,
compressed; regenerate figures on demand. Delete only simulator *checkpoints* and
intermediate trace files that the recipes can rebuild. When a reviewer asks in
June about a bar from a January run, or AE asks in August, or a follow-up paper
asks in two years, the manifest + stats pair answers; the alternative is
re-simulating weeks of compute under deadline pressure.

## Output format

```text
[Pinning audit] per evidence type: pinned / partially pinned / loose (list)
[Manifest coverage] experiments with manifests: N/M
[Figure regen] all paper figures scripted from raw stats: yes / no (list unscripted)
[Methodology text] simulator+mods / baseline table / sampling / model-estimate
                   disclosure: present-absent each
[Determinism] identical-rerun check passed: yes / no / untested
[Ledger] results-ledger current through: <date>
[Anonymity sweep] repo clean for anonymized linking: yes / issues listed
```