Conext Artifact Evaluation

---
name: conext-artifact-evaluation
description: Use when converting an accepted ACM CoNEXT paper's package into optional ACM reproducibility badges via the CoNEXT reproducibility committee — from the badge opt-in due before submission, to the one-page artifact description due within a week of acceptance, to the Available / Functional / Reusable / Reproduced criteria and evaluator-proof reuse docs.
---

# CoNEXT Artifact Evaluation

Turn the accepted paper's package into **optional ACM reproducibility badges**, administered by
CoNEXT's **reproducibility committee**. This is a distinct process from the paper review, with its
own timing: the decisive move — **opting in before the paper submission deadline** — happens long
before you have an accept. Treat badging as a track you enter at submission and complete after
acceptance.

## The badge timeline (opt-in is the gate)

```text
[Before submission deadline]  OPT IN for ACM badging  <-- miss this and you are ineligible this cycle
[Within ~1 week of accept]    send a ONE-PAGE artifact description + pointers to the committee
[Post-accept evaluation]      committee reviews the artifact against the ACM badge criteria
[Camera-ready]                approved badges appear on the PACMNET article (see conext-camera-ready)
```

## The ACM badges (what each requires)

CoNEXT uses the ACM Artifact Review and Badging scheme. In practice the ladder is:

| Badge | What it certifies | What you must provide |
|---|---|---|
| **Artifacts Available** | The artifact is publicly archived | A DOI-issuing archive (Zenodo/figshare/Software Heritage) with an open license |
| **Artifacts Evaluated — Functional** | It is documented, complete, and runs | Clear docs, a runnable package, and a "getting started" path that works |
| **Artifacts Evaluated — Reusable** | It is functional *and* others can build on it | Reusability docs, clean structure, parameterization, and a license that permits reuse |
| **Results Reproduced** | Key results were regenerated by evaluators | A scripted path that regenerates the paper's central figures/tables from the inputs |

Confirm the exact badge set and names offered by the current CoNEXT reproducibility committee — the
scheme is stable but wording is cycle-volatile (**待核实**).

## The one-page artifact description

The committee starts from your one-pager, so make it do work:

- **Inventory:** code, traces, configs, testbed scripts — with a pointer to each.
- **Requirements:** the hardware/software an evaluator needs. Networking artifacts often need
  **specific hardware** (a particular switch ASIC, NIC, or SDR); if evaluators need access to
  hardware you must provide, flag it prominently and describe how you will give access.
- **Claims supported:** which paper results the artifact reproduces, and which it cannot (be honest
  about hardware-bound results).
- **Getting started:** the shortest path to a first successful run.

## Networking-specific evaluation hurdles

- **Hardware dependence.** A result that needs a specific switch or NIC is hard for a remote
  evaluator; provide a testbed reservation, a remote-access path, or a faithful emulation for the
  parts that can run in software, and clearly mark what is hardware-bound.
- **Trace size and privacy.** Large or sensitive traces may not be fully releasable; ship a
  representative or synthetic subset plus the pipeline, and document the restriction.
- **Environment drift.** Pin firmware/OS/kernel versions and provide images; "works on our testbed"
  is not reproducible.
- **Live dependencies.** Cache anything that would otherwise require a live API/network call so the
  evaluator's run is deterministic.

## Reusability that survives an evaluator

- A README that assumes nothing about your lab's setup; every path and dependency explicit.
- A single command (or a short script) for the "getting started" run, and a documented full-reproduction
  path for the central results.
- An open license present and consistent across the archive.
- A mapping from **paper claim → artifact step → expected output**, so an evaluator can check
  "Results Reproduced" without guessing.

## Pre-evaluation checklist

```text
[Opt-in]        confirmed before the submission deadline? yes/no
[Archive]       DOI-issuing archive + open license? yes/no
[One-pager]     inventory + requirements + hardware-access plan + claims map? yes/no
[Functional]    fresh-environment run succeeds from the docs alone? yes/no
[Reusable]      structure + docs let a third party adapt it? yes/no
[Reproduced]    central figures/tables regenerate by script? yes/no
[Hardware]      hardware-bound results flagged; access path provided? yes/no
```

## Output format

```text
[Badge target] Available / Functional / Reusable / Reproduced (which are in scope)
[Opt-in status] done before submission? yes/no
[Artifact readiness] <per-badge gaps>
[Hardware plan] <how remote evaluators reach any required hardware>
[One-pager] <drafted / sent to committee>
```