Ndss Artifact Evaluation

---
name: ndss-artifact-evaluation
description: Use when preparing an NDSS artifact for evaluation after conditional acceptance — targeting the Available, Functional, and Reproduced badges, packaging attacks and measurements responsibly, writing the 2-page artifact appendix, and handling dangerous or embargoed material.
---

# NDSS Artifact Evaluation

NDSS added artifact evaluation to the symposium in recent editions (running on the
community `secartifacts` platform, verified for 2024-2026), and it is **opt-in after
conditional acceptance** — a separate track, not a submission requirement. Papers that pass
carry badges on the first page and add a 2-page artifact appendix (Call for Artifacts,
checked 2026-07-08). Decide early whether you are engineering for it, because the packaging
choices are cheap during the project and expensive afterward.

## The three badges

| Badge | Claim to the committee | What you must ship |
| --- | --- | --- |
| **Available** | The artifact is publicly, permanently retrievable | Deposit in an archival repository with a DOI (Zenodo, figshare); a bare GitHub URL is not "available" |
| **Functional** | It does what the paper says, is usable and documented | Build/run instructions that work on a clean machine; the components matching the paper's claims |
| **Reproduced** | The paper's results can be regenerated by others | One documented path per headline result, with expected output and tolerances |

Badges are independent — you can earn Available and Functional without Reproduced when the
world (live targets, proprietary data) genuinely prevents re-execution. Claim only what you
can defend; an evaluator who cannot reproduce a "Reproduced" claim writes that in the
report.

## From claims ledger to artifact

The `claims.yml` ledger built in `ndss-reproducibility` is the artifact's spine. Convert it
into an evaluator-facing package:

```text
artifact/
  README.md          # what this is, badges sought, hardware/time needs, safety notes
  INSTALL.md         # clean-machine setup; pinned deps / container image
  claims/            # one entry per headline result:
    fig4/  run.sh    # → expected_output/, runtime + resource note
    tab2/  run.sh
  data/              # snapshots or synthetic substitutes (never raw victim traffic)
  src/               # the code, with the dangerous parts gated (below)
  ETHICS.md          # disclosure status, gating rationale, usage boundaries
```

Give evaluators the shortest true path to each result. A "kick-the-tires" phase means the
first five minutes decide the evaluator's mood — make the smoke test one command.

## Packaging dangerous material responsibly

NDSS artifacts routinely include working exploits, attack tooling, or malware-adjacent
code. This is allowed; unbounded release is not.

- **Gate the weaponized parts.** Ship the analysis and measurement pipeline openly; put
  live exploit modules behind documented access conditions or a defanged demonstration
  mode consistent with your Ethics Considerations section.
- **Align with the disclosure clock.** If a vulnerability is under a coordinated-disclosure
  embargo, the artifact's release timing must respect it — state the date the full module
  becomes available.
- **Sandbox by default.** Anything that touches a network should target a supplied local
  testbed, not the Internet, out of the box. Document the blast radius.
- **No live victim data.** Replace with prefix-preserved or synthetic traces carrying the
  statistics the result needs (see `ndss-reproducibility`).

## When re-execution needs your lab

Hardware-bound results (SDRs, specific switches, IoT devices) cannot be reproduced by a
committee at their desks. Bridge the gap: provide recorded traces plus a replay harness so
the *analysis* is reproducible even when the *capture* is not, and say plainly in the
README which results are trace-replayable and which need the physical setup. This commonly
yields Functional + Reproduced-on-recorded-data rather than a failed full Reproduced.

## The 2-page artifact appendix

Written after evaluation for the camera-ready. It states the badges earned, the artifact's
scope, access instructions (the DOI), hardware/software requirements, and the mapping from
paper claims to artifact components. Keep it factual and navigational — it is a map for
future readers of the open-access paper, not a second results section.

## Timing

The opt-in comes shortly after (conditional) acceptance and runs on its own schedule
(2027 dates 待核实 — confirm on the Call for Artifacts). It overlaps the camera-ready
window (Jan 6, 2027), so a team that starts packaging at acceptance, not at the AE
deadline, avoids a two-front crunch.

## Output format

```text
[Badges targeted] Available / Functional / Reproduced — with feasibility per badge
[Package status] README / INSTALL / per-claim runners / data substitutes — done or gaps
[Danger handling] gated modules, sandbox default, disclosure-aligned release date
[Hardware gap] results needing physical setup + trace-replay bridge
[Appendix] 2-page artifact appendix drafted? claim→component map complete?
```