Cikm Artifact Evaluation

---
name: cikm-artifact-evaluation
description: Use when packaging the code, datasets, knowledge graphs, prompts, and demo systems around a CIKM paper — choosing the artifact form per track (research, applied, resource, demo), meeting the resource track's reuse-and-documentation bar, and staging anonymous review artifacts into citable public releases.
---

# CIKM Artifact Evaluation

CIKM has no verified badge-granting artifact track (2026 lineup: full, short,
applied, resource, demo — source map, 2026-07-08), but artifacts do more work here
than at most venues, because two of the five tracks are *about* artifacts: Resource
papers publish them, and Demonstration papers perform them. This skill covers the
artifact strategy per track and the staging from anonymous review object to public
citable release.

## Artifact role by track

| Track | The artifact is... | Bar it must clear |
|---|---|---|
| Full / Short Research | Supporting evidence, cited anonymously | Regenerates headline results; survives a ten-minute inspection |
| Applied Research | Proof of practice | Substantiates the launch/deployment claim the track requires |
| Resource | **The contribution itself** | Documented, licensed, maintained, adoptable by strangers |
| Demonstration | The thing being demonstrated | Runs live, in front of people, on conference-venue conditions |

## Resource-track packaging bar

A CIKM resource paper is graded on whether others will build on the object. The
verified call names data resources, software, evaluation tasks, and open-source
frameworks. Package accordingly:

- **Provenance and license first**: where the data came from, what consent/terms
  cover it, and a license a university lab and a company can both act on. An
  unlicensed dataset is unadoptable regardless of quality.
- **Schema and statistics**: field-level documentation, size, class balance, known
  gaps — the datasheet discipline, applied without the buzzword.
- **Baselines included**: a resource ships with reference numbers and the scripts
  that produce them, or every adopter re-invents the evaluation differently.
- **Maintenance posture**: versioning scheme, issue channel, and an honest
  statement of how long the hosting is funded. Reviewers of resource papers have
  watched too many datasets rot at dead URLs.
- All of it inside 4 pages (2026) — the paper is the resource's brochure and
  datasheet condensed; depth lives in the repository.

## KG-specific packaging

Knowledge graphs — a CIKM staple — need artifact fields other objects skip: the
dump date of any upstream KG, the extraction queries or rules that produced the
subgraph, entity-resolution decisions (what merged, what split), and per-relation
counts. Two teams "using Wikidata" are usually using different graphs; the artifact
is where that ambiguity dies.

## Demo systems as artifacts

A demo paper's artifact must run where demos actually happen: conference Wi-Fi,
projector resolution, a visitor who types adversarial queries. Package a local
fallback (cached index, canned dataset, offline model), a 90-second scripted tour,
and a reset mechanism that restores clean state between visitors. The 4-page paper
should include the architecture and what a visitor experiences — reviewers accept
demos they can imagine standing in front of.

## Licensing quick guide

License choice is an adoption decision, and CIKM's mixed academic/industry audience
makes it consequential:

| Object | Workable defaults | Watch out for |
|---|---|---|
| Code | MIT/Apache-2.0 for maximum reuse | Copyleft blocks industrial adopters — choose deliberately, not accidentally |
| Original data | CC BY (attribution) or CC BY-NC | NC clauses exclude exactly the applied audience CIKM attracts |
| Derived data (from a KG, a crawl, an API) | Whatever the upstream terms force | Upstream terms propagate: a Wikidata extraction, a scraped corpus, and an API dump each carry different obligations |
| Models/checkpoints | Code license or a stated model license | Base-model license inheritance for fine-tuned checkpoints |

Two failure modes recur in resource-track reviews: no license at all (unadoptable),
and a license the authors had no right to grant because upstream terms forbid
redistribution. Resolve the rights question before the paper claims release.

## Hosting for the decade, not the deadline

CIKM papers become baselines years later (`cikm-reproducibility`), so host on that
horizon: an institutional or platform repository under an organization account
rather than a personal one; an archival deposit with a DOI for the exact
proceedings-version snapshot; dataset hosting with a stated funding horizon rather
than a lab NAS URL. The cheapest insurance is the archival snapshot — it survives
graduations, lab moves, and platform pivots, all of which the venue's older
artifact links have visibly failed to.

## Anonymous-to-public staging

```text
Review stage:  anonymized hosting, no author-resolving URLs/metadata,
               referenced from the PDF where each claim needs it
Acceptance:    real repository public by camera-ready (13-day window in 2026 —
               prepare during the review wait)
Public stage:  license chosen; versioned release tagged to the proceedings
               numbers; archival DOI (e.g., a Zenodo-class deposit) so citation
               outlives the hosting; README linking the ACM DL record
```

The GenAI Usage Disclosure obligation follows the artifact too: generated code,
synthetic data, or AI-produced labels inside the package belong in the paper's
disclosure section (`cikm-reproducibility`).

## Naming and versioning conventions

Give the artifact the paper's short name (the mechanism name from
`cikm-writing-style`), version it semantically with the proceedings snapshot
tagged (e.g., `v1.0-cikm2026`), and keep the README's first line identical to the
paper title so search engines and readers connect the two objects. Post-
publication improvements go in later versions with a changelog; the tagged
snapshot stays frozen because it is what the paper's numbers mean.

## Output format

```text
[Track role] evidence / proof-of-practice / the-contribution / the-performance
[Package state] provenance / license / docs / baselines / maintenance
[KG fields] dump date / extraction / resolution decisions (if applicable)
[Staging] review-anonymous → public-citable, with dates
[Adoption blocker] <the single thing most likely to stop a stranger from using it>
```