K2 Vetting Eval

---
name: k2-vetting-eval
description: Evaluates automated exoplanet vetting pipelines on K2 transit lightcurves by comparing their planet candidate versus false positive dispositions against established ground truth from the NASA Exoplanet Archive. It probes the ability of tools to correctly identify true transiting planets while filtering out astrophysical false positives like eclipsing binaries and blended stars. Use when the user wants to benchmark on K2 Planet Candidate Catalog (DAVE Benchmark), or asks about evaluating this task. Reports disposition.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1901.07459
  bibtex_key: kostov2019dave
  confidence: high
---

# k2-vetting-eval

> Discovery and Vetting of Exoplanets I: Benchmarking K2 Vetting Tools — Kostov et al. (2019) (arXiv:1901.07459, 2019)

## What this evaluates

Evaluates automated exoplanet vetting pipelines on K2 transit lightcurves by comparing their planet candidate versus false positive dispositions against established ground truth from the NASA Exoplanet Archive. It probes the ability of tools to correctly identify true transiting planets while filtering out astrophysical false positives like eclipsing binaries and blended stars.

## Datasets

- **K2 Planet Candidate Catalog (DAVE Benchmark)** — total 772; splits: test (772); repo https://github.com/exoplanetvetting/DAVE

## Metrics

- `disposition` **(primary)** — range: other
  - Binary classification of each target as either a planet candidate (“PC”) or a false positive (“FP”), evaluated against NExScI ground truth. Calculated as the proportion of targets where the automated disposition matches the confirmed status.

## Input / output format

**Input**: EPIC ID, transit properties, and lightcurves from up to four detrending pipelines (AGP, EVEREST, PDC, SFF) for each K2 target.

**Output**: Binary disposition (“PC” or “FP”), reason code (e.g., “CO” for centroid offset), and additional comments.

## Scoring recipe

```python
def score_disposition(predictions, gold):
    correct = 0
    for p, g in zip(predictions, gold):
        gold_label = 'PC' if g == 'confirmed' else 'FP'
        if p == gold_label:
            correct += 1
    return correct / len(predictions)
```

## Common pitfalls

- Lightcurve pipeline choice significantly affects SNR and dispositions; a target may be classified differently depending on whether AGP, EVEREST, PDC, or SFF data is used.
- Low SNR targets (SNR < 7 or < 4) are automatically flagged as FPs by DAVE but often require visual inspection to avoid false rejections.
- Ground truth from NExScI can be inconsistent, with some targets listed as both PC and FP, or conflicting with recent literature.

## Evidence (verbatim from paper)

> The catalog lists the EPIC ID of each K2 planet candidate, the transit properties, disposition (planet candidate “PC” or false positive “FP”), the reason for the disposition (e.g. centroid offset during transit, as “CO”), as well as additional comments (for example, presence of field stars in the target’s aperture, as “FSAp”).

## Citation

```bibtex
@misc{kostov2019dave,
  title={Discovery and Vetting of Exoplanets I: Benchmarking K2 Vetting Tools},
  author={Kostov et al. (2019)},
  year={2019},
  note={arXiv:1901.07459}
}
```

- arXiv: 1901.07459