Kitchen Sink Anomaly Detection Eval

---
name: kitchen-sink-anomaly-detection-eval
description: Evaluates the robustness and sensitivity of various jet substructure feature sets (Energy Flow Polynomials, subjettiness, and their combination) for model-agnostic resonant anomaly detection in high-energy physics dijet events. It compares performance across Ideal Anomaly Detection (IAD) and CWoLa hunting setups using multiple Beyond Standard Model signal topologies. Use when the user wants to benchmark on LHCO & BSM dijet signals, or asks about evaluating this task. Reports max(SIC), sigma_0,min.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.20965
  bibtex_key: [REDACTED]
  confidence: high
---

# kitchen-sink-anomaly-detection-eval

> Kitchen Sink Anomaly Detection — Das et al. (2026) (arXiv:2604.20965, 2026)

## What this evaluates

Evaluates the robustness and sensitivity of various jet substructure feature sets (Energy Flow Polynomials, subjettiness, and their combination) for model-agnostic resonant anomaly detection in high-energy physics dijet events. It compares performance across Ideal Anomaly Detection (IAD) and CWoLa hunting setups using multiple Beyond Standard Model signal topologies.

## Datasets

- **LHCO & BSM dijet signals** — total ?; splits: train (-1), test (-1)

## Metrics

- `max(SIC)` **(primary)** — range: other
  - Maximum Significance Improvement Characteristic; measures the peak discovery significance improvement over the background-only expectation across varying signal injections ($N_{sig}$).
- `sigma_0,min` **(primary)** — range: other
  - Minimal initial signal significance required to achieve a 3σ evidence or 5σ discovery threshold, derived by linearly interpolating the median significance curve across 10 training runs.
- `regret (r_f)` — range: other
  - Relative performance gap of a feature set compared to the best-performing set, calculated as (metric_best - metric_f) / metric_best.

## Input / output format

**Input**: High-level jet substructure feature vectors (e.g., EFPs, N-subjettiness, baseline observables) computed for dijet events, optionally labeled as signal or background.

**Output**: Classifier decision scores or binary predictions from Gradient Boosted Decision Trees (GBDTs), used to compute discovery significance metrics.

## Scoring recipe

```python
# For each of 10 training samples:
#   train GBDT on features, predict on test set
#   compute Asimov significance for each signal injection N_sig
#   take max significance -> SIC
# Compute median SIC across 10 runs and interpolate
# Find initial significance where interpolated SIC >= 3σ or 5σ -> sigma_0,min
# Regret r_f = (metric_best - metric_f) / metric_best
```

## Common pitfalls

- Confusing the IAD setup (idealized, known signal distribution for training) with CWoLa hunting (unlabeled data, relies on background-to-signal ratio estimation).
- Assuming the 'Random' feature set represents random noise; it actually refers to random subsampling of features per tree to reduce training cost while maintaining performance.
- Overlooking that metrics are aggregated across multiple distinct BSM signal topologies, so a feature set's ranking can vary significantly depending on the specific decay topology (e.g., 2-prong vs. 4-prong).

## Evidence (verbatim from paper)

> For a 5σ discovery, we present the average and maximum of the minimal initial significance eq. (4) and regret eq. (5) for the different feature sets, over the investigated signal models in Table 3.

## Citation

```bibtex
@misc{das2026kitchensink,
  title={Kitchen Sink Anomaly Detection},
  author={Das et al. (2026)},
  year={2026},
  note={arXiv:2604.20965}
}
```

- arXiv: 2604.20965