Jet Tagging Resilience Eval

---
name: jet-tagging-resilience-eval
description: Evaluates the trade-off between classification performance (AUC) and model resilience (robustness to Monte Carlo simulation variations) in quark/gluon and top-quark jet tagging. It probes whether complex neural architectures generalize better to different physics simulators compared to simpler, physics-informed models. Use when the user wants to benchmark on Pythia 8 / Herwig 7 Jet Samples, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.19431
  bibtex_key: [REDACTED]
  confidence: high
---

# jet-tagging-resilience-eval

> The Pareto Frontier of Resilient Jet Tagging — Gambhir et al. (2025) (arXiv:2509.19431, 2025)

## What this evaluates

Evaluates the trade-off between classification performance (AUC) and model resilience (robustness to Monte Carlo simulation variations) in quark/gluon and top-quark jet tagging. It probes whether complex neural architectures generalize better to different physics simulators compared to simpler, physics-informed models.

## Datasets

- **Pythia 8 / Herwig 7 Jet Samples** — total ?; splits: train (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve, measuring binary classification performance across all decision thresholds.
- `Resilience` — range: percent
  - Percent difference in AUC between models evaluated on Pythia-generated test samples versus Herwig-generated test samples. Calculated as (AUC_Pythia - AUC_Herwig) / AUC_Pythia * 100.

## Input / output format

**Input**: Jet event features or raw particle data (e.g., angularities, energy-flow maps, or particle lists) representing quark/gluon or top/bottom/light jets.

**Output**: Binary classification probability or score indicating the likelihood of a jet belonging to a specific flavor (e.g., quark vs. gluon, or top vs. background).

## Scoring recipe

```python
auc_pythia = roc_auc_score(y_true_pythia, y_pred_pythia)
auc_herwig = roc_auc_score(y_true_herwig, y_pred_herwig)
resilience_pct = (auc_pythia - auc_herwig) / auc_pythia * 100
return {'AUC': auc_pythia, 'Resilience': resilience_pct}
```

## Common pitfalls

- Assuming higher AUC on a single simulator guarantees better physics analysis results, ignoring simulation dependence.
- Treating resilience as a fixed property rather than a trade-off; the Pareto frontier shows no model maximizes both AUC and resilience simultaneously.
- Expecting knowledge distillation to break the performance-resilience trade-off, as the study shows it fails to surpass the frontier.

## Evidence (verbatim from paper)

> Figure[1] shows the classifier AUC vs. resilience (AUC %-difference for Pythia vs. Herwig samples), for each of the models trained and for each set of hyperparameters considered: ‘optimal’ performance is at the lower-right corner of the figure. The Pareto frontier connecting models that optimize the AUC-resiliency tradeoff is highlighted, and models in the shaded region are Pareto-excluded.

## Citation

```bibtex
@misc{gambhir2025pareto,
  title={The Pareto Frontier of Resilient Jet Tagging},
  author={Gambhir et al. (2025)},
  year={2025},
  note={arXiv:2509.19431}
}
```

- arXiv: 2509.19431