Physionet Cinc Ecg Eval

---
name: physionet-cinc-ecg-eval
description: Evaluates a model's ability to perform multilabel classification of cardiac abnormalities from 12-lead ECG recordings. It specifically probes robustness to severe class imbalance and the capacity to handle diverse lead configurations and sampling rates in long-sequence cardiac signals. Use when the user wants to benchmark on PhysioNet/CinC Challenge 2021, or asks about evaluating this task. Reports Macro AUPRC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.13046
  bibtex_key: jiang2025onedimensional
  confidence: high
---

# physionet-cinc-ecg-eval

> One Dimensional CNN ECG Mamba for Multilabel Abnormality Classification in 12 Lead ECG — Jiang et al. (2025) (arXiv:2510.13046, 2025)

## What this evaluates

Evaluates a model's ability to perform multilabel classification of cardiac abnormalities from 12-lead ECG recordings. It specifically probes robustness to severe class imbalance and the capacity to handle diverse lead configurations and sampling rates in long-sequence cardiac signals.

## Datasets

- **PhysioNet/CinC Challenge 2021** — total 43101; splits: train (-1), test (-1)

## Metrics

- `Macro AUPRC` **(primary)** — range: [0, 1]
  - Macro-averaged Area Under the Precision-Recall Curve. Computed by calculating the AUPRC for each of the 26 classes independently and then averaging them, giving equal weight to minority and majority classes.
- `Macro AUROC` — range: [0, 1]
  - Macro-averaged Area Under the Receiver Operating Characteristic Curve. Measures the classifier’s ability to discriminate between positive and negative instances for each class, then averages the scores across all 26 categories.

## Input / output format

**Input**: Raw 12-lead ECG time-series recordings (sampled at 500 Hz, 1000 Hz, or 257 Hz), with support for diverse lead configurations (6, 4, 3, or 2-lead formats).

**Output**: Multilabel probability scores or binary predictions across 26 classes (25 cardiac abnormalities + sinus rhythm).

## Scoring recipe

```python
# preds: (N, 26) float probabilities
# gold: (N, 26) binary labels
auprcs = [auprc(gold[:, c], preds[:, c]) for c in range(26)]
aurocs = [auroc(gold[:, c], preds[:, c]) for c in range(26)]
macro_auprc = sum(auprcs) / 26
macro_auroc = sum(aurocs) / 26
```

## Common pitfalls

- Treating the task as single-label classification instead of multilabel, which misrepresents clinical reality where patients often have multiple comorbidities.
- Using micro-averaging or accuracy instead of macro-averaged AUPRC/AUROC, which would bias results toward majority classes and ignore the severe class imbalance.
- Failing to average the 5-fold cross-validation results, leading to high variance and unreliable performance estimates.
- Confusing the 2020 and 2021 dataset splits, as the 2021 dataset fully contains the 2020 dataset but has different class counts (26 vs 27) and sizes.

## Evidence (verbatim from paper)

> To assess the performance of the ECG classification model under the challenges of class imbalance and multi-label prediction, two evaluation metrics were employed: Macro Area Under the Precision–Recall Curve (AUPRC): This metric evaluates the predictive capacity of the model in all categories, with a particular emphasis on minority classes. By averaging precision–recall performance over all labels, Macro AUPRC is well-suited for imbalanced datasets, as it effectively reflects the trade-off between precision and recall under underrepresented conditions. Macro Area Under the Receiver Operating Characteristic Curve (AUROC): AUROC measures the classifier’s ability to discriminate between positive and negative instances while treating each class with equal importance. The macro-averaged formulation ensures that performance is not biased toward the majority classes, thus providing a balanced assessment of the discriminative power of the model.

## Citation

```bibtex
@misc{jiang2025onedimensional,
  title={One Dimensional CNN ECG Mamba for Multilabel Abnormality Classification in 12 Lead ECG},
  author={Jiang et al. (2025)},
  year={2025},
  note={arXiv:2510.13046}
}
```

- arXiv: 2510.13046