Ptbxl Ecg Classification Eval

---
name: ptbxl-ecg-classification-eval
description: Evaluates the ability of self-supervised pre-trained Vision Transformers to classify ECG signals across multiple diagnostic label hierarchies (e.g., all statements, ST-MEM labels, diagnostic subclasses, rhythm statements). Use when the user wants to benchmark on PTB-XL, or asks about evaluating this task. Reports macro AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2410.13867
  bibtex_key: weimann2024selfsupervised
  confidence: high
---

# ptbxl-ecg-classification-eval

> Self-Supervised Pre-Training with Joint-Embedding Predictive Architecture Boosts ECG Classification Performance — Weimann et al. (2024) (arXiv:2410.13867, 2024)

## What this evaluates

Evaluates the ability of self-supervised pre-trained Vision Transformers to classify ECG signals across multiple diagnostic label hierarchies (e.g., all statements, ST-MEM labels, diagnostic subclasses, rhythm statements).

## Datasets

- **PTB-XL** — total ?; splits: train (-1), val (-1), test (-1); HF `physionet/ptb-xl`

## Metrics

- `macro AUC` **(primary)** — range: [0, 1]
  - Computes the Area Under the Receiver Operating Characteristic (AUC) curve for each diagnostic label independently, then averages these AUC values across all K labels.

## Input / output format

**Input**: ECG time-series waveforms (typically 12-lead, processed as image-like patches/tokens for Vision Transformer input).

**Output**: Binary classification probabilities or scores for each diagnostic label in the target hierarchy.

## Scoring recipe

```python
def compute_macro_auc(y_true, y_pred, n_labels):
    auc_scores = []
    for i in range(n_labels):
        auc_scores.append(roc_auc_score(y_true[:, i], y_pred[:, i]))
    return np.mean(auc_scores)
```

## Common pitfalls

- Linear evaluation (frozen backbone + linear head) frequently outperforms end-to-end fine-tuning for JEPA-pretrained ViTs on PTB-XL.
- The ST-MEM baseline comparison uses a custom data split, making performance comparisons approximate rather than strictly controlled.
- Best downstream performance does not always correlate with lowest pre-training loss; checkpoint selection must be based on validation AUC rather than pre-training loss.

## Evidence (verbatim from paper)

> Performance is evaluated using the macro area under the receiver operating curve (macro AUC), the primary metric that is universally reported in related studies. We report the average macro AUC on the test set (fold 10) with the standard deviation in parentheses (0.0xx).

## Citation

```bibtex
@misc{weimann2024selfsupervised,
  title={Self-Supervised Pre-Training with Joint-Embedding Predictive Architecture Boosts ECG Classification Performance},
  author={Weimann et al. (2024)},
  year={2024},
  note={arXiv:2410.13867}
}
```

- arXiv: 2410.13867