Ptbxl Multi Label Ecg Eval

---
name: ptbxl-multi-label-ecg-eval
description: Evaluates deep learning architectures for multi-label classification of 12-lead ECG recordings into 23 diagnostic categories. Probes the trade-off between local morphological feature extraction and sequential temporal modeling under severe class imbalance. Use when the user wants to benchmark on PTB-XL, or asks about evaluating this task. Reports Macro AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.18830
  bibtex_key: jafari2026temporal
  confidence: high
---

# ptbxl-multi-label-ecg-eval

> How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification — Jafari et al. (2026) (arXiv:2601.18830, 2026)

## What this evaluates

Evaluates deep learning architectures for multi-label classification of 12-lead ECG recordings into 23 diagnostic categories. Probes the trade-off between local morphological feature extraction and sequential temporal modeling under severe class imbalance.

## Datasets

- **PTB-XL** — total ?; splits: test (-1); repo https://github.com/alireza720/Multi-Label-Classification

## Metrics

- `Macro AUROC` **(primary)** — range: [0, 1]
  - Area under the receiver operating characteristic curve, averaged across all 23 diagnostic classes. Computed by calculating the AUROC for each class independently and taking the unweighted mean.
- `Hamming Loss` — range: [0, 1]
  - Fraction of label assignments that are incorrect. Calculated as the number of misclassified labels divided by the total number of labels (N × C).
- `Subset Accuracy` — range: [0, 1]
  - Fraction of samples for which all predicted labels exactly match all true labels (exact match ratio).

## Input / output format

**Input**: Single 12-lead ECG time-series recording.

**Output**: Binary multi-label vector of length 23, where each element indicates the presence (1) or absence (0) of a specific diagnostic class.

## Scoring recipe

```python
def evaluate(y_true, y_pred):
    # y_true, y_pred: (N, 23) binary matrices
    hamming_loss = np.mean(y_true != y_pred)
    subset_acc = np.mean(np.all(y_true == y_pred, axis=1))
    macro_auroc = np.mean([roc_auc_score(y_true[:, c], y_pred[:, c]) for c in range(23)])
    macro_f1 = np.mean([f1_score(y_true[:, c], y_pred[:, c], average='macro') for c in range(23)])
    return {'hamming_loss': hamming_loss, 'subset_accuracy': subset_acc, 'macro_auroc': macro_auroc, 'macro_f1': macro_f1}
```

## Common pitfalls

- Threshold-dependent metrics (F1, AUPRC) are highly unstable for rare classes due to extreme prevalence skew, often showing near-perfect precision but low recall.
- Deeper or hybrid recurrent stacks (e.g., LSTM+BiLSTM) yield diminishing returns and may overfit, making simpler single-layer BiLSTM or CNN architectures more robust.
- High AUROC for minority classes can mask poor threshold-based predictive performance, misleadingly suggesting clinical utility for rare diagnoses.

## Evidence (verbatim from paper)

> Among the evaluated models, the BiLSTM model attains the lowest Hamming Loss (0.0338) and the highest Subset Accuracy (0.5723), reflecting fewer label-wise errors and improved joint label assignment in the multi-label setting. It also demonstrates competitive macro- and micro-averaged performance, including Macro AUROC (0.9202), Macro AUPRC (0.4715), and Micro F1 (0.6979), supporting its effectiveness in capturing temporal dependencies relevant to ECG interpretation.

## Citation

```bibtex
@misc{jafari2026temporal,
  title={How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification},
  author={Jafari et al. (2026)},
  year={2026},
  note={arXiv:2601.18830}
}
```

- arXiv: 2601.18830