Mit Bih Arrhythmia Eval

---
name: mit-bih-arrhythmia-eval
description: Evaluates a model's ability to classify raw 2-lead ECG signals into five arrhythmia types (normal, supraventricular, ventricular, fusion, unknown) using a multi-class classification setup. It probes temporal feature extraction and attention-based weighting of cardiac cycles without manual preprocessing. Use when the user wants to benchmark on MIT-BIH Arrhythmia Dataset, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2108.10226
  bibtex_key: liu2021ecgarrhythmia
  confidence: high
---

# mit-bih-arrhythmia-eval

> ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks — Liu et al. (2021) (arXiv:2108.10226, 2021)

## What this evaluates

Evaluates a model's ability to classify raw 2-lead ECG signals into five arrhythmia types (normal, supraventricular, ventricular, fusion, unknown) using a multi-class classification setup. It probes temporal feature extraction and attention-based weighting of cardiac cycles without manual preprocessing.

## Datasets

- **MIT-BIH Arrhythmia Dataset** — total 96000; splits: train (76800), test (19200); repo https://github.com/ziyuliu-lion/heart-arrhythmia-diagnosis-with-deep-learning

## Metrics

- `AUC` **(primary)** — range: [0.5, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
- `F-1 score` — range: [0, 1]
  - Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). Reported per class and averaged to handle severe class imbalance.

## Input / output format

**Input**: 2-channel ECG segment of 240 sampling points per channel (approx. 0.76 seconds), centered on the R-peak, with z-score normalization applied.

**Output**: Predicted class label or probability distribution over 5 arrhythmia categories: N (normal), S (supraventricular), V (ventricular), F (fusion), Q (unknown).

## Scoring recipe

```python
import numpy as np
from sklearn.metrics import roc_auc_score, f1_score

# AUC (One-vs-Rest for multi-class)
auc = roc_auc_score(y_true, y_pred_proba, multi_class='ovr', average='macro')

# F-1 score (Weighted average to handle class imbalance)
f1 = f1_score(y_true, y_pred, average='weighted')
```

## Common pitfalls

- Class imbalance is severe (e.g., F=154 vs N=16884), making accuracy and macro-averaged precision/recall misleading; AUC and weighted F-1 are explicitly preferred.
- The 80/20 split is random across all 96,000 samples, not stratified by patient; this risks data leakage if subject-level cross-validation is expected.
- Input windows are strictly fixed to 240 samples centered on the R-peak; models trained on variable-length or non-aligned segments will not match the reported protocol.

## Evidence (verbatim from paper)

> The AUC score falls in the range of $[0.5,1]$, the higher the better. Although the commonly used evaluation metrics (such as accuracy, precision, recall, and F-1 score) are not suitable as they all sensitive the sample amount, we use them to show the model’s performance on each single class.

## Citation

```bibtex
@misc{liu2021ecgarrhythmia,
  title={ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks},
  author={Liu et al. (2021)},
  year={2021},
  note={arXiv:2108.10226}
}
```

- arXiv: 2108.10226