Mit Bih Ecg Classification Eval

---
name: mit-bih-ecg-classification-eval
description: Evaluates a model's ability to classify individual ECG beats into standard AAMI categories (Normal, Supraventricular Ectopic, Ventricular Ectopic, etc.) on a patient-specific basis. It probes robustness to severe class imbalance and morphological variations in real-time clinical monitoring scenarios. Use when the user wants to benchmark on MIT-BIH arrhythmia database, or asks about evaluating this task. Reports F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.02215
  bibtex_key: [REDACTED]
  confidence: high
---

# mit-bih-ecg-classification-eval

> Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks — Malik et al. (2021) (arXiv:2110.02215, 2021)

## What this evaluates

Evaluates a model's ability to classify individual ECG beats into standard AAMI categories (Normal, Supraventricular Ectopic, Ventricular Ectopic, etc.) on a patient-specific basis. It probes robustness to severe class imbalance and morphological variations in real-time clinical monitoring scenarios.

## Datasets

- **MIT-BIH arrhythmia database** — total 100389; splits: test (-1)

## Metrics

- `F1-score` **(primary)** — range: [0, 1] or percent
  - Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall). Computed per class and averaged for multi-class or binary detection tasks.
- `Accuracy` — range: [0, 1] or percent
  - Ratio of correctly classified beats to total beats: (TP + TN) / Total.
- `Sensitivity` — range: [0, 1] or percent
  - True positive rate: TP / (TP + FN).
- `Specificity` — range: [0, 1] or percent
  - True negative rate: TN / (TN + FP).
- `Positive Predictivity` — range: [0, 1] or percent
  - Precision: TP / (TP + FP).

## Input / output format

**Input**: Single or dual-channel raw ECG beat segments resized to 128 samples.

**Output**: Discrete class label per beat (N, S, V, F, Q) or binary SVEB/VEB detection.

## Scoring recipe

```python
def compute_metrics(preds, gold, target):
    tp = sum(1 for p, g in zip(preds, gold) if p == g == target)
    fp = sum(1 for p, g in zip(preds, gold) if p == target and g != target)
    fn = sum(1 for p, g in zip(preds, gold) if p != target and g == target)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    accuracy = sum(1 for p, g in zip(preds, gold) if p == g) / len(gold)
    return accuracy, f1, precision, recall
```

## Common pitfalls

- Patient-specific partitioning violates standard i.i.d. assumptions; models are trained on a small common set plus patient-specific data, making cross-patient generalization evaluation fundamentally different from standard benchmarks.
- Severe class imbalance exists (e.g., ~39k Normal vs ~190 SVEB beats), so Accuracy alone is misleading; F1 and Sensitivity must be prioritized.
- Some competing methods use non-AAMI compliant data partitioning (e.g., training on data from 200 patients), which inflates performance unfairly compared to the strict patient-specific protocol.

## Evidence (verbatim from paper)

> In this study, the following standard metrics are used: classification accuracy (Acc), sensitivity (Sen), specificity (Spe), positive predictivity (Ppr), and F1-score (F1). Since there is a large variation in the number of beats from different classes (class imbalance) in the training/testing data (i.e. 39465/50354 type-N, 1277/5716 type-V, and 190/2571 type-S beats), sensitivity, specificity, positive predictive value and especially, F1-score are all relevant performance criteria for medical diagnosis applications.

## Citation

```bibtex
@misc{malik2021realtime,
  title={Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks},
  author={Malik et al. (2021)},
  year={2021},
  note={arXiv:2110.02215}
}
```

- arXiv: 2110.02215