Mina Ecg Af Eval

---
name: mina-ecg-af-eval
description: Evaluates deep learning models for binary classification of atrial fibrillation (AF) versus control using single-lead ECG recordings. It probes the model's ability to capture beat-level morphology, rhythm-level dynamics, and frequency-domain patterns for clinical AF detection. Use when the user wants to benchmark on PhysioNet Challenge 2017, or asks about evaluating this task. Reports PR-AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1905.11333
  bibtex_key: hong2019mina
  confidence: high
---

# mina-ecg-af-eval

> MINA: Multilevel Knowledge-Guided Attention for Modeling Electrocardiography Signals — Hong et al. (2019) (arXiv:1905.11333, 2019)

## What this evaluates

Evaluates deep learning models for binary classification of atrial fibrillation (AF) versus control using single-lead ECG recordings. It probes the model's ability to capture beat-level morphology, rhythm-level dynamics, and frequency-domain patterns for clinical AF detection.

## Datasets

- **PhysioNet Challenge 2017** — total 8528; splits: train (-1), val (-1), test (-1)

## Metrics

- `ROC-AUC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
- `PR-AUC` **(primary)** — range: [0, 1]
  - Area under the Precision-Recall Curve, measuring the trade-off between precision and recall across thresholds. Explicitly recommended by authors for this imbalanced dataset.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall, calculated as 2 * (precision * recall) / (precision + recall) at a 0.5 decision threshold.

## Input / output format

**Input**: 1D ECG voltage time series of exactly 3000 points (9 seconds), extracted via non-overlapping sliding window segmentation from raw recordings sampled at 300Hz.

**Output**: Binary classification label: AF patient (1) or control (0).

## Scoring recipe

```python
def evaluate(y_true, y_pred_prob):
    y_pred = (y_pred_prob >= 0.5).astype(int)
    roc_auc = roc_auc_score(y_true, y_pred_prob)
    pr_auc = average_precision_score(y_true, y_pred_prob)
    f1 = f1_score(y_true, y_pred)
    return {'ROC-AUC': roc_auc, 'PR-AUC': pr_auc, 'F1': f1}
```

## Common pitfalls

- The dataset is highly imbalanced (738 AF vs 7790 controls); ROC-AUC can be misleading, so PR-AUC is the recommended primary metric.
- Models must be evaluated over 5 independent runs with different random seeds, reporting mean ± standard deviation, rather than a single run.
- Raw recordings vary in length (9s to 60s); inputs must be strictly normalized to 3000 points to match the model's expected shape.

## Evidence (verbatim from paper)

> Performance was measured by the Area under the Receiver Operating Characteristic (ROC-AUC), Area under the Precision-Recall Curve (PR-AUC) and the F1 score. The PR-AUC is considered a better measure for imbalanced data like ours. Table 3 shows MINA outperforms all baselines, and shows 5.51% higher PR-AUC than the second best models.

## Citation

```bibtex
@misc{hong2019mina,
  title={MINA: Multilevel Knowledge-Guided Attention for Modeling Electrocardiography Signals},
  author={Hong et al. (2019)},
  year={2019},
  note={arXiv:1905.11333}
}
```

- arXiv: 1905.11333