Ecg Delineation Eval

---
name: ecg-delineation-eval
description: This benchmark evaluates a model's ability to accurately segment and delineate the onset and offset boundaries of P, QRS, and T waves in electrocardiogram (ECG) signals. It specifically probes robustness across diverse cardiac arrhythmias and tests the effectiveness of classification-guided post-processing in reducing false positive detections during atrial fibrillation and flutter. Use when the user wants to benchmark on Internal dataset, LUDB, QTDB, or asks about evaluating this task. Reports F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2304.06237
  bibtex_key: joung2023ecgsegmentation
  confidence: high
---

# ecg-delineation-eval

> Deep learning based ECG segmentation for delineation of diverse arrhythmias — Joung et al. (2023) (arXiv:2304.06237, 2023)

## What this evaluates

This benchmark evaluates a model's ability to accurately segment and delineate the onset and offset boundaries of P, QRS, and T waves in electrocardiogram (ECG) signals. It specifically probes robustness across diverse cardiac arrhythmias and tests the effectiveness of classification-guided post-processing in reducing false positive detections during atrial fibrillation and flutter.

## Datasets

- **Internal dataset** — total ?; splits: test (-1)
- **LUDB** — total ?; splits: test (-1)
- **QTDB** — total ?; splits: test (-1)

## Metrics

- `F1-score` **(primary)** — range: [0, 1]
  - Harmonic mean of sensitivity and positive predictive value: F1 = 2 * (Se * PPV) / (Se + PPV). Reported as a percentage in the paper.
- `Sensitivity (Se)` — range: [0, 1]
  - True positive rate: Se = TP / (TP + FN).
- `Positive Predictive Value (PPV)` — range: [0, 1]
  - Precision: PPV = TP / (TP + FP).
- `Mean error (m)` — range: ms
  - Average time deviation (in milliseconds) of correctly detected points from their ground truth annotations.
- `Standard deviation of error (σ)` — range: ms
  - Standard deviation of the time deviations for correctly detected points.

## Input / output format

**Input**: Single-lead ECG signal (typically lead II or the lead with the lowest delineation error), resampled to 500 Hz.

**Output**: Predicted onset and offset timestamps (in milliseconds) for P, QRS, and T waves per cardiac beat.

## Scoring recipe

```python
def compute_ecg_metrics(preds, gold, tolerance=150):
    TP, FP, FN = 0, 0, 0
    errors = []
    for p in preds:
        matched = [g for g in gold if abs(p - g) <= tolerance]
        if matched:
            TP += 1
            errors.append(abs(p - min(matched)))
        else:
            FP += 1
    for g in gold:
        if not any(abs(p - g) <= tolerance for p in preds):
            FN += 1
    Se = TP / (TP + FN) if (TP + FN) > 0 else 0
    PPV = TP / (TP + FP) if (TP + FP) > 0 else 0
    F1 = 2 * (Se * PPV) / (Se + PPV) if (Se + PPV) > 0 else 0
    m = sum(errors) / len(errors) if errors else 0
    sigma = (sum((e - m)**2 for e in errors) / len(errors))**0.5 if errors else 0
    return Se, PPV, F1, m, sigma
```

## Common pitfalls

- Evaluations must be repeated 20 times and averaged to match the reported results; single-run scores will not align with the paper.
- QTDB annotation format does not allow exact PPV calculation for some waves because absent manual annotations are treated as non-included rather than true negatives.
- Deep learning-based methods consistently exhibit higher standard deviation of error (σ) compared to wavelet-based methods, particularly for T-wave offsets.
- P-wave detection fails frequently during atrial fibrillation and flutter without classification-guided post-processing, leading to high false positive rates.

## Evidence (verbatim from paper)

> To ensure soundness, we follow the usual standard chosen by The Association for the Advancement of Medical Instrumentation(AAMI) [[42]], which considers an onset or an offset to be correctly detected if an algorithm locates the same type of annotation in a neighborhood of 150ms. Using this threshold value, we examine for each predicted point whether the prediction correctly detects a point in the ground truth annotation. If a ground truth annotation is correctly detected, we count a true positive(TP). ... Based on this, we calculate the following evaluation metrics: ... sensitivity Se=TP/(TP+FN) ... positive predictive value PPV=TP/(TP+FP) ... F1-score F1=2·(Se·PPV)/(Se+PPV)

## Citation

```bibtex
@misc{joung2023ecgsegmentation,
  title={Deep learning based ECG segmentation for delineation of diverse arrhythmias},
  author={Joung et al. (2023)},
  year={2023},
  note={arXiv:2304.06237}
}
```

- arXiv: 2304.06237