Mit Bih Ecg Adv Detection Eval

---
name: mit-bih-ecg-adv-detection-eval
description: Evaluates the robustness of ECG arrhythmia classifiers and adversarial detectors on real and synthetically generated adversarial ECG signals. It probes whether models maintain classification accuracy and can distinguish between genuine and adversarial cardiac signals under intra-patient and inter-patient data splits. Use when the user wants to benchmark on PhysioNet MIT-BIH Arrhythmia dataset, or asks about evaluating this task. Reports Accuracy (ACC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2107.07677
  bibtex_key: hossain2021ecgadv_gan
  confidence: high
---

# mit-bih-ecg-adv-detection-eval

> ECG-Adv-GAN: Detecting ECG Adversarial Examples with Conditional Generative Adversarial Networks — Hossain et al. (2021) (arXiv:2107.07677, 2021)

## What this evaluates

Evaluates the robustness of ECG arrhythmia classifiers and adversarial detectors on real and synthetically generated adversarial ECG signals. It probes whether models maintain classification accuracy and can distinguish between genuine and adversarial cardiac signals under intra-patient and inter-patient data splits.

## Datasets

- **PhysioNet MIT-BIH Arrhythmia dataset** — total ?; splits: train (-1), test (-1), intra-patient (-1), inter-patient (-1)

## Metrics

- `Accuracy (ACC)` **(primary)** — range: percent
  - Ratio of correctly classified instances to the total number of instances.
- `Sensitivity (SEN)` — range: percent
  - True positive rate: proportion of actual positives correctly identified.
- `Specificity (SPEC)` — range: percent
  - True negative rate: proportion of actual negatives correctly identified.
- `F1 Score` — range: [0, 1]
  - Harmonic mean of precision and recall.
- `AUC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring discriminative ability across thresholds.
- `MSE` — range: other
  - Mean Squared Error between original and adversarial signal amplitudes.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index measuring perceived change in structural information.

## Input / output format

**Input**: Normalized ECG signal segments (lead II) of 280 samples centered on the R-peak, with amplitude scaled to [0,1].

**Output**: Class label (N, S, V, or F) for arrhythmia classification; binary label (Real or Adversarial) for detection.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 0)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 0)
    acc = (tp + tn) / (tp + tn + fp + fn)
    sen = tp / (tp + fn) if (tp + fn) > 0 else 0
    spec = tn / (tn + fp) if (tn + fp) > 0 else 0
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    f1 = 2 * prec * sen / (prec + sen) if (prec + sen) > 0 else 0
    return acc, sen, spec, f1
```

## Common pitfalls

- Class imbalance in minority classes (S, V, F) requires SMOTE oversampling during training, which may not reflect real-world deployment.
- Baselines lack pre-trained weights, necessitating full retraining on the exact same splits for fair comparison.
- Adversarial examples are generated by the proposed model itself, potentially biasing detection evaluation toward the generator's specific perturbation style.

## Evidence (verbatim from paper)

> For metrics, we use Accuracy (ACC), Sensitivity(SEN), and Specificity(SPEC). We can see for the first experiment, except for the architecture given in [[3]], our model achieves the best score compared to other deep learning and machine learning derived architectures.

## Citation

```bibtex
@misc{hossain2021ecgadv_gan,
  title={ECG-Adv-GAN: Detecting ECG Adversarial Examples with Conditional Generative Adversarial Networks},
  author={Hossain et al. (2021)},
  year={2021},
  note={arXiv:2107.07677}
}
```

- arXiv: 2107.07677