Fed Ecg Eval

---
name: fed-ecg-eval
description: Evaluates federated learning algorithms on ECG classification tasks under non-IID and long-tailed label distribution challenges across multiple medical institutions. It probes how well FL methods generalize across heterogeneous clinical data and handle class imbalance without centralizing all data. Use when the user wants to benchmark on Fed-ECG, or asks about evaluating this task. Reports Micro F1-Score (Mi-F1).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.07050
  bibtex_key: zhang2024fedcvd
  confidence: high
---

# fed-ecg-eval

> FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data — Zhang et al. (2024) (arXiv:2411.07050, 2024)

## What this evaluates

Evaluates federated learning algorithms on ECG classification tasks under non-IID and long-tailed label distribution challenges across multiple medical institutions. It probes how well FL methods generalize across heterogeneous clinical data and handle class imbalance without centralizing all data.

## Datasets

- **Fed-ECG** — total ?; splits: train (-1), test (-1); repo https://github.com/SMILELab-FL/FedCVD

## Metrics

- `Micro F1-Score (Mi-F1)` **(primary)** — range: percent
  - Harmonic mean of global precision and recall calculated across all classes and samples, expressed as a percentage.
- `Mean Average Precision (mAP)` — range: percent
  - Mean of the average precision scores computed per class, expressed as a percentage.

## Input / output format

**Input**: ECG signal data from a specific medical institution (client).

**Output**: Predicted class labels for ECG classification.

## Scoring recipe

```python
def compute_mi_f1(preds, gold):
    tp = sum(1 for p, g in zip(preds, gold) if p == g)
    fp = sum(1 for p, g in zip(preds, gold) if p != g)
    fn = sum(1 for p, g in zip(preds, gold) if p != g)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    return 2 * prec * rec / (prec + rec) * 100
```

## Common pitfalls

- Evaluating only on the global test set misses the practical LOCAL performance per client, which is crucial for real-world deployment.
- Long-tail performance is often overlooked; Top-K drop and F1-STD are needed to capture class imbalance effects.
- Simulated non-IID partitions are easier than the dataset's natural partitioning, leading to overoptimistic FL benchmarks.

## Evidence (verbatim from paper)

> Table 2: The performance of different FL methods on Fed-ECG is reported using two metrics: Micro F1-Score (Mi-F1) and Mean Average Precision (mAP), both expressed as percentages (%).

## Citation

```bibtex
@misc{zhang2024fedcvd,
  title={FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data},
  author={Zhang et al. (2024)},
  year={2024},
  note={arXiv:2411.07050}
}
```

- arXiv: 2411.07050