Medi Aug Eval

---
name: medi-aug-eval
description: Evaluates the impact of six mix-based data augmentation methods on medical image classification performance. It probes how well models generalize to medical domains (brain MRI and eye fundus) when trained with different augmentation strategies and backbones. Use when the user wants to benchmark on Brain Tumor Classification Dataset, Eye Diseases Classification Dataset, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.18983
  bibtex_key: qi2025mediaug
  confidence: high
---

# medi-aug-eval

> MediAug: Exploring Visual Augmentation in Medical Imaging — Qi et al. (2025) (arXiv:2504.18983, 2025)

## What this evaluates

Evaluates the impact of six mix-based data augmentation methods on medical image classification performance. It probes how well models generalize to medical domains (brain MRI and eye fundus) when trained with different augmentation strategies and backbones.

## Datasets

- **Brain Tumor Classification Dataset** — total ?; splits: train (-1), test (-1)
- **Eye Diseases Classification Dataset** — total ?; splits: train (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Proportion of correctly classified instances out of the total number of instances.
- `Precision` — range: percent
  - Ratio of true positive predictions to the total number of positive predictions.
- `Recall` — range: percent
  - Ratio of true positive predictions to the total number of actual positives.
- `F1-Score` — range: percent
  - Harmonic mean of Precision and Recall.
- `ROC AUC` — range: percent
  - Area under the Receiver Operating Characteristic curve, reflecting the model's ability to distinguish between classes.

## Input / output format

**Input**: Medical images (MRI scans for brain tumors, fundus images for eye diseases) with corresponding multi-class labels.

**Output**: Predicted class labels for each image.

## Scoring recipe

```python
def evaluate(predictions, gold_labels):
    accuracy = sum(p == g for p, g in zip(predictions, gold_labels)) / len(gold_labels)
    precision = compute_precision(gold_labels, predictions)
    recall = compute_recall(gold_labels, predictions)
    f1 = 2 * (precision * recall) / (precision + recall)
    auc = compute_roc_auc(gold_labels, predictions)
    return accuracy, precision, recall, f1, auc
```

## Common pitfalls

- The brain tumor dataset has an imbalanced class distribution, which can skew accuracy if not handled properly.
- Certain classes exhibit significant feature overlap (e.g., glioma vs. meningioma, glaucoma vs. normal), making classification inherently difficult and potentially inflating error rates for specific classes.

## Evidence (verbatim from paper)

> Accuracy measures overall correctness. Precision assesses the accuracy of positive predictions, while Recall measures the ability to identify actual positives. Sensitivity (similar to Recall) focuses on detecting positives, and Specificity evaluates the identification of negatives. F1-Score balances Precision and Recall, and ROC AUC reflects the model’s ability to distinguish between classes, with higher values indicating better performance.

## Citation

```bibtex
@misc{qi2025mediaug,
  title={MediAug: Exploring Visual Augmentation in Medical Imaging},
  author={Qi et al. (2025)},
  year={2025},
  note={arXiv:2504.18983}
}
```

- arXiv: 2504.18983