Synomaly Medical Anomaly Eval

---
name: synomaly-medical-anomaly-eval
description: Evaluates unsupervised anomaly detection in medical imaging by training a generative model on healthy images and reconstructing anomalous inputs. It measures how well the model localizes and segments pathological regions by comparing the reconstruction residuals against ground-truth anomaly masks. Use when the user wants to benchmark on BraTS 2023 (Brain MRI), LiTS (Liver CT), Carotid US, or asks about evaluating this task. Reports Dice.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.04004
  bibtex_key: bi2024synomaly
  confidence: high
---

# synomaly-medical-anomaly-eval

> Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images — Bi et al. (2024) (arXiv:2411.04004, 2024)

## What this evaluates

Evaluates unsupervised anomaly detection in medical imaging by training a generative model on healthy images and reconstructing anomalous inputs. It measures how well the model localizes and segments pathological regions by comparing the reconstruction residuals against ground-truth anomaly masks.

## Datasets

- **BraTS 2023 (Brain MRI)** — total 5159; splits: train (3743), test_anomalous (1000), test_healthy (416); repo https://github.com/yuan-12138/Synomaly
- **LiTS (Liver CT)** — total 7467; splits: train (5820), test_anomalous (1000), test_healthy (647); repo https://github.com/yuan-12138/Synomaly
- **Carotid US** — total 8684; splits: train (7306), test_anomalous (545), test_healthy (833); repo https://github.com/yuan-12138/Synomaly

## Metrics

- `Dice` **(primary)** — range: [0, 1]
  - Dice = 2 * |A ∩ B| / (|A| + |B|), where A and B are the predicted and ground-truth anomaly masks. Measures overlap between predicted and actual anomaly regions.
- `Precision` — range: [0, 1]
  - Precision = TP / (TP + FP). Proportion of predicted anomaly pixels that are correctly identified as anomalies.
- `Recall` — range: [0, 1]
  - Recall = TP / (TP + FN). Proportion of actual anomaly pixels that are correctly detected by the model.

## Input / output format

**Input**: Anomalous medical image (MRI, CT, or ultrasound) provided as input to the generative model. Ground-truth anomaly segmentation masks are provided only for evaluation.

**Output**: A reconstructed 'healthy' image and a binary anomaly segmentation mask generated by thresholding the pixel-wise difference between the blurred input and blurred reconstructed image.

## Scoring recipe

```python
def compute_metrics(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = pred_mask.sum() + gt_mask.sum()
    dice = 2 * intersection / union if union > 0 else 0.0
    precision = intersection / pred_mask.sum() if pred_mask.sum() > 0 else 0.0
    recall = intersection / gt_mask.sum() if gt_mask.sum() > 0 else 0.0
    return dice, precision, recall
```

## Common pitfalls

- Inference hyperparameters (noise steps T, Gaussian kernel n, threshold Th) are optimized per method via grid search, meaning reported results reflect best-case tuning rather than fixed out-of-the-box settings.
- The anomaly mask is derived from the difference between Gaussian-blurred input and reconstructed images, which smooths residuals and may obscure fine-grained anomaly boundaries.
- The Carotid US dataset is in-house and not publicly released, limiting external reproducibility and cross-dataset generalization checks.

## Evidence (verbatim from paper)

> The evaluation metrics include the Dice, Precision, and Recall score, to ensure an objective and comprehensive comparison of different approaches.

## Citation

```bibtex
@misc{bi2024synomaly,
  title={Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images},
  author={Bi et al. (2024)},
  year={2024},
  note={arXiv:2411.04004}
}
```

- arXiv: 2411.04004