Chest Xray14 Eval

---
name: chest-xray14-eval
description: Evaluates a model's ability to perform multi-label classification of 14 thoracic diseases on chest X-ray images and localize pathological regions using attention maps. It probes whether anatomically grounded feature weighting improves disease detection over global feature fusion or saliency-based methods. Use when the user wants to benchmark on Chest X-ray14, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2105.12430
  bibtex_key: fang2021weighing
  confidence: high
---

# chest-xray14-eval

> Weighing Features of Lung and Heart Regions for Thoracic Disease Classification — Fang et al. (2021) (arXiv:2105.12430, 2021)

## What this evaluates

Evaluates a model's ability to perform multi-label classification of 14 thoracic diseases on chest X-ray images and localize pathological regions using attention maps. It probes whether anatomically grounded feature weighting improves disease detection over global feature fusion or saliency-based methods.

## Datasets

- **Chest X-ray14** — total ?; splits: benchmark split (-1), box set (-1)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, computed per disease class across varying classification thresholds to measure the trade-off between true positive and false positive rates.
- `IoU` — range: [0, 1]
  - Intersection over Union between the predicted pathological region (derived from Class Activation Maps) and the ground-truth bounding box.

## Input / output format

**Input**: Chest X-ray radiograph images.

**Output**: Per-image class probabilities for 14 thoracic diseases, and predicted bounding boxes for pathological regions via Class Activation Maps (CAM).

## Scoring recipe

```python
def compute_auroc(y_true, y_pred):
    fpr, tpr, _ = roc_curve(y_true, y_pred)
    return auc(fpr, tpr)

def compute_iou(pred_box, gt_box):
    intersection = area(pred_box & gt_box)
    union = area(pred_box | gt_box)
    return intersection / union if union > 0 else 0.0

# Per-disease AUROC
aurocs = [compute_auroc(gold[d], preds[d]) for d in diseases]
avg_auroc = mean(aurocs)

# Localization IoU
ious = [compute_iou(get_cam_box(img), img.gt_box) for img in box_set]
avg_iou = mean(ious)
```

## Common pitfalls

- Relying solely on global image features causes local discriminative information to be smoothed out.
- Methods using saliency maps or region proposals suffer from location deviation in pathological regions.
- Small pathological regions (e.g., nodules) are easily drowned out by global features without explicit feature weighting.

## Evidence (verbatim from paper)

> In Table 2, we report the classification performances of the proposed method and comparative methods in terms of AUROC scores, evaluated by the test set of the benchmark split.

## Citation

```bibtex
@misc{fang2021weighing,
  title={Weighing Features of Lung and Heart Regions for Thoracic Disease Classification},
  author={Fang et al. (2021)},
  year={2021},
  note={arXiv:2105.12430}
}
```

- arXiv: 2105.12430