Chestxray14 Plco Eval

---
name: chestxray14-plco-eval
description: Evaluates a model's ability to detect and localize multiple pathologies in high-resolution chest X-ray images. It specifically probes the model's robustness to severe class imbalance and its capacity to leverage explicit spatial location information for pathology classification. Use when the user wants to benchmark on ChestX-Ray14, PLCO, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1803.04565
  bibtex_key: gundel2018dnetloc
  confidence: medium
---

# chestxray14-plco-eval

> Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks — Gündel et al. (2018) (arXiv:1803.04565, 2018)

## What this evaluates

Evaluates a model's ability to detect and localize multiple pathologies in high-resolution chest X-ray images. It specifically probes the model's robustness to severe class imbalance and its capacity to leverage explicit spatial location information for pathology classification.

## Datasets

- **ChestX-Ray14** — total ?; splits: train (-1), val (-1), test (-1)
- **PLCO** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, computed independently for each pathology class and typically averaged across all classes to report overall performance.

## Input / output format

**Input**: 1024x1024 RGB chest X-ray images, normalized to ImageNet mean and standard deviation.

**Output**: C-dimensional vector of probabilities in [0,1] via sigmoid, where C=14 for ChestX-Ray14 or C=35 for the combined dataset, representing independent binary pathology predictions.

## Scoring recipe

```python
def compute_auc(predictions, labels):
    # predictions: (N, C) probabilities from sigmoid
    # labels: (N, C) binary ground truth
    auc_scores = []
    for c in range(C):
        auc_scores.append(roc_auc_score(labels[:, c], predictions[:, c]))
    return np.mean(auc_scores)
```

## Common pitfalls

- Using image-wise splits instead of patient-wise splits causes data leakage and artificially inflates performance.
- Failing to account for severe class imbalance without appropriate loss weighting or thresholding leads to biased predictions.
- Treating spatial location labels as mutually exclusive when multiple diseases can co-occur in different lung regions.

## Evidence (verbatim from paper)

> achieving the highest AUC on ChestX-Ray14 under fair evaluation conditions

## Citation

```bibtex
@misc{gundel2018dnetloc,
  title={Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks},
  author={Gündel et al. (2018)},
  year={2018},
  note={arXiv:1803.04565}
}
```

- arXiv: 1803.04565