Chestxray14 Multi Label Eval

---
name: chestxray14-multi-label-eval
description: Evaluates deep learning models for multi-label chest X-ray pathology classification. It probes the capability of CNN architectures to detect 14 specific thoracic diseases, testing the impact of transfer learning, network depth, and fusion of non-image patient demographics (age, gender, view position) on diagnostic accuracy. Use when the user wants to benchmark on ChestX-ray14, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1803.02315
  bibtex_key: baltruschat2018comparison
  confidence: high
---

# chestxray14-multi-label-eval

> Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification — Baltruschat et al. (2018) (arXiv:1803.02315, 2018)

## What this evaluates

Evaluates deep learning models for multi-label chest X-ray pathology classification. It probes the capability of CNN architectures to detect 14 specific thoracic diseases, testing the impact of transfer learning, network depth, and fusion of non-image patient demographics (age, gender, view position) on diagnostic accuracy.

## Datasets

- **ChestX-ray14** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, computed independently for each of the 14 pathologies and then averaged across all classes.
- `Spearman's rank correlation coefficient` — range: [-1, 1]
  - Measures the monotonic relationship between the prediction scores of two different models, averaged over all model pairs and 5 cross-validation folds.

## Input / output format

**Input**: Chest X-ray images resized to 256×256 or 480×480 pixels (center-cropped), optionally concatenated with non-image features (patient age, gender, view position).

**Output**: Per-pathology probability scores (or logits) for 14 binary classification tasks.

## Scoring recipe

```python
def compute_auc(predictions, labels):
    # predictions: (N, 14) probabilities
    # labels: (N, 14) binary ground truth
    aucs = []
    for i in range(14):
        auc = roc_auc_score(labels[:, i], predictions[:, i])
        aucs.append(auc)
    return np.mean(aucs)
```

## Common pitfalls

- Failing to split data at the patient level rather than image level, which causes data leakage since individual patients have multiple follow-up acquisitions.
- Comparing results across studies using only a single random data split, as AUC values show high variability across different splits and can lead to misleading conclusions about state-of-the-art performance.

## Evidence (verbatim from paper)

> We perform an ROC analysis using the area under the curve (AUC) for all pathologies, compare the classifier scores by Spearman's pairwise rank correlation coefficient, and employ the state-of-the-art method Gradient-weighted Class Activation Mapping (Grad-CAM) to gain more insight into our CNNs.

## Citation

```bibtex
@misc{baltruschat2018comparison,
  title={Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification},
  author={Baltruschat et al. (2018)},
  year={2018},
  note={arXiv:1803.02315}
}
```

- arXiv: 1803.02315