Label Noise Resilience Histo Eval

---
name: label-noise-resilience-histo-eval
description: This benchmark evaluates the robustness of histopathology image classification models to both uniform and asymmetric label noise. It compares the performance of contrastive deep embeddings against non-contrastive backbones and image-based noise-robust loss functions. The protocol measures how well classifiers maintain accuracy when training labels are corrupted. Use when the user wants to benchmark on NCT-CRC-HE-100K, PatchCamelyon, BACH, MHIST, LC25000, GasHisSDB, or asks about evaluating this task. Reports test accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2404.07605
  bibtex_key: [REDACTED]
  confidence: high
---

# label-noise-resilience-histo-eval

> Contrastive-Based Deep Embeddings for Label Noise-Resilient Histopathology Image Classification — Dedieu et al. (2024) (arXiv:2404.07605, 2024)

## What this evaluates

This benchmark evaluates the robustness of histopathology image classification models to both uniform and asymmetric label noise. It compares the performance of contrastive deep embeddings against non-contrastive backbones and image-based noise-robust loss functions. The protocol measures how well classifiers maintain accuracy when training labels are corrupted.

## Datasets

- **NCT-CRC-HE-100K** — total ?; splits: train (80000), val (20000), test (7180)
- **PatchCamelyon** — total ?; splits: train (262144), val (32768), test (32768)
- **BACH** — total ?; splits: train (320), val (80), test (83)
- **MHIST** — total ?; splits: train (1740), val (435), test (977)
- **LC25000** — total ?; splits: train (16000), val (4000), test (5000)
- **GasHisSDB** — total ?; splits: train (21303), val (5325), test (6656)

## Metrics

- `test accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted labels out of the total number of test instances. Reported as mean ± standard deviation across multiple independent training runs.

## Input / output format

**Input**: Histopathology image patches of varying resolutions (e.g., 224×224, 96×96, 2048×1536, 768×768, 160×160) with class labels subject to uniform or asymmetric noise injection during training.

**Output**: Predicted class label for each image patch.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
# Report mean ± std over 2 runs (image methods) or 4 runs (deep embedding methods)
```

## Common pitfalls

- Noise is applied during the training phase to the labels used for model fitting, not merely at test time.
- Standard deviation is computed over multiple independent training runs (2 for image-based baselines, 4 for deep embedding methods), not across different data splits.
- Asymmetric noise uses dataset-specific class transition matrices rather than a simple uniform flip rate.

## Evidence (verbatim from paper)

> Table 2: Average test accuracies with standard deviation under different uniform label noise ratios (%, 2 runs for image methods, 4 runs for deep embedding methods).

## Citation

```bibtex
@misc{dedieu2024contrastive,
  title={Contrastive-Based Deep Embeddings for Label Noise-Resilient Histopathology Image Classification},
  author={Dedieu et al. (2024)},
  year={2024},
  note={arXiv:2404.07605}
}
```

- arXiv: 2404.07605