Gamma Glaucoma Eval

---
name: gamma-glaucoma-eval
description: Evaluates multi-modal medical image analysis models for glaucoma staging by jointly processing 2D fundus images and 3D OCT volumes. It probes the model's ability to fuse cross-modality features and correctly classify patients into normal, early, or progressive glaucoma stages. Use when the user wants to benchmark on GAMMA Challenge, or asks about evaluating this task. Reports kappa.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2202.06511
  bibtex_key: wu2022gamma
  confidence: high
---

# gamma-glaucoma-eval

> GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges — Wu et al. (2022) (arXiv:2202.06511, 2022)

## What this evaluates

Evaluates multi-modal medical image analysis models for glaucoma staging by jointly processing 2D fundus images and 3D OCT volumes. It probes the model's ability to fuse cross-modality features and correctly classify patients into normal, early, or progressive glaucoma stages.

## Datasets

- **GAMMA Challenge** — total ?; splits: preliminary (-1), final test (-1)

## Metrics

- `kappa` **(primary)** — range: percent
  - Cohen's or Weighted Kappa coefficient, expressed as a percentage. It measures agreement between model predictions and ground truth labels while correcting for chance. For this ordinal classification task, it is typically computed as weighted kappa to account for the clinical severity ordering of normal, early, and progressive glaucoma.

## Input / output format

**Input**: Paired 2D fundus images and 3D OCT volumes for each patient.

**Output**: Predicted glaucoma stage: normal, early-glaucoma, or progressive-glaucoma.

## Scoring recipe

```python
def compute_kappa(preds, gold):
    labels = sorted(set(preds) | set(gold))
    conf = confusion_matrix(gold, preds, labels=labels)
    observed = np.trace(conf) / conf.sum()
    expected = np.dot(conf.sum(axis=0), conf.sum(axis=1)) / conf.sum()**2
    kappa = (observed - expected) / (1 - expected)
    return kappa * 100
```

## Common pitfalls

- Preliminary set scores often overestimate final test performance due to overfitting; teams should be evaluated strictly on the held-out final test set.
- Overall accuracy can mask clinically critical errors; predicting progressive glaucoma as early is more dangerous than the reverse, so progressive-class accuracy should be prioritized in clinical deployment.
- The final competition ranking uses a weighted sum across three tasks (0.4*glaucoma + 0.3*fundus + 0.3*OCT), but the primary grading metric remains kappa.

## Evidence (verbatim from paper)

> The quantitative scores of the glaucoma grading task measured by kappa are shown in Table 3. We reported their performances in the preliminary stage (evaluation on preliminary set) and the final stage (evaluation on final test set).

## Citation

```bibtex
@misc{wu2022gamma,
  title={GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges},
  author={Wu et al. (2022)},
  year={2022},
  note={arXiv:2202.06511}
}
```

- arXiv: 2202.06511