Histgen Wsi Report Eval

---
name: histgen-wsi-report-eval
description: Evaluates a model's ability to generate clinical histopathology reports from gigapixel whole slide images (WSIs). It probes cross-modal alignment between dense visual patches and concise textual descriptions using standard natural language generation metrics. Use when the user wants to benchmark on TCGA WSI-Report, or asks about evaluating this task. Reports BLEU-4.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2403.05396
  bibtex_key: guo2024histgen
  confidence: high
---

# histgen-wsi-report-eval

> HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction — Guo et al. (2024) (arXiv:2403.05396, 2024)

## What this evaluates

Evaluates a model's ability to generate clinical histopathology reports from gigapixel whole slide images (WSIs). It probes cross-modal alignment between dense visual patches and concise textual descriptions using standard natural language generation metrics.

## Datasets

- **TCGA WSI-Report** — total 7753; splits: train (-1), val (-1), test (-1); repo https://github.com/dddavid4real/HistGen

## Metrics

- `BLEU-4` **(primary)** — range: [0, 1]
  - Computes 4-gram precision between predicted and reference reports, penalizing overly short outputs via a brevity penalty. Values range from 0 to 1.
- `METEOR` — range: [0, 1]
  - Measures unigram precision and recall with stemming and synonym matching, applying a penalty for fragmentation. Values range from 0 to 1.
- `ROUGE-L` — range: [0, 1]
  - Calculates the F1-score of the longest common subsequence between the prediction and reference, capturing sentence-level fluency. Values range from 0 to 1.

## Input / output format

**Input**: Sequence of visual patch features extracted from a whole slide image (WSI) via a pre-trained encoder (e.g., DINOv2 ViT-L).

**Output**: Natural language text string representing the histopathology report.

## Scoring recipe

```python
import nltk
from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction
from rouge_score import rouge_scorer

def compute_metrics(pred: str, gold: str) -> dict:
    pred_tok = nltk.word_tokenize(pred)
    gold_tok = nltk.word_tokenize(gold)
    bleu4 = sentence_bleu([gold_tok], pred_tok, smoothing_function=SmoothingFunction().method4)
    scorer = rouge_scorer.RougeScorer(['rougeL'], use_stemmer=True)
    rouge_l = scorer.score(gold, pred)['rougeL'].fmeasure
    return {'bleu4': bleu4, 'rougeL': rouge_l}
```

## Common pitfalls

- WSIs are gigapixel-sized; naive full-image processing is infeasible. Models must use multiple instance learning (MIL) or patch pooling to handle the sequence length.
- Standard NLG metrics (BLEU, ROUGE) measure lexical overlap and may not correlate well with clinical diagnostic accuracy or report quality.
- External dataset results use Monte Carlo cross-validation, so scores should be reported as mean ± standard deviation over multiple runs, not single splits.

## Evidence (verbatim from paper)

> For report generation, our model is evaluated on the WSI-report dataset (specified in Sec. 2.1), using NLG metrics including BLEU [24], METEOR [11], and ROUGE-L [20].

## Citation

```bibtex
@misc{guo2024histgen,
  title={HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction},
  author={Guo et al. (2024)},
  year={2024},
  note={arXiv:2403.05396}
}
```

- arXiv: 2403.05396