Hir Grambarcodes Eval

---
name: hir-grambarcodes-eval
description: Evaluates histopathology image retrieval and classification performance using high-order texture features (Gram barcodes) extracted from CNN layers. It probes the model's ability to capture tissue texture patterns for accurate image matching and class prediction. Use when the user wants to benchmark on KimiaPath24, CRC, EMC, or asks about evaluating this task. Reports η_total, Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2111.15519
  bibtex_key: lifshitz2021grambarcodes
  confidence: high
---

# hir-grambarcodes-eval

> Gram Barcodes for Histopathology Tissue Texture Retrieval — Lifshitz et al. (2021) (arXiv:2111.15519, 2021)

## What this evaluates

Evaluates histopathology image retrieval and classification performance using high-order texture features (Gram barcodes) extracted from CNN layers. It probes the model's ability to capture tissue texture patterns for accurate image matching and class prediction.

## Datasets

- **KimiaPath24** — total ?; splits: train (-1), test (-1)
- **CRC** — total ?; splits: 5-fold cross-validation (-1)
- **EMC** — total ?; splits: 5-fold cross-validation (-1)

## Metrics

- `η_total` **(primary)** — range: percent
  - Overall retrieval accuracy score combining precision and recall components as defined in the paper.
- `η_p` — range: percent
  - Precision component of the retrieval score.
- `η_w` — range: percent
  - Recall/Weighted component of the retrieval score.
- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly classified query images based on top-3 retrieval voting.

## Input / output format

**Input**: Histopathology image passed through a pre-trained VGG19 network to extract Gram barcodes from specified convolutional layers.

**Output**: Predicted tissue class, determined by the mode of the classes of the top-3 most similar retrieved images (fallback to top-1 if no mode).

## Scoring recipe

```python
def predict_class(query_feat, db_feats, db_labels):
    sims = cosine_sim(query_feat, db_feats)
    top3_idx = argsort(sims, k=3, desc=True)
    top3_labels = [db_labels[i] for i in top3_idx]
    counts = Counter(top3_labels)
    if len(counts) == 1 or counts[top3_labels[0]] > 1:
        return top3_labels[0]
    else:
        return top3_labels[0]
def compute_accuracy(preds, gold):
    return sum(p == g for p, g in zip(preds, gold)) / len(gold)
```

## Common pitfalls

- Uses retrieval-based voting (top-3 mode) instead of direct classification, which changes the evaluation dynamics compared to standard supervised benchmarks.
- Metrics differ across datasets: KimiaPath24 reports η_total/η_p/η_w, while CRC and EMC report standard accuracy with 5-fold CV, complicating cross-dataset comparison.

## Evidence (verbatim from paper)

> We achieved a top η_total score of 87.79% (η_p= 93.21%, η_w= 94.19%) on the provided training and testing sets using VGG19 layers ‘conv2_1’, ‘conv3_1’, and ‘conv5_1’ for Gram barcode generation. In our experiments, the predicted class of any query image is the mode of the classes of the top-3 most similar images retrieved by our algorithm. If there is no mode, the class of the most similar image (the top-1 image) is predicted to be the class of the query image.

## Citation

```bibtex
@misc{lifshitz2021grambarcodes,
  title={Gram Barcodes for Histopathology Tissue Texture Retrieval},
  author={Lifshitz et al. (2021)},
  year={2021},
  note={arXiv:2111.15519}
}
```

- arXiv: 2111.15519