Hierarchical Visual Recognition Eval

---
name: hierarchical-visual-recognition-eval
description: Evaluates a model's ability to perform fine-grained, taxonomy-aware visual recognition by predicting hierarchical biological labels (order, family, genus, species) from images. It specifically probes whether the model maintains logical consistency across taxonomic levels while accurately identifying leaf-level species, including generalization to unseen/novel categories. Use when the user wants to benchmark on iNaturalist-2021, TerraIncognita, or asks about evaluating this task. Reports Hierarchical Consistent Accuracy (HCA).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.00431
  bibtex_key: he2026tara
  confidence: high
---

# hierarchical-visual-recognition-eval

> Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models — He et al. (2026) (CVPR 2026, 2026)

## What this evaluates

Evaluates a model's ability to perform fine-grained, taxonomy-aware visual recognition by predicting hierarchical biological labels (order, family, genus, species) from images. It specifically probes whether the model maintains logical consistency across taxonomic levels while accurately identifying leaf-level species, including generalization to unseen/novel categories.

## Datasets

- **iNaturalist-2021** — total ?; splits: train (-1), val (-1)
- **TerraIncognita** — total ?; splits: known (-1), novel (-1)

## Metrics

- `Hierarchical Consistent Accuracy (HCA)` **(primary)** — range: [0, 1]
  - HCA = (1/N) * Σ_{i=1}^{N} Π_{j=1}^{L^i} 1[f_θ(x^i; Y_j) = y^i_j]. Computes the proportion of test samples whose predicted taxonomic path exactly matches the ground truth from root to leaf.
- `Leaf-Level Accuracy (Acc_leaf)` — range: [0, 1]
  - Acc_leaf = (1/N) * Σ_{i=1}^{N} 1[f_θ(x^i; Y_L) = y^i_L]. Measures discriminative ability at the most fine-grained (leaf) level.
- `Point-Overlap Ratio (POR)` — range: [0, 1]
  - POR = (1/N) * Σ_{i=1}^{N} (Σ_{j=1}^{L_i} 1[f_θ(x_i; Y_j) = y^i_j]) / L_i. Averages the proportion of correctly predicted nodes across the hierarchy, allowing partial correctness.
- `Strict Point-Overlap Ratio (S-POR)` — range: [0, 1]
  - S-POR = (1/N) * Σ_{i=1}^{N} (1/L_i) * max_{1≤a≤b≤L_i} [(b-a+1) * Π_{j=a}^{b} 1[f_θ(x_i; Y_j) = y^i_j]]. Rewards only the longest contiguous segment of correct predictions normalized by depth.
- `Top Overlap Ratio (TOR)` — range: [0, 1]
  - TOR = (1/N) * Σ_{i=1}^{N} (1/(L_i-1)) * Σ_{j=1}^{L_i-1} 1[f_θ(x_i; Y_j) = y^i_j] * 1[f_θ(x_i; Y_{j+1}) = y^i_{j+1}]. Evaluates pairwise consistency between consecutive taxonomic layers.
- `F1 Score` — range: [0, 1]
  - Harmonic mean of precision and recall computed at the Order and Family levels. Used for TerraIncognita where full taxonomic labels are missing for novel species.

## Input / output format

**Input**: Resized image (328×328) paired with a text prompt/question specifying a taxonomic level (order, family, genus, or species). Evaluated in a 1-shot VQA setting.

**Output**: Predicted taxonomic label(s) for the specified level(s). For TerraIncognita, the model returns a label or 'Unknown' if uncertain.

## Scoring recipe

```python
def compute_hca(predictions, golds, hierarchy_depths):
    correct = 0
    for pred, gold, depth in zip(predictions, golds, hierarchy_depths):
        if len(pred) < depth:
            pred = pred + [None] * (depth - len(pred))
        if all(p == g for p, g in zip(pred, gold)):
            correct += 1
    return correct / len(predictions)
```

## Common pitfalls

- HCA requires an exact match across the entire taxonomic path from root to leaf, making it significantly stricter than leaf-level accuracy.
- TerraIncognita novel species lack complete taxonomic annotations, so evaluation is restricted to Order and Family levels only.
- Input images are strictly resized to 328×328 with zero data augmentation, which differs from standard training pipelines and may impact generalization benchmarks.

## Evidence (verbatim from paper)

> Hierarchical Consistent Accuracy (HCA). This metric is defined as HCA = (1/N) sum_{i=1}^{N} prod_{j=1}^{L^{i}} 1[f_{	heta}(x^{i}; Y_{j}) = y^{i}_{j}]. Here, N is the number of test samples, L^{i} is the depth of the hierarchy for the i-th input x^{i}, and Y_{j} denotes the label set at level j. HCA computes the proportion of samples whose predicted paths exactly match the ground truth from root to leaf. It is therefore a stricter criterion than flat accuracy and serves as our primary evaluation metric for hierarchical classification.

## Citation

```bibtex
@misc{he2026tara,
  title={Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models},
  author={He et al. (2026)},
  year={2026},
  note={CVPR 2026}
}
```

- arXiv: 2603.00431