Hierarchical Mlc Image Eval

---
name: hierarchical-mlc-image-eval
description: Evaluates a model's ability to perform hierarchical multi-label image classification on remote sensing scenes. It probes how well the model captures label dependencies and hierarchy structures while predicting multiple overlapping categories per image. Use when the user wants to benchmark on UCM, AID, DFC-15, MLRSNet, or asks about evaluating this task. Reports AUPRC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.11783
  bibtex_key: stoimchev2026helm
  confidence: high
---

# hierarchical-mlc-image-eval

> HELM: Hierarchical and Explicit Label Modeling with Graph Learning for Multi-Label Image Classification — Stoimchev et al. (2026) (arXiv:2603.11783, 2026)

## What this evaluates

Evaluates a model's ability to perform hierarchical multi-label image classification on remote sensing scenes. It probes how well the model captures label dependencies and hierarchy structures while predicting multiple overlapping categories per image.

## Datasets

- **UCM** — total ?; splits: test (-1)
- **AID** — total ?; splits: test (-1)
- **DFC-15** — total ?; splits: test (-1)
- **MLRSNet** — total ?; splits: test (-1)

## Metrics

- `AUPRC` **(primary)** — range: [0, 1]
  - Area under the precision-recall curve computed across all classes, typically averaged macro to treat all labels equally regardless of frequency.
- `Ranking Loss` — range: [0, 1]
  - Measures the average fraction of label pairs that are incorrectly ordered, penalizing cases where a relevant label is ranked lower than a non-relevant one.

## Input / output format

**Input**: Remote sensing scene images.

**Output**: Binary or probability scores for each hierarchical label in the predefined taxonomy.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_scores):
    # y_true: binary matrix (n_samples, n_labels)
    # y_pred_scores: float matrix (n_samples, n_labels)
    auprc = average_precision_score(y_true, y_pred_scores, average='macro')
    ranking_loss = ranking_loss_score(y_true, y_pred_scores)
    return {'AUPRC': auprc, 'Ranking Loss': ranking_loss}
```

## Common pitfalls

- Ranking Loss is often confused with standard ranking metrics like NDCG; it specifically measures pairwise label order violations in multi-label settings.
- AUPRC should be averaged macro across all hierarchical levels/classes, not micro, to avoid bias toward frequent labels.
- Hierarchical constraints (parent-child label dependencies) are not enforced during inference in the baseline, leading to inconsistent predictions.

## Evidence (verbatim from paper)

> The complete HELM model achieves the overall best or second-best results, with the highest AUPRC on UCM (0.904) and the lowest Ranking Loss across all datasets (0.022, 0.017, 0.006, 0.024).

## Citation

```bibtex
@misc{stoimchev2026helm,
  title={HELM: Hierarchical and Explicit Label Modeling with Graph Learning for Multi-Label Image Classification},
  author={Stoimchev et al. (2026)},
  year={2026},
  note={arXiv:2603.11783}
}
```

- arXiv: 2603.11783