Entropy Adaptive Merging Eval

---
name: entropy-adaptive-merging-eval
description: Evaluates the robustness of an entropy-adaptive model merging framework under heterogeneous domain shifts. It probes whether a merged model can adapt to unseen target domains using only forward passes and entropy-based coefficients, without backpropagation or labeled target data. Use when the user wants to benchmark on MiDog Atypical, Organs, Histopantum, ISIC Skin, Messidor, PACS, VLCS, Office-Home, TerraIncognita, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.21372
  bibtex_key: ambekar2026mean
  confidence: high
---

# entropy-adaptive-merging-eval

> The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging — Ambekar et al. (2026) (arXiv:2602.21372, 2026)

## What this evaluates

Evaluates the robustness of an entropy-adaptive model merging framework under heterogeneous domain shifts. It probes whether a merged model can adapt to unseen target domains using only forward passes and entropy-based coefficients, without backpropagation or labeled target data.

## Datasets

- **MiDog Atypical** — total 454; splits: test (-1)
- **Organs** — total 1645; splits: test (-1)
- **Histopantum** — total 281142; splits: test (-1)
- **ISIC Skin** — total 10015; splits: test (-1)
- **Messidor** — total 1200; splits: test (-1)
- **PACS** — total 9991; splits: test (-1)
- **VLCS** — total 10729; splits: test (-1)
- **Office-Home** — total 15500; splits: test (-1)
- **TerraIncognita** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly classified samples over the target stream or batch. Calculated as (number of correct predictions) / (total samples).

## Input / output format

**Input**: A batch of target-domain images (typically size 32) fed into K frozen, pre-trained domain-specific ViT models.

**Output**: Predicted class labels for each image in the batch.

## Scoring recipe

```python
correct = 0
total = 0
for batch in target_stream:
    preds = merged_model(batch)
    correct += (preds == batch.labels).sum()
    total += len(batch)
accuracy = (correct / total) * 100
```

## Common pitfalls

- Assuming backpropagation or target-time updates are used at test time; the protocol explicitly requires a forward-only pass with no backpropagation.
- Ignoring the leave-one-domain-out domain generalization setup; each model is trained on a single source domain and evaluated on a completely held-out target domain.
- Applying naive mean merging without decoupling encoder and classifier weights, which the paper shows causes severe performance degradation due to layer-wise misalignment.

## Evidence (verbatim from paper)

> We report accuracy over the target stream. All experiments are run on NVIDIA A100 GPUs; our method is non-parametric and incurs only lightweight per-batch overhead. Table[1] reports medical-domain results for both ViT-B/32 and ViT-B/16 under the standard leave-one-domain-out domain generalization protocol, using both offline and online merging with a test-time batch size of 32 samples.

## Citation

```bibtex
@misc{ambekar2026mean,
  title={The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging},
  author={Ambekar et al. (2026)},
  year={2026},
  note={arXiv:2602.21372}
}
```

- arXiv: 2602.21372