Valueground Eval

---
name: valueground-eval
description: Evaluates whether multimodal large language models (MLLMs) can maintain consistent culture-conditioned value judgments when response options are replaced with minimally contrastive visual proxies. It probes cross-modal prediction stability and the ability to ground textual value tendencies in subtle visual contrasts. Use when the user wants to benchmark on ValueGround, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.06484
  bibtex_key: wang2026valueground
  confidence: high
---

# valueground-eval

> ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs — Wang et al. (2026) (arXiv:2604.06484, 2026)

## What this evaluates

Evaluates whether multimodal large language models (MLLMs) can maintain consistent culture-conditioned value judgments when response options are replaced with minimally contrastive visual proxies. It probes cross-modal prediction stability and the ability to ground textual value tendencies in subtle visual contrasts.

## Datasets

- **ValueGround** — total ?; splits: test (-1); repo https://github.com/NL2G/ValueGround

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correct predictions where the model's output matches the ground-truth option (A or B) for each country-question-image pair.

## Input / output format

**Input**: Main task: a country $c$, a World Values Survey question $q$, and a pair of minimally contrastive images $I_q=(I_A, I_B)$ representing the two verbal options.

**Output**: A single choice indicating either option A or option B, corresponding to the image that better matches the value tendency associated with country $c$ for question $q$.

## Scoring recipe

```python
correct = 0
total = 0
for country, question, image_pair, gold_option in dataset:
    pred = model(country, question, image_pair)
    if pred == gold_option:
        correct += 1
    total += 1
accuracy = (correct / total) * 100
```

## Common pitfalls

- Models may rely on country-conditioned textual priors rather than actually grounding the judgment in the visual contrast, leading to inflated text-only performance that doesn't transfer to the main task.
- High alignment accuracy (92.8%) does not guarantee main task success, as the bottleneck is multimodal integration under country conditioning, not visual contrast recognition.

## Evidence (verbatim from paper)

> Averaged across models, accuracy drops from 72.8% in text-only and 92.8% in alignment to 65.8% on the main task. This pattern indicates that success cannot be explained by country-conditioned textual prediction or visual contrast recognition alone.

## Citation

```bibtex
@misc{wang2026valueground,
  title={ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2604.06484}
}
```

- arXiv: 2604.06484