Vc Ifeval Eval

---
name: vc-ifeval-eval
description: Evaluates multimodal large language models' ability to follow instructions containing vision-dependent constraints, such as spatial, stylistic, and structural requirements. It isolates the contribution of visual input to instruction adherence and assesses generalization on standard visual reasoning tasks. Use when the user wants to benchmark on VC-IFEval, MM-IFEval, IFEval, or asks about evaluating this task. Reports instruction-following accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.03198
  bibtex_key: he2026empowering
  confidence: low
---

# vc-ifeval-eval

> Empowering Reliable Visual-Centric Instruction Following in MLLMs — He et al. (2026) (arXiv:2601.03198, 2026)

## What this evaluates

Evaluates multimodal large language models' ability to follow instructions containing vision-dependent constraints, such as spatial, stylistic, and structural requirements. It isolates the contribution of visual input to instruction adherence and assesses generalization on standard visual reasoning tasks.

## Datasets

- **VC-IFEval** — total ?; splits: test (-1)
- **MM-IFEval** — total ?; splits: test (-1)
- **IFEval** — total ?; splits: test (-1)

## Metrics

- `instruction-following accuracy` **(primary)** — range: [0, 1]
  - Standard benchmark-specific accuracy metrics for instruction following and visual reasoning tasks as implemented in VLMEvalKit.

## Input / output format

**Input**: Multimodal inputs (image + text instruction) for VC-IFEval, MM-IFEval, and VQA benchmarks; text-only prompts for IFEval.

**Output**: Natural language text responses generated by the MLLM, evaluated against ground-truth constraints or answers.

## Scoring recipe

```python
For each instance:
  pred = model.generate(image, prompt)
  score = benchmark_metric.evaluate(pred, gold)
aggregate_scores = mean(score across all instances)
return aggregate_scores
```

## Common pitfalls

- Multimodal instruction following benchmarks often conflate language-only capabilities with true visual grounding.
- Isolating visual input contributions requires a hybrid evaluation protocol to prevent language priors from dominating.

## Evidence (verbatim from paper)

> We evaluate our models on a series of instruction-following and visual reasoning benchmarks, including MM-IFEval (Ding et al., 2025), VC-IFEval, and IFEval (Zhou et al., 2023), where IFEval is language-only while the others are multimodal.

## Citation

```bibtex
@misc{he2026empowering,
  title={Empowering Reliable Visual-Centric Instruction Following in MLLMs},
  author={He et al. (2026)},
  year={2026},
  note={arXiv:2601.03198}
}
```

- arXiv: 2601.03198