Lvlm Hallucination Eval

---
name: lvlm-hallucination-eval
description: Evaluates Large Vision-Language Models on their ability to generate factually consistent outputs aligned with visual input, specifically measuring the reduction of object hallucinations in open-ended generation while preserving general multimodal reasoning and visual grounding capabilities. Use when the user wants to benchmark on POPE, CHAIR, HallusionBench, AMBER, VizWiz, MME, LLaVA-Wild, MM-Vet, or asks about evaluating this task. Reports CHAIR (object hallucination score).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.15556
  bibtex_key: lyu2026pade
  confidence: high
---

# lvlm-hallucination-eval

> Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs — Lyu et al. (2026) (arXiv:2602.15556, 2026)

## What this evaluates

Evaluates Large Vision-Language Models on their ability to generate factually consistent outputs aligned with visual input, specifically measuring the reduction of object hallucinations in open-ended generation while preserving general multimodal reasoning and visual grounding capabilities.

## Datasets

- **POPE** — total ?; splits: test (-1)
- **CHAIR** — total ?; splits: test (-1)
- **HallusionBench** — total ?; splits: test (-1)
- **AMBER** — total ?; splits: test (-1)
- **VizWiz** — total ?; splits: test (-1)
- **MME** — total ?; splits: test (-1)
- **LLaVA-Wild** — total ?; splits: test (-1)
- **MM-Vet** — total ?; splits: test (-1)

## Metrics

- `CHAIR (object hallucination score)` **(primary)** — range: percent
  - Measures the proportion of hallucinated objects in open-ended captions relative to ground truth. Standard recall/precision-based scoring is used per benchmark convention.
- `POPE (binary hallucination classification)` — range: [0, 1]
  - Accuracy of binary classification distinguishing hallucinated vs. non-hallucinated responses.
- `HallusionBench (visual consistency)` — range: [0, 1]
  - Accuracy on fine-grained visual consistency questions.
- `AMBER (grounded reasoning)` — range: [0, 1]
  - Accuracy on visually grounded reasoning and generation tasks.
- `General Multimodal Accuracy` — range: percent
  - Standard accuracy or score aggregation across VizWiz, MME, LLaVA-Wild, and MM-Vet for visual understanding and reasoning.

## Input / output format

**Input**: Image(s) paired with text instructions/prompts for visual understanding, reasoning, or open-ended captioning tasks.

**Output**: Text responses generated via sampling-based decoding (default settings).

## Scoring recipe

```python
def compute_metrics(predictions, gold):
    scores = {}
    for bench in ['POPE', 'CHAIR', 'HallusionBench', 'AMBER', 'VizWiz', 'MME', 'LLaVA-Wild', 'MM-Vet']:
        if bench == 'CHAIR':
            scores[bench] = calculate_hallucination_recall(predictions[bench], gold[bench])
        elif bench == 'POPE':
            scores[bench] = binary_accuracy(predictions[bench], gold[bench])
        else:
            scores[bench] = standard_accuracy(predictions[bench], gold[bench])
    return scores
```

## Common pitfalls

- Applying intervention at intermediate model layers instead of the final layer, which the ablation shows yields suboptimal performance due to diffuse attention.
- Using excessively large intervention strength (lambda), which introduces strong perturbations that deviate from the model's learned attention dynamics and degrade performance.
- Relying on perturbed visual inputs or external auxiliary models for evaluation, which the paper contrasts with its training-free, internal-dynamics approach.

## Evidence (verbatim from paper)

> The hallucination-focused benchmarks include POPE*Li et al. ([2023])* (binary hallucination classification), CHAIR*Rohrbach et al. ([2018])* (object hallucination in open-ended captioning), HallusionBench*Guan et al. ([2024])* (fine-grained visual consistency), and AMBER*Wang et al. ([2023])* (visually grounded reasoning and generation).

## Citation

```bibtex
@misc{lyu2026pade,
  title={Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs},
  author={Lyu et al. (2026)},
  year={2026},
  note={arXiv:2602.15556}
}
```

- arXiv: 2602.15556