Evaluates training-free multimodal agents on retrieval-augmented generation, general reasoning, and hallucination robustness by testing a polarized latent graph memory that injects logical constraints at inference time. Use when the user wants to benchmark on MRAMG-Bench, MRAG-Bench, Visual-RAG, MMMU, MMStar, HallusionBench, or asks about evaluating this task. Reports performance.
Installs into .claude/skills of the current project.
Are you the author of Polarmem Multimodal Eval?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/qhjqhj00-polarmem-multimodal-eval)
---
name: polarmem-multimodal-eval
description: Evaluates training-free multimodal agents on retrieval-augmented generation, general reasoning, and hallucination robustness by testing a polarized latent graph memory that injects logical constraints at inference time. Use when the user wants to benchmark on MRAMG-Bench, MRAG-Bench, Visual-RAG, MMMU, MMStar, HallusionBench, or asks about evaluating this task. Reports performance.
metadata:
skill_kind: dataset_eval
source_arxiv: 2602.00415
bibtex_key: [REDACTED]
confidence: high
---
# polarmem-multimodal-eval
> PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Multimodal Agents — Chen et al. (2026) (arXiv:2602.00415, 2026)
## What this evaluates
Evaluates training-free multimodal agents on retrieval-augmented generation, general reasoning, and hallucination robustness by testing a polarized latent graph memory that injects logical constraints at inference time.
## Datasets
- **MRAMG-Bench** — total ?; splits: test (-1)
- **MRAG-Bench** — total ?; splits: test (-1)
- **Visual-RAG** — total ?; splits: test (-1)
- **MMMU** — total ?; splits: test (-1)
- **MMStar** — total ?; splits: test (-1)
- **HallusionBench** — total ?; splits: test (-1)
## Metrics
- `performance` **(primary)** — range: percent
- Standard benchmark performance score calculated as the percentage of correctly answered questions or appropriately graded responses per each benchmark's official evaluation protocol.
## Input / output format
**Input**: Multimodal query (image + text prompt) from the benchmark, augmented with retrieved evidence snippets formatted as '[Fact Check: {Status}] {Content}', truncated to a fixed token limit per item to fit the context window.
**Output**: Text response/answer generated by the VLM backbone to the query.
## Scoring recipe
```python
def compute_performance(predictions, gold_labels):
correct = sum(1 for p, g in zip(predictions, gold_labels) if evaluate_answer(p, g) == 'correct')
return (correct / len(gold_labels)) * 100
```
## Common pitfalls
- Residual retrieval noise in long contexts can cause diminishing or negative returns on stronger models.
- Retrieved items are truncated to a fixed maximum length (in tokens) to fit within the model context window, which may discard relevant evidence.
- All memory-based methods must use the same memory sources and the same retrieval budget (Top-K evidence) to ensure fair comparison.
## Evidence (verbatim from paper)
> Table 1: Performance comparison on six benchmarks. PolarMem (PolarMem) significantly boosts retrieval-augmented benchmarks (MRAMG/MRAG/Visual-RAG) for smaller and mid-size backbones, while showing diminishing or occasionally negative returns on stronger models for general reasoning (MMMU/MMStar) and hallucination robustness (HallusionBench), likely due to residual retrieval noise in long contexts.
## Citation
```bibtex
@misc{chen2026polarmem,
title={PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Multimodal Agents},
author={Chen et al. (2026)},
year={2026},
note={arXiv:2602.00415}
}
```
- arXiv: 2602.00415
