Phycritic Eval

---
name: phycritic-eval
description: Evaluates multimodal models' ability to perform physical reasoning, spatial cognition, and egocentric task planning, as well as their capacity to act as reliable critics/judges for physical AI tasks. Use when the user wants to benchmark on PhyCritic-Bench, VL-RewardBench, Multimodal-RewardBench, CosmosReason1-Bench, CV-Bench, EgoPlanBench2, or asks about evaluating this task. Reports accuracy (overall/macro).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.11124
  bibtex_key: xiong2026phycritic
  confidence: high
---

# phycritic-eval

> PhyCritic: Multimodal Critic Models for Physical AI — Xiong et al. (2026) (arXiv:2602.11124, 2026)

## What this evaluates

Evaluates multimodal models' ability to perform physical reasoning, spatial cognition, and egocentric task planning, as well as their capacity to act as reliable critics/judges for physical AI tasks.

## Datasets

- **PhyCritic-Bench** — total ?; splits: test (-1)
- **VL-RewardBench** — total ?; splits: test (-1)
- **Multimodal-RewardBench** — total ?; splits: test (-1)
- **CosmosReason1-Bench** — total ?; splits: test (-1)
- **CV-Bench** — total ?; splits: test (-1)
- **EgoPlanBench2** — total ?; splits: test (-1)

## Metrics

- `accuracy (overall/macro)` **(primary)** — range: percent
  - Percentage of correctly answered questions or correctly judged critiques across the benchmark. Reported as overall accuracy and macro-averaged across sub-tasks.
- `2D/3D spatial score` — range: percent
  - Accuracy on 2D and 3D spatial cognition sub-tasks within CV-Bench.

## Input / output format

**Input**: Multimodal prompts consisting of images or video frames paired with text questions or instructions for physical reasoning, spatial cognition, egocentric planning, or critique/judgment tasks.

**Output**: Textual predictions, answers, or structured critique/judgment outputs. For critic benchmarks, the model generates its own physics-aware prediction before evaluating candidate responses.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for pred, gold in zip(predictions, gold_labels) if normalize(pred) == normalize(gold))
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Requires ground-truth answers for the self-prediction reward during training, limiting applicability to fully open-ended scenarios.
- Benchmarks focus on curated physical QA and specific sub-tasks, which may not generalize to unstructured real-world embodied settings.
- Macro-averaged scores can mask performance drops on specific sub-domains like bridge-v2 or failure-robustness.

## Evidence (verbatim from paper)

> On PhyCritic-Bench (Tab. [2]), our model attains the best overall accuracy among open-source 7B/8B models (68.0)... On CosmosReason1-Bench, it achieves the best open-source accuracy (63.9)... We evaluate our approach on both critic performance and its capacity for handling Physical-AI tasks. For critic performance, we use our curated PhyCritic-Bench for physical-related critique, together with two general reward benchmarks—VL-RewardBench and Multimodal-RewardBench... For physical capacity, we evaluate our methods on three physical-related multimodal benchmarks. CosmosReason1-Bench probes physical commonsense... CV-Bench assesses whether visual representations support downstream control... EgoPlanBench2 targets egocentric daily-task planning...

## Citation

```bibtex
@misc{xiong2026phycritic,
  title={PhyCritic: Multimodal Critic Models for Physical AI},
  author={Xiong et al. (2026)},
  year={2026},
  note={arXiv:2602.11124}
}
```

- arXiv: 2602.11124