Rodent Bench Eval

---
name: rodent-bench-eval
description: Evaluates multimodal large language models on temporal segmentation and fine-grained behavioral annotation of rodent videos. It probes capabilities in long-video processing, distinguishing subtle or rare behaviors, and handling diverse experimental paradigms and camera angles. Use when the user wants to benchmark on Rodent-Bench-Long, Rodent-Bench-Short, or asks about evaluating this task. Reports Weighted Matthew’s Correlation Coefficient (MCC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.18540
  bibtex_key: [REDACTED]
  confidence: high
---

# rodent-bench-eval

> Rodent-Bench — Heap et al. (arXiv:2602.18540, 2026)

## What this evaluates

Evaluates multimodal large language models on temporal segmentation and fine-grained behavioral annotation of rodent videos. It probes capabilities in long-video processing, distinguishing subtle or rare behaviors, and handling diverse experimental paradigms and camera angles.

## Datasets

- **Rodent-Bench-Long** — total ?; splits: test (-1)
- **Rodent-Bench-Short** — total ?; splits: test (-1)

## Metrics

- `Weighted Matthew’s Correlation Coefficient (MCC)` **(primary)** — range: [-1, 1]
  - A correlation coefficient between predicted and observed classifications, weighted by class frequency. Calculated as (TP*TN - FP*FN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN)), where TP, TN, FP, FN are true/false positives/negatives. Ranges from -1 to 1.

## Input / output format

**Input**: Video clips (short or long duration) accompanied by task-specific prompts instructing the model to perform temporal segmentation and behavioral annotation.

**Output**: JSON-formatted output containing segment keys (e.g., 'end_time') and corresponding timestamps or labels for each behavioral segment in the video.

## Scoring recipe

```python
def compute_weighted_mcc(predictions, ground_truth):
    # 1. Align predicted and ground truth temporal segments
    # 2. Map behavioral labels to discrete classes
    # 3. Compute confusion matrix weighted by class prevalence
    # 4. Apply MCC formula: (TP*TN - FP*FN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN))
    # 5. Return scalar MCC value
    return mcc_value
```

## Common pitfalls

- Models often produce malformed JSON (e.g., truncated outputs or incorrect keys like 'end_long_time' instead of 'end_time'), breaking automated parsing.
- Performance drops significantly on videos with non-standard camera angles, visual filters requiring color recognition, or behaviors shorter than one second where the distinction between 'freezing' and stillness is subtle.

## Evidence (verbatim from paper)

> Figure 3: Weighted Matthew’s Correlation Coefficient (MCC) performance across models. (a) Rodent-Bench-Long: Gemini-2.5-Pro achieves the highest performance with lower variance compared to Gemini-2.5-Flash.

## Citation

```bibtex
@misc{heap2026rodentbench,
  title={Rodent-Bench},
  author={Heap et al.},
  year={2026},
  note={arXiv:2602.18540}
}
```

- arXiv: 2602.18540