Omni Safetybench Eval

---
name: omni-safetybench-eval
description: Evaluates the safety alignment and refusal capabilities of Audio-Visual Large Language Models (OLLMs) when exposed to harmful unimodal, dual-modal, and omni-modal inputs. It specifically probes whether models maintain consistent safety boundaries across modality combinations and reveals vulnerabilities in cross-modal comprehension-aware safety. Use when the user wants to benchmark on Omni-SafetyBench, or asks about evaluating this task. Reports Safety-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.07173
  bibtex_key: pan2025omisafetybench
  confidence: high
---

# omni-safetybench-eval

> Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models — Pan et al. (2025) (arXiv:2508.07173, 2025)

## What this evaluates

Evaluates the safety alignment and refusal capabilities of Audio-Visual Large Language Models (OLLMs) when exposed to harmful unimodal, dual-modal, and omni-modal inputs. It specifically probes whether models maintain consistent safety boundaries across modality combinations and reveals vulnerabilities in cross-modal comprehension-aware safety.

## Datasets

- **Omni-SafetyBench** — total 972; splits: test (972); repo https://github.com/THU-BPM/Omni-SafetyBench

## Metrics

- `C-ASR` — range: percent
  - Conditional Attack Success Rate: the percentage of harmful inputs where the model fails to refuse or generates unsafe content, determined by judge model evaluation.
- `C-RR` — range: percent
  - Conditional Refusal Rate: the percentage of harmful inputs where the model explicitly refuses to answer, determined by judge model evaluation.
- `Safety-score` **(primary)** — range: [0, 1]
  - A comprehensive metric combining understanding, safety, and refusal evaluations by the Qwen-Plus judge model to quantify overall safety performance on a 0-1 scale.
- `CMSC-score` — range: [0, 1]
  - Cross-Modal Safety Consistency score: quantifies how consistently a model maintains its safety performance across different modality paradigms (unimodal, dual-modal, omni-modal).

## Input / output format

**Input**: Audio, visual, and text prompts (unimodal, dual-modal, or omni-modal) containing harmful or joint-harm cases.

**Output**: Model-generated response (text/audio/video), which is subsequently evaluated by the Qwen-Plus judge model for understanding, safety, and refusal.

## Scoring recipe

```python
judge = Qwen_Plus()
results = []
for instance in dataset:
    response = model.generate(instance.input)
    judgment = judge.evaluate(response, instance.prompt)
    results.append(judgment)

c_asr = sum(1 for r in results if r.safety == unsafe) / len(results)
c_rr = sum(1 for r in results if r.refusal == true) / len(results)
safety_score = mean(r.safety_score for r in results)
cmsc_score = compute_consistency_across_modalities(results)
```

## Common pitfalls

- Assuming unimodal safety performance generalizes to dual-modal or omni-modal inputs, as the benchmark shows significant performance drops under complex joint inputs.
- Relying solely on automated refusal detection without comprehension-aware judging, which can miss cases where models understand the harm but fail to refuse appropriately.
- Ignoring cross-modal consistency; a model might score well on average but fail catastrophically on specific modality combinations (e.g., image-audio-text).

## Evidence (verbatim from paper)

> We employ the Conditional Attack Success Rate (C-ASR), Conditional Refusal Rate (C-RR), and a comprehensive metric, the Safety-score, to represent safety performance. To evaluate cross-modal safety consistency, we use the Cross-Modal Safety Consistency score (CMSC-score) as a quantification metric. To balance labeling accuracy and cost, we use Qwen-Plus... as the judge model to evaluate responses for understanding, safety, and refusal

## Citation

```bibtex
@misc{pan2025omisafetybench,
  title={Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models},
  author={Pan et al. (2025)},
  year={2025},
  note={arXiv:2508.07173}
}
```

- arXiv: 2508.07173