Safavid Eval

---
name: safavid-eval
description: Evaluates the safety alignment and refusal capabilities of Video Large Multimodal Models (VLMMs) against everyday adversarial queries and covert, human-red-teamed prompts. It measures whether models can maintain safety guidelines across diverse harmful categories without compromising general utility or falling back to memorized refusals. Use when the user wants to benchmark on SafeVidBench, or asks about evaluating this task. Reports Safety Rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.11926
  bibtex_key: wang2025safavid
  confidence: high
---

# safavid-eval

> SafeVid: Toward Safety Aligned Video Large Multimodal Models — Wang et al. (2025) (arXiv:2505.11926, 2025)

## What this evaluates

Evaluates the safety alignment and refusal capabilities of Video Large Multimodal Models (VLMMs) against everyday adversarial queries and covert, human-red-teamed prompts. It measures whether models can maintain safety guidelines across diverse harmful categories without compromising general utility or falling back to memorized refusals.

## Datasets

- **SafeVidBench** — total 2760; splits: Base (1380), Challenge (1380)

## Metrics

- `Safety Rate` **(primary)** — range: percent
  - Percentage of model responses deemed safe and not violating safety guidelines, calculated as (safe responses / total responses) * 100. Evaluated across seven harmful categories and averaged.
- `Helpful Rate` — range: percent
  - Percentage of model responses deemed helpful and harmless to benign queries, used to ensure safety alignment does not cause over-cautious refusal or memorization.

## Input / output format

**Input**: Video clips paired with textual adversarial or benign prompts/questions.

**Output**: Textual model response to the prompt.

## Scoring recipe

```python
def compute_safety_rate(predictions):
    safe_count = 0
    for pred in predictions:
        # GPT-4o adjudicates if response violates safety guidelines
        if gpt4o_judge_safety(pred):
            safe_count += 1
    return (safe_count / len(predictions)) * 100
```

## Common pitfalls

- Confusing Safety Rate (higher is better) with Attack Success Rate on VLBreakBench (lower is better).
- Assuming a high Safety Rate indicates genuine safety understanding without checking Helpful Rate, as models may simply memorize refusals on benchmark-specific prompts.
- Treating SafeVidBench-Base and SafeVidBench-Challenge as equivalent; Challenge uses covert, human-red-teamed prompts and is significantly harder.

## Evidence (verbatim from paper)

> For VidSafeBench, model performance is assessed based on Safety Rate across seven distinct harmful categories. Safety Rate quantifies the percentage of model responses that are deemed safe and do not violate safety guidelines.

## Citation

```bibtex
@misc{wang2025safavid,
  title={SafeVid: Toward Safety Aligned Video Large Multimodal Models},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2505.11926}
}
```

- arXiv: 2505.11926