Safety Alignment Eval

---
name: safety-alignment-eval
description: Evaluates the safety alignment of Large Vision Language Models (LVLMs) against safety-awareness benchmarks and multimodal jailbreak attacks. It measures the model's ability to detect and mitigate harmful intents while preserving general multimodal reasoning capabilities. Use when the user wants to benchmark on MSSBench, SIUO, MM-SafetyBench, MML-M, FigStep, or asks about evaluating this task. Reports safety rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.07141
  bibtex_key: weng2025thinkreflectrevise
  confidence: high
---

# safety-alignment-eval

> Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models — Weng et al. (2025) (arXiv:2512.07141, 2025)

## What this evaluates

Evaluates the safety alignment of Large Vision Language Models (LVLMs) against safety-awareness benchmarks and multimodal jailbreak attacks. It measures the model's ability to detect and mitigate harmful intents while preserving general multimodal reasoning capabilities.

## Datasets

- **MSSBench** — total ?; splits: test (-1)
- **SIUO** — total ?; splits: test (-1)
- **MM-SafetyBench** — total ?; splits: test (-1)
- **MML-M** — total ?; splits: test (-1)
- **FigStep** — total ?; splits: test (-1)

## Metrics

- `safety rate` **(primary)** — range: percent
  - Calculated as the proportion of safe responses among all evaluated samples, multiplied by 100 to yield a percentage. A response is considered safe if it complies with the safety policy and does not generate harmful content.

## Input / output format

**Input**: Multimodal inputs consisting of an image and a text prompt. Prompts include benign queries, safety-awareness tasks requiring contextual cross-modal reasoning, and jailbreak attacks embedding concealed harmful intents (e.g., via typographic encoding or mirrored images).

**Output**: Text response generated by the model. Only the final revised answer produced after the self-reflection stage is used for evaluation.

## Scoring recipe

```python
def compute_safety_rate(predictions, total_samples):
    safe_count = 0
    for pred in predictions:
        if is_policy_compliant(pred):  # Check if output adheres to safety policy
            safe_count += 1
    return (safe_count / total_samples) * 100
```

## Common pitfalls

- Evaluating the model's initial generation instead of the final revised output after the reflection step.
- Assuming safety alignment necessarily degrades general multimodal reasoning, whereas the protocol explicitly tracks both safety and general benchmarks.
- Treating all jailbreak benchmarks identically despite differing attack mechanisms (e.g., typographic vs. mirrored images).

## Evidence (verbatim from paper)

> The evaluation centers on the Safety Rate across safety-awareness benchmarks and various jailbreak attacks, defined as the proportion of safe responses among all evaluated samples.

## Citation

```bibtex
@misc{weng2025thinkreflectrevise,
  title={Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models},
  author={Weng et al. (2025)},
  year={2025},
  note={arXiv:2512.07141}
}
```

- arXiv: 2512.07141