Fraud R1 Eval

---
name: fraud-r1-eval
description: This benchmark evaluates large language models' robustness against multi-round fraud and phishing inducements. It probes whether models can successfully identify and defend against deceptive prompts across five fraud categories under both standard helpful-assistant and role-play settings, while also measuring cross-lingual performance gaps. Use when the user wants to benchmark on Fraud-R1, or asks about evaluating this task. Reports Defense Success Rate (DSR).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2502.12904
  bibtex_key: yang2025fraudr1
  confidence: high
---

# fraud-r1-eval

> Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements — Yang et al. (2025) (arXiv:2502.12904, 2025)

## What this evaluates

This benchmark evaluates large language models' robustness against multi-round fraud and phishing inducements. It probes whether models can successfully identify and defend against deceptive prompts across five fraud categories under both standard helpful-assistant and role-play settings, while also measuring cross-lingual performance gaps.

## Datasets

- **Fraud-R1** — total 8564; splits: test (8564); repo https://github.com/kaustpradalab/Fraud-R1

## Metrics

- `Defense Success Rate (DSR)` **(primary)** — range: percent
  - The percentage of fraud cases where the model successfully identifies the inducement and refuses or defends against it across multiple conversation rounds. Calculated as (number of successfully defended cases / total cases) × 100.

## Input / output format

**Input**: Multi-turn conversational prompts simulating fraud/phishing scenarios (fake job postings, impersonation, phishing scams, online relationship, fraudulent service) in English and Chinese, delivered under either a standard Helpful-Assistant or a Role-play instruction.

**Output**: The model's generated response at each turn of the multi-round conversation.

## Scoring recipe

```python
successful_count = 0
for case in dataset:
    response = model.generate(case.prompt)
    if judge_gpt4o_mini.evaluate(response, case.gold_label) == "success":
        successful_count += 1
dsr = (successful_count / len(dataset)) * 100
```

## Common pitfalls

- Role-play prompts significantly degrade fraud detection performance compared to standard assistant prompts.
- Models show a pronounced performance gap between English and Chinese inputs, with English generally yielding higher DSR.
- Multi-round escalation increases the number of turns needed for detection, raising real-world financial risk.

## Evidence (verbatim from paper)

> Table[2] presents the comprehensive Defense Success Rate (DSR) of different LLMs, including overall and category-specific scores across Helpful Assistant and Role-play settings. Our evaluation employs GPT-4o-mini as an automated judge to assess the responses of the model over multiple rounds.

## Citation

```bibtex
@misc{yang2025fraudr1,
  title={Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements},
  author={Yang et al. (2025)},
  year={2025},
  note={arXiv:2502.12904}
}
```

- arXiv: 2502.12904