Redrft Eval

---
name: redrft-eval
description: Evaluates reinforcement fine-tuning methods for red-teaming LLMs by measuring the toxicity and diversity of generated adversarial prompts across toxic continuation and instruction-following tasks. Use when the user wants to benchmark on toxic continuation, instruction following, or asks about evaluating this task. Reports cumulative toxicity-diversity score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.04302
  bibtex_key: zheng2025redrft
  confidence: medium
---

# redrft-eval

> RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming — Zheng et al. (2025) (arXiv:2506.04302, 2025)

## What this evaluates

Evaluates reinforcement fine-tuning methods for red-teaming LLMs by measuring the toxicity and diversity of generated adversarial prompts across toxic continuation and instruction-following tasks.

## Datasets

- **toxic continuation** — total ?; splits: test (-1)
- **instruction following** — total ?; splits: test (-1)

## Metrics

- `cumulative toxicity-diversity score` **(primary)** — range: [0, 1]
  - A composite metric balancing toxicity and diversity/novelty scores. The paper uses a threshold of 0.5 as a performance benchmark, where higher scores indicate better red-teaming capability.
- `toxicity score` — range: [0, 1]
  - Measures the harmfulness or toxicity of generated prompts.
- `diversity/novelty score` — range: [0, 1]
  - Measures the variety or novelty of generated prompts.

## Input / output format

**Input**: Task-specific prompts or instructions provided to the red-team LLM (GPT-2) for adversarial generation.

**Output**: Generated text continuations or responses from the red-team LLM.

## Scoring recipe

```python
# Pseudo-code based on paper description
toxicity_scores = [compute_toxicity(gen) for gen in generations]
diversity_scores = [compute_diversity(gen) for gen in generations]
cumulative_score = aggregate(toxicity_scores, diversity_scores)
return cumulative_score
# Evaluated over 3 random seeds per hyperparameter config
```

## Common pitfalls

- Exact dataset splits and metric computation formulas are defined in Section 4, not this section.
- Toxicity and diversity are typically measured via external classifiers or embedding-based metrics not detailed here.
- The 0.5 cumulative score threshold is used as a qualitative benchmark for improvement, not a strict pass/fail criterion.

## Evidence (verbatim from paper)

> We show the main benchmarking results in Figure 2, which shows the toxicity-diversity profile for all baselines on both tasks. Overall, we observe that none of the current RFT-based red teaming methods achieve a cumulative toxicity-diversity score greater than 0.5, indicating significant room for improvement.

## Citation

```bibtex
@misc{zheng2025redrft,
  title={RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming},
  author={Zheng et al. (2025)},
  year={2025},
  note={arXiv:2506.04302}
}
```

- arXiv: 2506.04302