Toxic Comment Classification Eval

---
name: toxic-comment-classification-eval
description: Evaluates transformer and RNN models on their ability to classify toxic comments while measuring classification accuracy and inference speed. It specifically probes identity-based bias by measuring how well models distinguish between toxic and normal comments across demographic subgroups. Use when the user wants to benchmark on Civil Comments, or asks about evaluating this task. Reports Macro AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2301.11125
  bibtex_key: coretinduchene2023toxic
  confidence: high
---

# toxic-comment-classification-eval

> A benchmark for toxic comment classification on Civil Comments dataset — Coretin Duchêne et al. (2023) (arXiv:2301.11125, 2023)

## What this evaluates

Evaluates transformer and RNN models on their ability to classify toxic comments while measuring classification accuracy and inference speed. It specifically probes identity-based bias by measuring how well models distinguish between toxic and normal comments across demographic subgroups.

## Datasets

- **Civil Comments** — total ?; splits: test (-1)

## Metrics

- `Macro AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, computed macro-averaged across classes.
- `Macro F1` — range: [0, 1]
  - F1 score computed macro-averaged across classes using a prediction threshold of 0.5.
- `GMB-Subgroup-AUC` — range: [0, 1]
  - Generalized mean (power mean) of Subgroup AUC values calculated per identity with more than 500 test examples.

## Input / output format

**Input**: Raw text of a user comment from the Civil Comments dataset.

**Output**: Binary toxicity prediction (toxic vs. normal) with associated probability scores, plus batch-level inference time.

## Scoring recipe

```python
predictions: list of float probabilities [0,1]
gold: list of int labels [0,1]
threshold = 0.5
preds_binary = [1 if p >= threshold else 0 for p in predictions]
macro_auroc = compute_auroc(gold, predictions, average='macro')
macro_f1 = f1_score(gold, preds_binary, average='macro')
# Bias metrics: compute Subgroup/BPSN/BNSP AUC per identity (filtering for >500 examples)
# Aggregate each bias metric across identities using generalized mean (power mean with exponent p)
# Report GMB-Subgroup-AUC, GMB-BPSN-AUC, GMB-BNSP-AUC
# Inference time: average batch processing time over 6,000 test batches
```

## Common pitfalls

- Evaluation is strictly restricted to the test set; training or validation metrics are not reported for final model comparison.
- Bias metrics (Sub. AUC, BPSN, BNSP) are only calculated for identities that have more than 500 examples in the test dataset.
- Inference time is reported as an average per batch over 6,000 batches, not as per-instance latency.

## Evidence (verbatim from paper)

> To measure the model's performance, we use similar metrics that were used during the kaggle : Macro AUROC, Macro F1 and Micro F1 with a threshold of 0.5, Precision and Recall. ... To combine these metrics across identities, we used the generalized mean (GM) or power mean with exponent p ... So, we report the following three bias metrics for our comparison: GMB-Subgroup-AUC is the GM for the Subgroup AUC, GMB-BPSN-AUC is the GM of the BPSN AUC, GMB-BNSP-AUC is the GM of the BNSP AUC. We restrict the evaluation to the test set only. ... Only identities with more than 500 examples in the test dataset will be included in the evaluation calculation.

## Citation

```bibtex
@misc{coretinduchene2023toxic,
  title={A benchmark for toxic comment classification on Civil Comments dataset},
  author={Coretin Duchêne et al. (2023)},
  year={2023},
  note={arXiv:2301.11125}
}
```

- arXiv: 2301.11125