Toxic Language Classification Eval

---
name: toxic-language-classification-eval
description: Evaluates binary toxic language classification under extreme data scarcity and severe class imbalance. It probes how well classifiers can detect the minority 'threat' class when trained on a very small labeled dataset, and measures the effectiveness of various data augmentation techniques in improving recall and macro-F1. Use when the user wants to benchmark on Seed, or asks about evaluating this task. Reports macro-averaged F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2009.12344
  bibtex_key: juuti2020little
  confidence: high
---

# toxic-language-classification-eval

> A little goes a long way: Improving toxic language classification despite data scarcity — Juuti et al. (2020) (arXiv:2009.12344, 2020)

## What this evaluates

Evaluates binary toxic language classification under extreme data scarcity and severe class imbalance. It probes how well classifiers can detect the minority 'threat' class when trained on a very small labeled dataset, and measures the effectiveness of various data augmentation techniques in improving recall and macro-F1.

## Datasets

- **Seed** — total 8000; splits: train (8000); repo https://github.com/ssg-research/language-data-augmentation

## Metrics

- `macro-averaged F1-score` **(primary)** — range: [0, 1]
  - Unweighted mean of the F1-scores for the threat (minority) and non-threat (majority) classes. F1 = 2 * (precision * recall) / (precision + recall) for each class.
- `precision (threat)` — range: [0, 1]
  - Ratio of correctly predicted threat documents to all documents predicted as threat.
- `recall (threat)` — range: [0, 1]
  - Ratio of correctly predicted threat documents to all actual threat documents.
- `ROC-AUC` — range: [0, 1]
  - Area under the receiver operating characteristic curve, computed across probability thresholds in [0,1].

## Input / output format

**Input**: Text documents labeled as either 'threat' (minority) or non-threat (majority).

**Output**: Binary class prediction (threat or non-threat) based on the highest conditional probability, or continuous probability scores for ROC-AUC computation.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, y_prob=None):
    tp = sum((y_true == 1) & (y_pred == 1))
    fp = sum((y_true == 0) & (y_pred == 1))
    fn = sum((y_true == 1) & (y_pred == 0))
    tn = sum((y_true == 0) & (y_pred == 0))
    prec_th = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    rec_th = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1_th = 2 * prec_th * rec_th / (prec_th + rec_th) if (prec_th + rec_th) > 0 else 0.0
    prec_maj = tn / (tn + fn) if (tn + fn) > 0 else 0.0
    rec_maj = tn / (tn + fp) if (tn + fp) > 0 else 0.0
    f1_maj = 2 * prec_maj * rec_maj / (prec_maj + rec_maj) if (prec_maj + rec_maj) > 0 else 0.0
    f1_macro = (f1_th + f1_maj) / 2.0
    auc = roc_auc_score(y_true, y_prob) if y_prob is not None else None
    return {'precision_threat': prec_th, 'recall_threat': rec_th, 'f1_macro': f1_macro, 'auc': auc}
```

## Common pitfalls

- The majority class F1-score is trivially 1.00 due to extreme class imbalance, so macro-averaged F1 or minority-class metrics must be used to assess performance.
- Classifiers default to a 0.5 probability threshold for binary decisions; relaxing this threshold is required to compute ROC-AUC.
- On extremely small datasets (e.g., Seed), large models like BERT may degenerate to predicting only the majority class unless augmented or randomly restarted.

## Evidence (verbatim from paper)

> We compared precision and recall for the minority class (threat), and the macro-averaged F1-score for each classifier and augmentation technique. (For brevity, we use “F1-score” from now on.) The majority class F1-score remained $1.00$ (two digit rounding) across all our experiments.

## Citation

```bibtex
@misc{juuti2020little,
  title={A little goes a long way: Improving toxic language classification despite data scarcity},
  author={Juuti et al. (2020)},
  year={2020},
  note={arXiv:2009.12344}
}
```

- arXiv: 2009.12344