Chinese Toxicity Detection Eval

---
name: chinese-toxicity-detection-eval
description: This protocol evaluates models on Chinese toxicity detection across two tasks: binary sentence-level classification and fine-grained toxic span extraction. It measures classification accuracy and precision/recall, while also assessing the model's ability to extract contiguous, human-readable toxic spans and the faithfulness of those explanations via confidence masking. Use when the user wants to benchmark on COLD, ToxiCN, CNTP, or asks about evaluating this task. Reports F1, Overlap F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.12321
  bibtex_key: li2026toxitrace
  confidence: high
---

# chinese-toxicity-detection-eval

> ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection — Li et al. (2026) (arXiv:2604.12321, 2026)

## What this evaluates

This protocol evaluates models on Chinese toxicity detection across two tasks: binary sentence-level classification and fine-grained toxic span extraction. It measures classification accuracy and precision/recall, while also assessing the model's ability to extract contiguous, human-readable toxic spans and the faithfulness of those explanations via confidence masking.

## Datasets

- **COLD** — total 32480; splits: test (5323)
- **ToxiCN** — total 12011; splits: test (2411)
- **CNTP** — total 2533; splits: test (2533)

## Metrics

- `Accuracy (Acc)` — range: [0, 1]
  - Proportion of correctly classified instances out of the total number of instances.
- `Precision (P)` — range: [0, 1]
  - Ratio of true positive predictions to all positive predictions (TP / (TP + FP)).
- `Recall (R)` — range: [0, 1]
  - Ratio of true positive predictions to all actual positive instances (TP / (TP + FN)).
- `F1` **(primary)** — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (P * R) / (P + R).
- `Macro-F1` — range: [0, 1]
  - Unweighted mean of F1 scores computed per class, treating all classes equally.
- `Overlap F1` **(primary)** — range: [0, 1]
  - Span-level F1 where a prediction is correct if its overlap with the gold span exceeds 50%. Precision and Recall are computed over these binary correct/incorrect judgments.
- `Character-level F1` — range: [0, 1]
  - Token-level F1 score computed by matching extracted character spans against ground-truth character spans.
- `IoU` — range: [0, 1]
  - Intersection over Union of character indices between predicted and gold spans.

## Input / output format

**Input**: Chinese text sentences (for classification) or sentences with ground-truth toxic spans (for span extraction evaluation).

**Output**: Binary label (toxic/non-toxic) for classification; contiguous toxic span(s) or character-level attribution scores for extraction.

## Scoring recipe

```python
def compute_classification_metrics(preds, golds):
    tp = fp = fn = 0
    for p, g in zip(preds, golds):
        if p == 1 and g == 1: tp += 1
        elif p == 1 and g == 0: fp += 1
        elif p == 0 and g == 1: fn += 1
    acc = (tp + (len(preds) - tp - fp)) / len(preds)
    p = tp / (tp + fp) if (tp + fp) > 0 else 0
    r = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * p * r / (p + r) if (p + r) > 0 else 0
    return acc, p, r, f1

def compute_overlap_f1(pred_spans, gold_spans):
    correct = 0
    for p, g in zip(pred_spans, gold_spans):
        if len(g) == 0: continue
        overlap = len(set(p) & set(g)) / len(g)
        if overlap > 0.5: correct += 1
    return correct / len(gold_spans)
```

## Common pitfalls

- Zero-shot LLMs perform poorly on Chinese toxicity tasks; fair comparison requires fine-tuning or instruction tuning, as noted in the experimental setup.
- Span extraction uses a strict >50% overlap threshold, which can penalize partial matches even if they capture the core toxic phrase.
- Character-level metrics in Chinese often yield fragmented spans; the protocol uses BiCSE to enforce contiguous span extraction for better evaluation.

## Evidence (verbatim from paper)

> To evaluate toxic content detection performance, we used five widely adopted metrics: Accuracy ($Acc$), Recall ($R$), Precision ($P$), $F_{1}$ and Macro-$F_{1}$ Score.

## Citation

```bibtex
@misc{li2026toxitrace,
  title={ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection},
  author={Li et al. (2026)},
  year={2026},
  note={arXiv:2604.12321}
}
```

- arXiv: 2604.12321