Mgs Stereotype Detection Eval

---
name: mgs-stereotype-detection-eval
description: Evaluates the ability of classifiers to detect and categorize stereotypes across multiple intersecting dimensions (race, gender, profession, religion) in text. It also probes cross-dataset generalization and quantifies generative bias deviation in LLMs. Use when the user wants to benchmark on MGS Dataset (MGSD), StereoSet, CrowsPairs, or asks about evaluating this task. Reports Macro F1 Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2404.01768
  bibtex_key: wu2024stereotypedetection
  confidence: high
---

# mgs-stereotype-detection-eval

> Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach — Wu et al. (2024) (arXiv:2404.01768, 2024)

## What this evaluates

Evaluates the ability of classifiers to detect and categorize stereotypes across multiple intersecting dimensions (race, gender, profession, religion) in text. It also probes cross-dataset generalization and quantifies generative bias deviation in LLMs.

## Datasets

- **MGS Dataset (MGSD)** — total 51867; splits: test (-1)
- **StereoSet** — total ?; splits: test (-1)
- **CrowsPairs** — total ?; splits: test (-1)

## Metrics

- `Macro F1 Score` **(primary)** — range: [0, 1]
  - The unweighted mean of recall and precision calculated per stereotype dimension (Race, Gender, Profession, Religion) and averaged across classes.
- `Precision` — range: [0, 1]
  - The ratio of correctly predicted positive stereotype instances to the total predicted positives, macro-averaged across dimensions.
- `Recall` — range: [0, 1]
  - The ratio of correctly predicted positive stereotype instances to the total actual positives, macro-averaged across dimensions.
- `Accuracy` — range: [0, 1]
  - The proportion of total instances correctly classified across all stereotype dimensions.

## Input / output format

**Input**: Text snippets or sentences containing potential stereotypical content, labeled with one or multiple stereotype dimensions (Race, Gender, Profession, Religion).

**Output**: Predicted stereotype dimension(s) and stereotype label per instance.

## Scoring recipe

```python
def compute_macro_metrics(y_true, y_pred, classes):
    precisions, recalls, f1s = [], [], []
    for c in classes:
        tp = sum(1 for t, p in zip(y_true, y_pred) if t == c and p == c)
        fp = sum(1 for t, p in zip(y_true, y_pred) if t != c and p == c)
        fn = sum(1 for t, p in zip(y_true, y_pred) if t == c and p != c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        precisions.append(prec)
        recalls.append(rec)
        f1s.append(f1)
    return sum(precisions)/len(precisions), sum(recalls)/len(recalls), sum(f1s)/len(f1s)
```

## Common pitfalls

- Evaluating only on a single stereotype dimension ignores the paper's core focus on intersectionality and multi-dimensional detection.
- Cross-dataset generalization scores vary drastically; testing solely on CrowsPairs or StereoSet without MGSD may misrepresent model robustness.
- Explainability tools like SHAP and LIME provide local approximations that may not accurately reflect the model's global decision boundaries.

## Evidence (verbatim from paper)

> The results in Table [1] show that multiple-dimension stereotype detector consistently outperform single-dimension stereotype counterparts across all stereotype dimensions—Race, Profession, Gender, Religion—as well as in all Macro evaluation metrics: Precision, Recall, and F1 Score.

## Citation

```bibtex
@misc{wu2024stereotypedetection,
  title={Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach},
  author={Wu et al. (2024)},
  year={2024},
  note={arXiv:2404.01768}
}
```

- arXiv: 2404.01768