Bias Detection Mitigation Eval

---
name: bias-detection-mitigation-eval
description: Evaluates a pipeline for detecting and mitigating representation bias and explicit stereotypes in text corpora. It measures how well the pipeline generates attribute-specific word lists, quantifies demographic representation imbalances, and identifies stereotypical language compared to human annotations and baselines. Use when the user wants to benchmark on Small Heap, Small Heap Neutral, StereoSet (filtered), Small Heap Annotated, or asks about evaluating this task. Reports DR score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.10734
  bibtex_key: gorg2025textualbias
  confidence: high
---

# bias-detection-mitigation-eval

> Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation — Görge et al. (2025) (arXiv:2512.10734, 2025)

## What this evaluates

Evaluates a pipeline for detecting and mitigating representation bias and explicit stereotypes in text corpora. It measures how well the pipeline generates attribute-specific word lists, quantifies demographic representation imbalances, and identifies stereotypical language compared to human annotations and baselines.

## Datasets

- **Small Heap** — total 50000000; splits: test (-1)
- **Small Heap Neutral** — total ?; splits: test (-1)
- **StereoSet (filtered)** — total 237; splits: test (237)
- **Small Heap Annotated** — total 134; splits: test (134)

## Metrics

- `DR score` **(primary)** — range: [0, 0.5]
  - Quantifies representation bias by measuring deviation from a uniform distribution across sensitive attribute groups. Calculated per attribute; for gender, the maximum value is 0.5.
- `Stereotype score` — range: [0, 1]
  - Assesses the strength of explicit stereotypes in text on a continuous scale from 0 (small/no stereotype) to 1 (strong stereotype).

## Input / output format

**Input**: Text snippets or sentences from corpora (e.g., Small Heap, StereoSet intersentence examples).

**Output**: For word list generation: lists of attribute-specific word tuples. For bias measurement: DR scores per attribute. For stereotype detection: a continuous score between 0 and 1 indicating stereotype strength.

## Scoring recipe

```python
def calc_dr(group_counts, total):
    expected = total / len(group_counts)
    return sum(abs(c - expected) for c in group_counts) / (2 * total)

def calc_overlap(generated, reference):
    return len(set(generated) & set(reference)) / len(reference)

def calc_stereotype(text, model):
    return model.score(text) # Returns float in [0, 1]
```

## Common pitfalls

- The maximum possible DR score varies by sensitive attribute (e.g., 0.5 for gender), so scores must be interpreted relative to their attribute-specific bounds.
- StereoSet evaluation strictly excludes anti-stereotypes and only uses intersentence examples, which limits direct comparison to full StereoSet benchmarks.
- Frequency-based ranking for age/religion attributes yields ambiguous results; generation-based ranking is required instead.

## Evidence (verbatim from paper)

> We validate the $DR$ score as a bias indicator for representation biases by comparing $DR_{gender}$ scores between the Small Heap and Small Heap Neutral datasets.

## Citation

```bibtex
@misc{gorg2025textualbias,
  title={Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation},
  author={Görge et al. (2025)},
  year={2025},
  note={arXiv:2512.10734}
}
```

- arXiv: 2512.10734