Fairness And Downstream Eval

---
name: fairness-and-downstream-eval
description: Evaluates demographic fairness and downstream NLU task performance of language models. It measures bias across gender, race, and age using established fairness benchmarks, and verifies that fairness interventions do not degrade accuracy on standard classification and regression tasks. Use when the user wants to benchmark on HolisticBias, WEAT/SEAT, CrowS-Pairs, GLUE, or asks about evaluating this task. Reports Fairscore.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2205.12586
  bibtex_key: qian2022perturbation
  confidence: high
---

# fairness-and-downstream-eval

> Perturbation Augmentation for Fairer NLP — Qian et al. (2022) (arXiv:2205.12586, 2022)

## What this evaluates

Evaluates demographic fairness and downstream NLU task performance of language models. It measures bias across gender, race, and age using established fairness benchmarks, and verifies that fairness interventions do not degrade accuracy on standard classification and regression tasks.

## Datasets

- **HolisticBias** — total ?; splits: test (-1)
- **WEAT/SEAT** — total ?; splits: test (-1)
- **CrowS-Pairs** — total ?; splits: test (-1)
- **GLUE** — total ?; splits: dev (-1)

## Metrics

- `Fairscore` **(primary)** — range: percent
  - Percentage of classifier predictions that change when the input is demographically altered using the perturber. Lower values indicate higher fairness robustness.
- `HolisticBias` — range: percent
  - Percentage of descriptor pairs by axis where the distribution of pseudo-log-likelihoods in templated sentences significantly differs. Lower values indicate less bias.
- `WEAT/SEAT` — range: percent
  - Percentage of statistically significant association tests and their average effect size. Lower values indicate less bias.
- `CrowS-Pairs` — range: percent
  - Percentage of examples where the model assigns a higher likelihood to the stereotyping sentence over the less stereotyping sentence. Scores closer to 50% indicate demographic parity.
- `GLUE Accuracy/Correlation` — range: other
  - Matthew's correlation for CoLA, Pearson's correlation for STS-B, and accuracy for all other tasks. Reported as median of 5 seeded runs.

## Input / output format

**Input**: Single sentences or sentence pairs for NLU tasks; templated sentences containing demographic descriptors for fairness metrics.

**Output**: Model predictions (class labels) or token likelihoods for sentence pairs.

## Scoring recipe

```python
def compute_fairscore(model, eval_set, perturber):
    original_preds = [model.predict(x) for x in eval_set]
    perturbed_set = [perturber(x) for x in eval_set]
    perturbed_preds = [model.predict(x) for x in perturbed_set]
    return sum(p != q for p, q in zip(original_preds, perturbed_preds)) / len(eval_set) * 100
```

## Common pitfalls

- Conflicting results across different fairness metrics (e.g., CrowS-Pairs vs WEAT/SEAT) require careful interpretation.
- Fairscore is only defined for classification tasks, excluding regression tasks like STS-B.
- For CrowS-Pairs, a score closer to 50% indicates a fairer model, whereas lower scores are better for HolisticBias and WEAT/SEAT.

## Evidence (verbatim from paper)

> Finally, we compute the fairscore as an extrinsic fairness evaluation metric. Recall that, given a classifier and evaluation set, the fairscore of the classifier is the percentage of predictions that change when the input is demographically altered with the perturber.

## Citation

```bibtex
@misc{qian2022perturbation,
  title={Perturbation Augmentation for Fairer NLP},
  author={Qian et al. (2022)},
  year={2022},
  note={arXiv:2205.12586}
}
```

- arXiv: 2205.12586