Fairx Gcig Eval

---
name: fairx-gcig-eval
description: Evaluates whether a model maintains predictive utility while achieving procedural fairness (explanation invariance across protected groups) and outcome fairness. It probes the alignment between equalized odds and group-level feature attribution consistency. Use when the user wants to benchmark on Adult, German Credit, COMPAS, Bank Marketing, or asks about evaluating this task. Reports GCIG.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.11140
  bibtex_key: [REDACTED]
  confidence: high
---

# fairx-gcig-eval

> Procedural Fairness via Group Counterfactual Explanation — Popoola et al. (2026) (arXiv:2603.11140, 2026)

## What this evaluates

Evaluates whether a model maintains predictive utility while achieving procedural fairness (explanation invariance across protected groups) and outcome fairness. It probes the alignment between equalized odds and group-level feature attribution consistency.

## Datasets

- **Adult** — total 4522; splits: train (-1), val (-1), test (-1)
- **German Credit** — total 1000; splits: train (-1), val (-1), test (-1)
- **COMPAS** — total 6172; splits: train (-1), val (-1), test (-1)
- **Bank Marketing** — total 45211; splits: train (-1), val (-1), test (-1)

## Metrics

- `GCIG` **(primary)** — range: other
  - Group Counterfactual Integrated Gradients. Computes Integrated Gradients for each instance relative to group-conditional baselines and measures the cross-group variation in feature attributions. Lower values indicate higher explanation invariance.
- `EO Gap` — range: other
  - Equalized Odds gap. Measures the absolute difference in True Positive Rates and False Positive Rates across protected groups. Lower values indicate better outcome fairness.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall. Higher values indicate better predictive utility.

## Input / output format

**Input**: Tabular dataset with standardized continuous features, one-hot encoded categorical features, a binary target label, and a protected attribute column.

**Output**: Binary predictions for utility/fairness metrics, and per-instance feature attribution vectors (Integrated Gradients) for procedural fairness computation.

## Scoring recipe

```python
def compute_metrics(preds, gold, groups, model, inputs):
    tp = sum((p == 1) & (g == 1) for p, g in zip(preds, gold))
    fp = sum((p == 1) & (g == 0) for p, g in zip(preds, gold))
    fn = sum((p == 0) & (g == 1) for p, g in zip(preds, gold))
    f1 = 2 * tp / (2 * tp + fp + fn) if (2 * tp + fp + fn) > 0 else 0.0
    tpr_a = sum((p == 1) & (g == 1) & (gr == 'A') for p, g, gr in zip(preds, gold, groups)) / max(sum((g == 1) & (gr == 'A') for g, gr in zip(gold, groups)), 1)
    tpr_b = sum((p == 1) & (g == 1) & (gr == 'B') for p, g, gr in zip(preds, gold, groups)) / max(sum((g == 1) & (gr == 'B') for g, gr in zip(gold, groups)), 1)
    eo_gap = abs(tpr_a - tpr_b)
    return f1, eo_gap
```

## Common pitfalls

- Assuming low EO Gap guarantees low GCIG; the paper shows only weak correlation (r=0.244) and >94% unexplained variance.
- Using a single global baseline for Integrated Gradients instead of group-conditional baselines, which GCIG explicitly requires.
- Failing to stratify data splits by both label and protected attribute, which can bias disparity estimates.

## Evidence (verbatim from paper)

> To address RQ1, Table 4.2 summarizes predictive performance (F1), outcome fairness (EO gap), and procedural fairness (GCIG) across four datasets. Results are reported as mean ± standard deviation over 5-fold cross-validation. Across all datasets, FairX consistently reduces GCIG relative to the unconstrained baseline and all outcome-focused baselines.

## Citation

```bibtex
@misc{popoola2026procedural,
  title={Procedural Fairness via Group Counterfactual Explanation},
  author={Popoola et al. (2026)},
  year={2026},
  note={arXiv:2603.11140}
}
```

- arXiv: 2603.11140