Faircontrast Tabular Eval

---
name: faircontrast-tabular-eval
description: This evaluation probes a model's ability to learn fair representations from tabular data by balancing predictive accuracy with demographic parity. It measures how effectively the model mitigates bias across privileged and unprivileged groups defined by sensitive attributes such as gender or age. Use when the user wants to benchmark on Adult, German Credit, Heritage Health, or asks about evaluating this task. Reports Demographic Parity (DP).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.02017
  bibtex_key: tayebi2025faircontrast
  confidence: high
---

# faircontrast-tabular-eval

> FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data — Tayebi et al. (2025) (arXiv:2510.02017, 2025)

## What this evaluates

This evaluation probes a model's ability to learn fair representations from tabular data by balancing predictive accuracy with demographic parity. It measures how effectively the model mitigates bias across privileged and unprivileged groups defined by sensitive attributes such as gender or age.

## Datasets

- **Adult** — total 48842; splits: unspecified (-1)
- **German Credit** — total 1000; splits: unspecified (-1)
- **Heritage Health** — total 50000; splits: unspecified (-1)

## Metrics

- `Accuracy` — range: [0, 1]
  - Fraction of correctly predicted labels out of total instances.
- `Demographic Parity (DP)` **(primary)** — range: [0, 1]
  - Absolute difference in the rate of positive predictions between the privileged group (S=1) and the unprivileged group (S=0). Lower values indicate higher fairness.

## Input / output format

**Input**: Tabular feature vectors containing standard attributes and a sensitive attribute column (e.g., gender or age), along with a target label.

**Output**: Predicted class label (or probability) for each instance.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, sensitive_attr):
    accuracy = np.mean(y_pred == y_true)
    p_priv = np.mean(y_pred[sensitive_attr == 1])
    p_unpriv = np.mean(y_pred[sensitive_attr == 0])
    dp = abs(p_priv - p_unpriv)
    return accuracy, dp
```

## Common pitfalls

- Results are reported as average accuracy and maximum DP over five random seeds, not a single run.
- DP is mathematically defined for binary sensitive attributes, but Heritage Health uses nine age categories; the paper does not specify how DP is aggregated for multi-category sensitive attributes.
- The privileged group (S=1) definition varies by dataset (e.g., older age for German Credit, specific gender for Adult) and must be correctly assigned before computing DP.

## Evidence (verbatim from paper)

> In this study, we adopt Demographic Parity (DP), also known as statistical parity, as our main fairness metric. Demographic parity ensures that the probability of receiving a favorable outcome is being equitably distributed across groups(privileged and unprivileged). Specifically, this metric requires that the likelihood of all positive predictions (both true positives and false positives) be similar across these groups. Thus, discrimination or disparities can be quantified by measuring the difference between the conditional probabilities of positive predictions for the privileged and unprivileged groups: |P(Ŷ=1∣X,S=1)−P(Ŷ=1∣X,S=0)|

## Citation

```bibtex
@misc{tayebi2025faircontrast,
  title={FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data},
  author={Tayebi et al. (2025)},
  year={2025},
  note={arXiv:2510.02017}
}
```

- arXiv: 2510.02017