Behavioral Prediction Eval

---
name: behavioral-prediction-eval
description: Predicts individual strategic decisions by conditioning on structured psychometric trait profiles (e.g., Big Five personality traits). It probes a model's ability to map high-dimensional psychological embeddings to discrete behavioral outcomes in unseen situational contexts. Use when the user wants to benchmark on Strategic Scenario Dataset, or asks about evaluating this task. Reports balanced accuracy, macro-F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.17222
  bibtex_key: yellin2026decoding
  confidence: high
---

# behavioral-prediction-eval

> Decoding the Human Factor: High Fidelity Behavioral Prediction for Strategic Foresight — Yellin et al. (2026) (arXiv:2602.17222, 2026)

## What this evaluates

Predicts individual strategic decisions by conditioning on structured psychometric trait profiles (e.g., Big Five personality traits). It probes a model's ability to map high-dimensional psychological embeddings to discrete behavioral outcomes in unseen situational contexts.

## Datasets

- **Strategic Scenario Dataset** — total ?; splits: train (-1), test (-1)

## Metrics

- `balanced accuracy` **(primary)** — range: [0, 1]
  - Computes the average recall across all classes, correcting for class imbalance by treating each class equally regardless of sample size.
- `macro-F1` **(primary)** — range: [0, 1]
  - Calculates the unweighted mean of the F1 scores for each class, providing a harmonic mean of precision and recall per class.
- `accuracy` — range: [0, 1]
  - Standard classification accuracy: the proportion of correctly predicted labels out of the total number of predictions.

## Input / output format

**Input**: Structured psychometric trait profiles (ranging from 5 to 74 traits) paired with a strategic scenario/question.

**Output**: Discrete class label corresponding to one of five multiple-choice behavioral outcomes.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, n_classes=5):
    acc = np.mean(y_true == y_pred)
    recalls = [np.mean(y_pred[y_true==c] == c) if np.sum(y_true==c) > 0 else 0 for c in range(n_classes)]
    bal_acc = np.mean(recalls)
    f1s = [f1_score([1 if y==c else 0 for y in y_true],
                    [1 if y==c else 0 for y in y_pred], average='binary') for c in range(n_classes)]
    macro_f1 = np.mean(f1s)
    return acc, bal_acc, macro_f1
```

## Common pitfalls

- Class imbalance in response labels makes standard accuracy misleading; balanced accuracy and macro-F1 must be reported as primary metrics.
- Confidence intervals are computed via participant-level bootstrap resampling, not standard sample-level bootstrapping.
- Chance performance is approximately 0.20 due to the 5-way multiple-choice format, so scores near 0.20-0.30 are not necessarily poor.

## Evidence (verbatim from paper)

> Because the response labels are class-imbalanced, we report balanced accuracy and macro-F1 as primary metrics, as they better reflect performance across classes. We additionally report standard accuracy as a familiar reference measure.

## Citation

```bibtex
@misc{yellin2026decoding,
  title={Decoding the Human Factor: High Fidelity Behavioral Prediction for Strategic Foresight},
  author={Yellin et al. (2026)},
  year={2026},
  note={arXiv:2602.17222}
}
```

- arXiv: 2602.17222