Profile Image Conjoint Eval

---
name: profile-image-conjoint-eval
description: This protocol evaluates the causal impact of specific profile image features (smile, body-shot, and gender) on lender selection preferences in a simulated micro-lending marketplace. It uses a conjoint-style choice experiment with GAN-generated images to isolate how visual cues influence funding decisions independent of borrower creditworthiness. Use when the user wants to benchmark on Custom GAN-generated profile images, or asks about evaluating this task. Reports Average Treatment Effect (ATE).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2209.01235
  bibtex_key: athey2022smiles
  confidence: high
---

# profile-image-conjoint-eval

> Smiles in Profiles: Improving Efficiency While Reducing Disparities in Online Marketplaces — Athey et al. (2022) (arXiv:2209.01235, 2022)

## What this evaluates

This protocol evaluates the causal impact of specific profile image features (smile, body-shot, and gender) on lender selection preferences in a simulated micro-lending marketplace. It uses a conjoint-style choice experiment with GAN-generated images to isolate how visual cues influence funding decisions independent of borrower creditworthiness.

## Datasets

- **Custom GAN-generated profile images** — total 4920; splits: test (4920)

## Metrics

- `Average Treatment Effect (ATE)` **(primary)** — range: percent
  - Estimated via logistic regression of a binary choice outcome on indicator variables for smile, body-shot, and male features. Reported as the average marginal effect (change in probability of being chosen) for each feature.

## Input / output format

**Input**: Pairs of GAN-generated profile images, each manipulated to vary in smile, body-shot, and male features, presented to subjects alongside basic borrower context.

**Output**: Binary selection: subject chooses one profile from each pair.

## Scoring recipe

```python
# For each choice instance i:
# y_i = 1 if profile j is chosen, else 0
# X_i = [male_j, smile_j, body_shot_j, profile_fixed_effects]
# Fit logistic regression: P(y_i=1) = sigmoid(alpha*male + beta*smile + gamma*body_shot + mu)
# ATE = marginal effect of each feature on P(y_i=1) at sample means
```

## Common pitfalls

- Subjects are recruited from Prolific and are not actual Kiva lenders, so preferences may not generalize to real platform users.
- Only 6 choice instances per subject may limit the precision of individual-level preference estimates.
- GAN-generated images, while realistic, may lack subtle cues present in real borrower photos.

## Evidence (verbatim from paper)

> Average treatment effects. Subjects were given a pair of profile pictures and asked which one they preferred. Suppose the preference has a systematic component, which depends on male, smile, and body-shot, and there is also an idiosyncratic random component ε_ij. The utility of subject i choosing the option j is written: u_ij = α*male_j + β*smile_j + γ*body-shot_j + μ_j + ε_ij. ... We find that all the features of interest are statistically significant and have high magnitudes: male and body-shot lead to lower outcomes, while smile leads to higher outcomes. ... The average marginal effect of male is a 31% reduction in the probability of being selected; there is a 17% drop for the body-shot, and a 34% increase for smile.

## Citation

```bibtex
@misc{athey2022smiles,
  title={Smiles in Profiles: Improving Efficiency While Reducing Disparities in Online Marketplaces},
  author={Athey et al. (2022)},
  year={2022},
  note={arXiv:2209.01235}
}
```

- arXiv: 2209.01235