Gwtc3 Spin Population Eval

---
name: gwtc3-spin-population-eval
description: Evaluates the ability of population inference models to explain the observed spin distributions of binary black holes in the GWTC-3 catalog. It probes how well different astrophysical formation scenarios (e.g., field vs. dynamical assembly, zero-spin subpopulations) fit the gravitational-wave data. Use when the user wants to benchmark on GWTC-3, or asks about evaluating this task. Reports Bayes factor ($\mathcal{B}$).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2209.02206
  bibtex_key: tong2022population
  confidence: high
---

# gwtc3-spin-population-eval

> The population properties of spinning black holes using Gravitational-wave Transient Catalog 3 — Hui Tong et al. (2022) (arXiv:2209.02206, 2022)

## What this evaluates

Evaluates the ability of population inference models to explain the observed spin distributions of binary black holes in the GWTC-3 catalog. It probes how well different astrophysical formation scenarios (e.g., field vs. dynamical assembly, zero-spin subpopulations) fit the gravitational-wave data.

## Datasets

- **GWTC-3** — total ?; splits: full (-1); repo https://github.com/HuiTong5/GWTC-3_pop_spin

## Metrics

- `Bayes factor ($\mathcal{B}$)` **(primary)** — range: other
  - Ratio of Bayesian evidences between two competing models: $\mathcal{B} = Z_1 / Z_2$. The natural logarithm $\ln \mathcal{B}$ is reported. Values $>1$ (or $\ln \mathcal{B} > 0$) favor the numerator model. Incorporates an Occam penalty for model complexity.

## Input / output format

**Input**: Gravitational-wave strain data and individual event posterior samples for binary black hole mergers from the GWTC-3 catalog.

**Output**: Model selection statistics including $\ln \mathcal{B}$ and $\Delta \ln \mathcal{L}_{\max}$ for each predefined population model variant (e.g., DEFAULT, EXTENDED, ISOSUBPOP, NONIDENTICAL).

## Scoring recipe

```python
def compute_bayes_factor(model_evidence, reference_evidence):
    return model_evidence / reference_evidence

def evaluate_models(event_data, model_variants):
    evidences = {}
    for model in model_variants:
        evidences[model] = compute_bayesian_evidence(event_data, model)
    reference = max(evidences.values())
    results = {}
    for model, evidence in evidences.items():
        results[model] = {
            'ln_B': np.log(evidence / reference),
            'max_likelihood': compute_max_likelihood(event_data, model)
        }
    return results
```

## Common pitfalls

- Confusing Bayes factor with maximum likelihood ratio; Bayes factors penalize complex models via the Occam factor even if they achieve higher likelihood.
- Including outlier events like GW191109 can drastically shift model preferences due to systematic uncertainties or data quality issues.
- Undersampling in high-dimensional hyperparameter spaces can lead to underestimated maximum likelihood values for nested models.

## Evidence (verbatim from paper)

> We carry out population inference using the model variants summarized in Table I. Our findings—excluding GW191109—are summarized in Table II. The table shows both Bayes factors and maximum likelihood ratios in order to separate out how the Bayes factor is influenced by the quality of fit versus the Occam penalty.

## Citation

```bibtex
@misc{tong2022population,
  title={The population properties of spinning black holes using Gravitational-wave Transient Catalog 3},
  author={Hui Tong et al. (2022)},
  year={2022},
  note={arXiv:2209.02206}
}
```

- arXiv: 2209.02206