Bbh Population Synthesis Eval

---
name: bbh-population-synthesis-eval
description: Evaluates whether simulated gravitational-wave observations (detection rates and chirp mass distributions) can distinguish between different compact binary population synthesis models of binary black hole formation under realistic detector sensitivities and observing durations. Use when the user wants to benchmark on Simulated aLIGO O1/O2 BBH detections, or asks about evaluating this task. Reports posterior probability.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1504.07802
  bibtex_key: stevenson2015distinguishing
  confidence: high
---

# bbh-population-synthesis-eval

> Distinguishing compact binary population synthesis models using gravitational-wave observations of coalescing binary black holes — Simon Stevenson et al. (2015) (arXiv:1504.07802, 2015)

## What this evaluates

Evaluates whether simulated gravitational-wave observations (detection rates and chirp mass distributions) can distinguish between different compact binary population synthesis models of binary black hole formation under realistic detector sensitivities and observing durations.

## Datasets

- **Simulated aLIGO O1/O2 BBH detections** — total ?; splits: (unstated)

## Metrics

- `posterior probability` **(primary)** — range: [0, 1]
  - Bayesian posterior probability of each candidate population synthesis model given the simulated detection data, calculated via repeated Monte Carlo draws (10,000 repeats).

## Input / output format

**Input**: Simulated BBH detection events (chirp masses, merger rates) drawn from a 'true' universe model, compared against a fixed set of candidate population synthesis models.

**Output**: Posterior probability distribution over the candidate models.

## Scoring recipe

```python
def compute_posterior_prob(predictions, gold_model, candidate_models):
    posteriors = []
    for model in candidate_models:
        likelihood = compute_likelihood(predictions, model)
        posterior = likelihood  # Assuming uniform prior
        posteriors.append(posterior)
    total = sum(posteriors)
    posteriors = [p / total for p in posteriors]
    return posteriors
```

## Common pitfalls

- Models with similar merger rates and chirp mass distributions remain degenerate even with ~100 observations (e.g., standard model vs. variations 5, 6, 7).
- Early observing runs (O1) have low detection rates, making it impossible to distinguish many models without longer observation times (O2+).
- Selection biases and detector noise curves significantly affect distinguishability and must be incorporated into simulations.

## Evidence (verbatim from paper)

> We simulate the observed BBH (binary black hole) systems, assuming the universe matches one of the models from Dominik et al. (2012), and calculate the posterior probability for each model. We repeat the experiment 10000 times before turning to the next model to simulate the universe.

## Citation

```bibtex
@misc{stevenson2015distinguishing,
  title={Distinguishing compact binary population synthesis models using gravitational-wave observations of coalescing binary black holes},
  author={Simon Stevenson et al. (2015)},
  year={2015},
  note={arXiv:1504.07802}
}
```

- arXiv: 1504.07802