Ground Motion Synthesis Eval

---
name: ground-motion-synthesis-eval
description: Evaluates the ability of a generative model to synthesize realistic 3-component broadband ground motion acceleration time histories conditioned on seismic parameters. It probes the model's capacity to match empirical spectral intensities (PSA, FAS, EAS) and capture aleatory variability across different frequency bands and tectonic settings. Use when the user wants to benchmark on BBP dataset, Kik-net dataset, or asks about evaluating this task. Reports Normalized model residual (epsilon).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2309.03447
  bibtex_key: shi2023gano
  confidence: high
---

# ground-motion-synthesis-eval

> Broadband Ground Motion Synthesis via Generative Adversarial Neural Operators: Development and Validation — Shi et al. (2023) (arXiv:2309.03447, 2023)

## What this evaluates

Evaluates the ability of a generative model to synthesize realistic 3-component broadband ground motion acceleration time histories conditioned on seismic parameters. It probes the model's capacity to match empirical spectral intensities (PSA, FAS, EAS) and capture aleatory variability across different frequency bands and tectonic settings.

## Datasets

- **BBP dataset** — total ?; splits: train (-1), test (-1)
- **Kik-net dataset** — total ?; splits: train (-1), test (-1)

## Metrics

- `Normalized model residual (epsilon)` **(primary)** — range: other
  - epsilon = (ln(IM_observed) - mean(ln(IM_synthetic))) / std(ln(IM_synthetic)). The mean of epsilon quantifies model bias, and the standard deviation quantifies whether the synthetic ensemble's variability matches the empirical dataset.
- `PSA` — range: other
  - 5%-damped pseudo-spectral acceleration. Calculated as rotation-independent (RotD50) for horizontal components and as the 5%-damped response spectrum of the vertical component, over periods 0.01-10 sec.
- `FAS` — range: other
  - Fourier Amplitude Spectrum. Horizontal FAS is the power mean spectrum of the two horizontal components; vertical FAS is calculated directly from the vertical component.
- `EAS` — range: other
  - Effective Amplitude Spectrum. Calculated by smoothing the power-mean FAS (horizontal) or vertical FAS using a Konno-Ohmachi smoothing window with bandwidth 0.0333.

## Input / output format

**Input**: Conditional parameters (moment magnitude M, rupture distance R_rup, V_S30, faulting style f_type) and a 1D Gaussian Random Field sample.

**Output**: 3-component normalized acceleration time series waveforms and three normalization constants (peak ground accelerations).

## Scoring recipe

```python
def evaluate(predictions, gold, conditional_vars, dataset):
    # 1. Generate 100 synthetic realizations per scenario
    synthetics = generate_100_samples(predictions, conditional_vars)
    # 2. Compute Intensity Measures (IM: PSA, FAS, or EAS)
    im_obs = compute_IM(gold)
    im_synth = [compute_IM(s) for s in synthetics]
    # 3. Filter usable frequency range (Kik-net only)
    if dataset == 'Kik-net':
        im_obs, im_synth = filter_by_noise_ratio(im_obs, im_synth)
    # 4. Compute log-space statistics of synthetic IMs
    log_im_synth = np.log(im_synth)
    mean_log = np.mean(log_im_synth)
    std_log = np.std(log_im_synth)
    # 5. Compute normalized residual epsilon
    log_im_obs = np.log(im_obs)
    epsilon = (log_im_obs - mean_log) / std_log
    return epsilon.mean(), epsilon.std()
```

## Common pitfalls

- Frequency range filtering must be applied to Kik-net data based on noise-to-signal ratio (LUF/HUF) before computing IMs, whereas BBP uses the full frequency range.
- PSA is computed as rotation-independent (RotD50) for horizontal components, not the standard geometric mean.
- Residuals are computed in log-space; a well-calibrated model should yield mean epsilon ~ 0 and std epsilon ~ 1.

## Evidence (verbatim from paper)

> The statistical evaluation of the generated ground motions was performed (i) by comparing the PSA and FAS, for BBP dataset, or EAS, for Kik-net dataset, ordinates of the observed and synthetically generated ground motions for specific scenario bins and (ii) by performing residual analysis of the PSA and FAS or EAS of the training and testing datasets, for both the horizontal and vertical directions. ... The comparison is performed by: (i) generating 100 synthetic 3-component realizations with the same (M, R_rup, V_S30, f_type) conditional variable values of the observed ground motion, (ii) computing the relevant IM from the observed and synthetic ground motions (IM^observed, IM^cGM-GANO, respectively), (iii) calculating the mean and standard deviation of IM^cGM-GANO in log space, and (iv) computing the normalized model residual, epsilon, for each observation as: epsilon = (ln(IM^observed) - mean(ln(IM^cGM-GANO))) / std(ln(IM^cGM-GANO))

## Citation

```bibtex
@misc{shi2023gano,
  title={Broadband Ground Motion Synthesis via Generative Adversarial Neural Operators: Development and Validation},
  author={Shi et al. (2023)},
  year={2023},
  note={arXiv:2309.03447}
}
```

- arXiv: 2309.03447