Stead Magnitude Estimation Eval

---
name: stead-magnitude-estimation-eval
description: Evaluates a deep learning model's ability to directly estimate earthquake magnitude (both local ML and duration Md) from raw, unprocessed single-station seismograms without normalization or instrument response correction. It probes the model's robustness to site effects, regional calibration differences, and varying signal-to-noise ratios. Use when the user wants to benchmark on STEAD, or asks about evaluating this task. Reports mean error.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1911.05975
  bibtex_key: mousavi2019machine
  confidence: high
---

# stead-magnitude-estimation-eval

> A Machine-Learning Approach for Earthquake Magnitude Estimation — Mousavi et al. (2019) (arXiv:1911.05975, 2019)

## What this evaluates

Evaluates a deep learning model's ability to directly estimate earthquake magnitude (both local ML and duration Md) from raw, unprocessed single-station seismograms without normalization or instrument response correction. It probes the model's robustness to site effects, regional calibration differences, and varying signal-to-noise ratios.

## Datasets

- **STEAD** — total 300000; splits: train (-1), val (-1), test (-1)

## Metrics

- `mean error` **(primary)** — range: other
  - Average difference between predicted and cataloged magnitude values across the test set.
- `standard deviation of prediction error` — range: other
  - Standard deviation of the differences between predicted and true magnitudes.
- `coefficient of determination` — range: [0, 1]
  - R^2 score measuring the proportion of variance in the true magnitudes explained by the model.

## Input / output format

**Input**: Band-pass filtered (1.0-40.0 Hz) single-station seismogram waveforms, ≤ 30 seconds long (from 1 second before P-wave onset to end of S coda), with SNR > 20 dB.

**Output**: A single floating-point magnitude value (predicting either ML or Md).

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth):
    errors = predictions - ground_truth
    mean_error = np.mean(errors)
    std_error = np.std(errors)
    ss_res = np.sum((ground_truth - predictions) ** 2)
    ss_tot = np.sum((ground_truth - np.mean(ground_truth)) ** 2)
    r2 = 1 - (ss_res / ss_tot) if ss_tot != 0 else 0
    return {'mean_error': mean_error, 'std_error': std_error, 'r2': r2}
```

## Common pitfalls

- Performance degrades significantly at magnitude extremes due to fewer training/test samples at the bounds.
- Site effects (surface vs. borehole stations) and regional calibration differences heavily influence accuracy, making global models sensitive to station type.
- Signal-to-noise ratio has a greater impact on performance than network depth, requiring careful data filtering to avoid misleading results.

## Evidence (verbatim from paper)

> We use a selected portion of STanford EArthquake Dataset (STEAD) 15 for training of the network. STEAD is a global dataset of labeled seismograms including local earthquake and seismic noise waveforms. Here, we only used ~ 300,000 earthquake waveforms recorded at epicentral distances of less than 1 degree, for which their full waveforms (from 1 second before P until the end of the S coda) are equal or less than 30 seconds. ... used 70% of each subset for the training and 10% and 20% for the validation and testing respectively. ... Overall the network is able to predict earthquake magnitudes with a mean error close to zero and standard deviation of ~ 0.2. ... Although the coefficient of determination decreases slightly, due to relatively smaller size of the training set, we see only a small improvement in standard deviation of prediction error.

## Citation

```bibtex
@misc{mousavi2019machine,
  title={A Machine-Learning Approach for Earthquake Magnitude Estimation},
  author={Mousavi et al. (2019)},
  year={2019},
  note={arXiv:1911.05975}
}
```

- arXiv: 1911.05975