Prime Dp Eval

---
name: prime-dp-eval
description: Evaluates a pre-trained seismic model's multi-task capability on single-station waveforms, specifically phase picking (Pg, Sg, Pn, Sn), P-wave polarization classification, and seismic event type classification. The protocol tests generalization across temporal splits and transfer learning on local data to mitigate dataset imbalance. Use when the user wants to benchmark on CSNCD, or asks about evaluating this task. Reports recall.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2408.01919
  bibtex_key: yu2024prime_dp
  confidence: high
---

# prime-dp-eval

> PRIME-DP: Pre-trained Integrated Model for Earthquake Data Processing — Yu et al. (2024) (arXiv:2408.01919, 2024)

## What this evaluates

Evaluates a pre-trained seismic model's multi-task capability on single-station waveforms, specifically phase picking (Pg, Sg, Pn, Sn), P-wave polarization classification, and seismic event type classification. The protocol tests generalization across temporal splits and transfer learning on local data to mitigate dataset imbalance.

## Datasets

- **CSNCD** — total 1300000; splits: train (-1), val (-1), test (-1); repo https://github.com/cangyeone/prime

## Metrics

- `recall` **(primary)** — range: [0, 1]
  - Fraction of correctly detected phase arrivals (Pg/Sg) out of all ground-truth arrivals within a specified time window.
- `accuracy` — range: [0, 1]
  - Fraction of correctly classified samples for P-polarity (up/down) and event type (7 classes) out of the total test samples.

## Input / output format

**Input**: Three-component seismic waveforms (Z, N, E) normalized by maximum value, shape R^{3x10240} (102.4s at 100Hz).

**Output**: Per time step: phase probability distribution over 5 classes. For polarity: binary classification over a 10.24s window centered on Pg. For event type: 7-class classification vector.

## Scoring recipe

```python
def compute_metrics(preds, golds):
    # Phase picking recall
    true_phases = set(golds['phase_times'])
    pred_phases = set(preds['phase_times'])
    recall = len(true_phases & pred_phases) / len(true_phases) if true_phases else 0
    
    # Polarity & Event accuracy
    correct = sum(1 for p, g in zip(preds['polarity'], golds['polarity']) if p == g)
    acc_polar = correct / len(golds['polarity'])
    
    correct = sum(1 for p, g in zip(preds['event_type'], golds['event_type']) if p == g)
    acc_event = correct / len(golds['event_type'])
    
    return recall, acc_polar, acc_event
```

## Common pitfalls

- Dataset is highly imbalanced (99.5% natural earthquakes), which can artificially inflate event classification accuracy without transfer learning or class weighting.
- P-polarity labels are only annotated for P-wave sections, not across all waveforms, requiring careful masking or windowing during evaluation to avoid penalizing the model on non-P-wave segments.

## Evidence (verbatim from paper)

> We have divided the CSNCD dataset into training, validation, and test datasets, assigning data from 2009 to 2019 to the training set, data from 2020 to the validation set, and data from 2021 and 2022 to the test set. The model achieves over 85% recall for Pg/Sg phases and 80% accuracy in P-polarity classification, while demonstrating strong transfer learning capability via fine-tuning—achieving 95.1% event classification accuracy without architectural changes.

## Citation

```bibtex
@misc{yu2024prime_dp,
  title={PRIME-DP: Pre-trained Integrated Model for Earthquake Data Processing},
  author={Yu et al. (2024)},
  year={2024},
  note={arXiv:2408.01919}
}
```

- arXiv: 2408.01919