Tidmad Denoising Eval

---
name: tidmad-denoising-eval
description: Evaluates the ability of traditional and deep learning denoising algorithms to recover sinusoidal dark matter signals from ultra-long, noisy time series data collected by the ABRACADABRA experiment. Use when the user wants to benchmark on TIDMAD, or asks about evaluating this task. Reports mean square error.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2406.04378
  bibtex_key: fry2024tidmad
  confidence: medium
---

# tidmad-denoising-eval

> TIDMAD: Time Series Dataset for Discovering Dark Matter with AI Denoising — Fry et al. (2024) (arXiv:2406.04378, 2024)

## What this evaluates

Evaluates the ability of traditional and deep learning denoising algorithms to recover sinusoidal dark matter signals from ultra-long, noisy time series data collected by the ABRACADABRA experiment.

## Datasets

- **TIDMAD** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/jessicafry/TIDMAD

## Metrics

- `mean square error` **(primary)** — range: other
  - Average of the squared differences between predicted and ground truth values. (Note: The specific benchmarking metric used for final model comparison is referenced in Section 4 but not detailed in this section.)

## Input / output format

**Input**: Ultra-long time series segments (SQUID and injected signals), processed in frequency-split chunks for most models.

**Output**: Single floating point number per time step (regression) or 256-class classification decision per time step (mapping to 8-bit integers -128 to 127).

## Scoring recipe

```python
def compute_mse(predictions, gold):
    return np.mean((predictions - gold) ** 2)
# Note: Final benchmarking metric is detailed in Sec. 4 of the paper.
```

## Common pitfalls

- Memory constraints require segmenting ultra-long time series before model input.
- Frequency splitting is required for all models except WaveNet due to broad input frequency spectrum.
- Classification models use Focal Loss to handle class-imbalanced 256-class labels.

## Evidence (verbatim from paper)

> FC-Net outputs a single floating point number at each time step, and the training is conducted by minimizing the mean square error between this floating point number and corresponding ground truth time series at every sample.

## Citation

```bibtex
@misc{fry2024tidmad,
  title={TIDMAD: Time Series Dataset for Discovering Dark Matter with AI Denoising},
  author={Fry et al. (2024)},
  year={2024},
  note={arXiv:2406.04378}
}
```

- arXiv: 2406.04378