Few Shot Ts Gen Eval

---
name: few-shot-ts-gen-eval
description: Evaluates the ability of a generative model to produce high-fidelity time series data under extreme data scarcity (few-shot fine-tuning). It probes cross-domain generalization and robustness to varying sequence lengths and channel dimensions by comparing generated samples against real test data. Use when the user wants to benchmark on ECG200, ETTh2, ETTm1, ETTm2, ILI, Weather, Synthetic sine wave, or asks about evaluating this task. Reports contextFID (c-FID).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.20446
  bibtex_key: gonen2025timeseriesgeneration
  confidence: high
---

# few-shot-ts-gen-eval

> Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach — Gonen et al. (2025) (arXiv:2505.20446, 2025)

## What this evaluates

Evaluates the ability of a generative model to produce high-fidelity time series data under extreme data scarcity (few-shot fine-tuning). It probes cross-domain generalization and robustness to varying sequence lengths and channel dimensions by comparing generated samples against real test data.

## Datasets

- **ECG200** — total ?; splits: test (-1)
- **ETTh2** — total ?; splits: test (-1)
- **ETTm1** — total ?; splits: test (-1)
- **ETTm2** — total ?; splits: test (-1)
- **ILI** — total ?; splits: test (-1)
- **Weather** — total ?; splits: test (-1)
- **Synthetic sine wave** — total ?; splits: test (-1)

## Metrics

- `Discriminative Score (Disc.)` — range: other (lower is better)
  - Measures how well a discriminator distinguishes real time series from generated ones. Lower values indicate generated samples are more realistic.
- `Predictive Score (Pred.)` — range: other (lower is better)
  - Evaluates the predictive accuracy of a surrogate model trained on real data when applied to generated data. Lower values indicate better predictive fidelity.
- `contextFID (c-FID)` **(primary)** — range: other (lower is better)
  - Fréchet Inception Distance adapted for time series, measuring the distance between feature distributions of real and generated sequences. Lower values indicate higher sample quality and diversity.

## Input / output format

**Input**: Time series sequences with varying lengths and channel dimensions, provided as a few-shot fine-tuning subset (e.g., 5%, 10%, 15%, or fixed counts like #10, #25, #50).

**Output**: Generated time series sequences matching the input length and channel dimensions.

## Scoring recipe

```python
# For each dataset and few-shot subset size:
# 1. Fine-tune pre-trained model on the subset.
# 2. Generate N samples from the fine-tuned model.
# 3. Compute Discriminative Score (Disc.) between generated and real test set.
# 4. Compute Predictive Score (Pred.) between generated and real test set.
# 5. Compute contextFID (c-FID) between generated and real test set.
# Return average of Disc., Pred., c-FID across datasets/subsets. Lower is better.
```

## Common pitfalls

- All three metrics (Disc., Pred., c-FID) are lower-is-better, contrary to typical accuracy metrics.
- The benchmark uses both percentage-based (5%, 10%, 15%) and absolute count-based (#10, #25, #50) few-shot subset sizes, which must be tracked separately.
- Pre-training without dataset token conditioning leads to domain ambiguity and poor performance if fine-tuning is skipped.

## Evidence (verbatim from paper)

> We report the averaged Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID) in Tab. 1. Across all subset sizes and evaluation metrics, our method consistently outperforms the baselines, demonstrating strong performance in both percentage-based and count-based few-shot settings.

## Citation

```bibtex
@misc{gonen2025timeseriesgeneration,
  title={Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach},
  author={Gonen et al. (2025)},
  year={2025},
  note={arXiv:2505.20446}
}
```

- arXiv: 2505.20446