Tsver Eval

---
name: tsver-eval
description: This benchmark evaluates an AI system's ability to perform fact verification using time-series evidence. It probes multi-timeframe temporal reasoning, cross-series numerical analysis, and the generation of factually consistent justifications aligned with human annotations. Use when the user wants to benchmark on TSVer, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.01101
  bibtex_key: strong2025tsver
  confidence: high
---

# tsver-eval

> TSVer: A Benchmark for Fact Verification Against Time-Series Evidence — Strong et al. (2025) (arXiv:2511.01101, 2025)

## What this evaluates

This benchmark evaluates an AI system's ability to perform fact verification using time-series evidence. It probes multi-timeframe temporal reasoning, cross-series numerical analysis, and the generation of factually consistent justifications aligned with human annotations.

## Datasets

- **TSVer** — total 287; splits: test (-1), development (-1); repo https://github.com/marekstrong/TSVer

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Proportion of correctly predicted verdicts (SUPPORTED or REFUTED) out of the total number of evaluation instances.
- `TSCS` — range: percent
  - TSCS = (1/N) Σ (F1_i * J_bar_i), where F1_i measures the dataset-level selection accuracy and J_bar_i is the average Jaccard Index over matched time ranges.
- `Ev²R` — range: percent
  - F1 score computed from precision and recall over atomic facts extracted from model-generated justifications compared against reference justifications.

## Input / output format

**Input**: A natural language claim paired with retrieved time-series data formatted as Markdown-style tables (or raw numerical records).

**Output**: A binary verdict (SUPPORTED or REFUTED), a textual justification explaining the verdict, and the set of retrieved time-series datasets with their corresponding time ranges.

## Scoring recipe

```python
def compute_tscs(predictions, gold):
    total = 0.0
    for pred, g in zip(predictions, gold):
        ds_f1 = f1_score(pred.datasets, g.datasets)
        jaccs = []
        for gt_ds in g.datasets:
            if gt_ds in pred.datasets:
                pred_r = pred.time_ranges[gt_ds]
                gt_r = g.time_ranges[gt_ds]
                jaccs.append(jaccard_index(pred_r, gt_r))
        avg_j = sum(jaccs) / len(jaccs) if jaccs else 0.0
        total += ds_f1 * avg_j
    return (total / len(predictions)) * 100
```

## Common pitfalls

- Models frequently over-retrieve time series or time spans, exceeding context windows and causing inference failures.
- Surface-level lexical metrics like METEOR fail to correlate with human judgments of factual adequacy in justifications.
- BPE tokenization of floating-point numbers yields inefficient representations, degrading numerical reasoning on continuous data.

## Evidence (verbatim from paper)

> We propose the Time Series Coverage Score (TSCS), a metric that jointly captures the accuracy of both time series selection and temporal coverage. TSCS combines a dataset-level F1 score with a temporal Jaccard Index to evaluate the quality of each retrieval instance.

## Citation

```bibtex
@misc{strong2025tsver,
  title={TSVer: A Benchmark for Fact Verification Against Time-Series Evidence},
  author={Strong et al. (2025)},
  year={2025},
  note={arXiv:2511.01101}
}
```

- arXiv: 2511.01101