Easytpp Eval

---
name: easytpp-eval
description: Evaluates neural Temporal Point Process models on event sequence prediction tasks, specifically forecasting the timing and categorical type of future events given historical sequences. Use when the user wants to benchmark on Retweet, Taxi, or asks about evaluating this task. Reports TIME RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2307.08097
  bibtex_key: xue2023easytpp
  confidence: high
---

# easytpp-eval

> EasyTPP: Towards Open Benchmarking Temporal Point Processes — Xue et al. (2023) (arXiv:2307.08097, 2023)

## What this evaluates

Evaluates neural Temporal Point Process models on event sequence prediction tasks, specifically forecasting the timing and categorical type of future events given historical sequences.

## Datasets

- **Retweet** — total ?; splits: test (-1)
- **Taxi** — total ?; splits: test (-1)

## Metrics

- `TIME RMSE` **(primary)** — range: other
  - Root Mean Squared Error between predicted and actual event timestamps. Lower values indicate better temporal prediction accuracy.
- `TYPE ERROR RATE` — range: percent
  - Fraction of incorrectly predicted event types compared to ground truth. Lower values indicate better categorical prediction accuracy.

## Input / output format

**Input**: Historical sequence of event timestamps and event types.

**Output**: Predicted timestamp and event type for the next event.

## Scoring recipe

```python
def compute_metrics(pred_times, gold_times, pred_types, gold_types):
    time_rmse = math.sqrt(mean((p - g)**2 for p, g in zip(pred_times, gold_times)))
    type_err = sum(1 for p, g in zip(pred_types, gold_types) if p != g) / len(gold_types)
    return {'TIME RMSE': time_rmse, 'TYPE ERROR RATE': type_err}
```

## Common pitfalls

- Early stopping is performed on the held-out dev set using log-likelihood, not the test set.
- Results are averaged over 5 independent runs to account for randomness.
- Models are evaluated on fixed horizon splits (e.g., AVG 5 EVENTS, AVG 10 EVENTS) rather than variable lengths.

## Evidence (verbatim from paper)

> METRICS (TIME RMSE / TYPE ERROR RATE)

## Citation

```bibtex
@misc{xue2023easytpp,
  title={EasyTPP: Towards Open Benchmarking Temporal Point Processes},
  author={Xue et al. (2023)},
  year={2023},
  note={arXiv:2307.08097}
}
```

- arXiv: 2307.08097