Uci Gp Benchmark Eval

---
name: uci-gp-benchmark-eval
description: Evaluates the time-accuracy trade-offs of approximate Gaussian Process regression methods against exact baselines and simple models across multiple UCI regression datasets. It measures how quickly approximations converge to near-exact performance while tracking predictive quality over time. Use when the user wants to benchmark on UCI regression datasets, or asks about evaluating this task. Reports NLPD.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2402.09849
  bibtex_key: ober2024recommendations
  confidence: high
---

# uci-gp-benchmark-eval

> Recommendations for Baselines and Benchmarking Approximate Gaussian Processes — Ober et al. (2024) (arXiv:2402.09849, 2024)

## What this evaluates

Evaluates the time-accuracy trade-offs of approximate Gaussian Process regression methods against exact baselines and simple models across multiple UCI regression datasets. It measures how quickly approximations converge to near-exact performance while tracking predictive quality over time.

## Datasets

- **UCI regression datasets** — total ?; splits: train (-1), test (-1)

## Metrics

- `NLPD` **(primary)** — range: other
  - Negative Log Predictive Density; measures the average negative log-likelihood of the true targets under the predictive distribution.
- `RMSE` — range: other
  - Root Mean Squared Error; computed as the square root of the mean squared difference between predicted and true targets.
- `LML/ELBO` — range: other
  - Log Marginal Likelihood or Evidence Lower Bound; measures the model's fit to the training data, with SGPR reporting upper bounds.

## Input / output format

**Input**: Feature vectors and target values from UCI regression datasets.

**Output**: Predicted means and variances per test point; evaluation reports NLPD, RMSE, and LML/ELBO at specified time checkpoints.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_mean, y_pred_var, model):
    rmse = np.sqrt(np.mean((y_true - y_pred_mean)**2))
    nlpd = -np.mean(0.5 * np.log(2 * np.pi * y_pred_var) + 0.5 * ((y_true - y_pred_mean)**2 / y_pred_var))
    lml = model.log_marginal_likelihood()
    return {'RMSE': rmse, 'NLPD': nlpd, 'LML': lml}
```

## Common pitfalls

- Reporting only final accuracy instead of time-accuracy trade-offs across multiple checkpoints, obscuring convergence behavior.
- Using default solver tolerances or preconditioner sizes for iterative methods, leading to convergence failures or erratic metric spikes.
- Omitting linear regression and constant function baselines, which masks whether the GP approximation actually captures signal.

## Evidence (verbatim from paper)

> We run each method over 5 different seeds, and additionally use exact GPR baselines, where possible. and additionally use linear regression, constant function, and, where possible, GPR baselines. We plot LML approximations (including upper bounds for SGPR), RMSEs, and NLPDs.

## Citation

```bibtex
@misc{ober2024recommendations,
  title={Recommendations for Baselines and Benchmarking Approximate Gaussian Processes},
  author={Ober et al. (2024)},
  year={2024},
  note={arXiv:2402.09849}
}
```

- arXiv: 2402.09849