Grinsztajn45 Eval

---
name: grinsztajn45-eval
description: Evaluates the predictive performance of deep neural networks on tabular data across classification and regression tasks, comparing them against tree-based models and other DNNs. It probes how well architectures handle numerical-only versus heterogeneous (numerical + categorical) features at different dataset scales. Use when the user wants to benchmark on Grinsztajn45, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.18446
  bibtex_key: chen2023trompt
  confidence: high
---

# grinsztajn45-eval

> Trompt: Towards a Better Deep Neural Network for Tabular Data — Chen et al. (2023) (arXiv:2305.18446, 2023)

## What this evaluates

Evaluates the predictive performance of deep neural networks on tabular data across classification and regression tasks, comparing them against tree-based models and other DNNs. It probes how well architectures handle numerical-only versus heterogeneous (numerical + categorical) features at different dataset scales.

## Datasets

- **Grinsztajn45** — total ?; splits: train (-1), test (-1); repo https://github.com/LeoGrin/tabular-benchmark

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard classification accuracy: fraction of correctly predicted class labels out of total test samples.
- `r2-score` — range: [0, 1]
  - Standard coefficient of determination for regression: 1 - (sum of squared residuals / total sum of squares).

## Input / output format

**Input**: Tabular dataset rows containing numerical and/or categorical features, normalized according to the Grinsztajn45 benchmark protocol.

**Output**: Predicted class labels for classification tasks, or continuous values for regression tasks.

## Scoring recipe

```python
if task == 'classification':
    acc = sum(pred == gold) / len(gold)
elif task == 'regression':
    ss_res = sum((gold - pred) ** 2)
    ss_tot = sum((gold - mean(gold)) ** 2)
    r2 = 1 - (ss_res / ss_tot)
return acc if classification else r2
```

## Common pitfalls

- The original Grinsztajn45 benchmark omitted some baseline models due to incomplete results; this paper adds two tree-based models for comparison, which may skew direct head-to-head baselines.
- Trompt uses a smaller hyperparameter search space than competing models, meaning performance gaps may partly reflect search budget rather than pure architectural superiority.
- Results are aggregated and reported by dataset size (medium/large) and feature type (numerical only vs. heterogeneous), so reporting raw per-dataset scores without this grouping will not match the paper's figures.

## Evidence (verbatim from paper)

> The performance and ablation study of Trompt primarily focus on the Grinsztajn45 benchmark (Grinsztajn et al., 2022). This benchmark comprises datasets from various domains and follows a unified methodology for evaluating different models, providing a fair and comprehensive assessment. The evaluation metrics are accuracy and r2-score for classification and regression tasks, respectively.

## Citation

```bibtex
@misc{chen2023trompt,
  title={Trompt: Towards a Better Deep Neural Network for Tabular Data},
  author={Chen et al. (2023)},
  year={2023},
  note={arXiv:2305.18446}
}
```

- arXiv: 2305.18446