Tabular Icl Eval

---
name: tabular-icl-eval
description: This evaluation probes a model's ability to perform few-shot in-context learning and standard classification on high-dimensional, heterogeneous tabular data. It specifically measures how well biaxial attention and meta-learning improve performance across medical, financial, and energy domains, and how robust the model is to varying support set sizes and selection strategies. Use when the user wants to benchmark on TALENT, OpenML-CC18, or asks about evaluating this task. Reports accuracy (ACC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.00181
  bibtex_key: bouadi2025orionbix
  confidence: high
---

# tabular-icl-eval

> Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning — Mohamed Bouadi et al. (arXiv:2512.00181, 2025)

## What this evaluates

This evaluation probes a model's ability to perform few-shot in-context learning and standard classification on high-dimensional, heterogeneous tabular data. It specifically measures how well biaxial attention and meta-learning improve performance across medical, financial, and energy domains, and how robust the model is to varying support set sizes and selection strategies.

## Datasets

- **TALENT** — total ?; splits: train (-1), test (-1)
- **OpenML-CC18** — total ?; splits: train (-1), test (-1)

## Metrics

- `accuracy (ACC)` **(primary)** — range: [0, 1]
  - Fraction of correctly predicted class labels out of the total number of test instances.
- `class-weighted F1` — range: [0, 1]
  - Macro-averaged F1 score weighted by class support to account for dataset imbalance.
- `mean rank` — range: other
  - Average rank of a model across all datasets within a specific domain, ranked by accuracy.

## Input / output format

**Input**: Tabular feature matrix for a support set (k examples sampled from training, all classes represented) and a test set. For standard evaluation, the model receives the support set as in-context examples and the test set as queries.

**Output**: Predicted class label for each test instance.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    acc = np.mean(y_true == y_pred)
    f1 = f1_score(y_true, y_pred, average='weighted')
    return acc, f1

# For mean rank across datasets in a domain:
# ranks = [model_acc for model in models]
# mean_rank = np.mean(scipy.stats.rankdata(ranks, method='min'))
```

## Common pitfalls

- Mean rank is computed per domain (Medical, Finance, Energy) rather than globally across all datasets.
- Few-shot support sets must explicitly ensure all classes are represented when sampling k examples.
- Official train/test splits from the original benchmarks are used by default, overriding any custom 80/20 splits unless explicitly stated for support quality analysis.

## Evidence (verbatim from paper)

> For each dataset, we report overall classification accuracy (ACC), class-weighted F1 to account for imbalance, and mean rank across datasets within each domain based on accuracy. All models use official train/test splits unless noted.

## Citation

```bibtex
@misc{bouadi2025orionbix,
  title={Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning},
  author={Mohamed Bouadi et al.},
  year={2025},
  note={arXiv:2512.00181}
}
```

- arXiv: 2512.00181