Bishep Tabular Eval

---
name: bishep-tabular-eval
description: Evaluates the performance of the BiSHop model on tabular classification and regression tasks, probing its ability to handle mixed feature types, bi-directional cellular learning, and generalized sparse modern Hopfield layers. Use when the user wants to benchmark on Tabular Benchmarks (Adult, Bank, Blastchar, Income, SeismicBump, Shrutime, Spambase, Qsar, Jannis, CR), or asks about evaluating this task. Reports AUC (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2404.03830
  bibtex_key: xu2024bishep
  confidence: high
---

# bishep-tabular-eval

> BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model — Chenwei Xu et al. (arXiv:2404.03830, 2024)

## What this evaluates

Evaluates the performance of the BiSHop model on tabular classification and regression tasks, probing its ability to handle mixed feature types, bi-directional cellular learning, and generalized sparse modern Hopfield layers.

## Datasets

- **Tabular Benchmarks (Adult, Bank, Blastchar, Income, SeismicBump, Shrutime, Spambase, Qsar, Jannis, CR)** — total ?; splits: test (-1)

## Metrics

- `AUC (%)` **(primary)** — range: percent
  - Area under the Receiver Operating Characteristic curve, reported as a percentage.
- `R2` — range: other
  - Coefficient of determination for regression tasks, measuring the proportion of variance in the dependent variable predictable from the independent variables.

## Input / output format

**Input**: Tabular dataset rows containing mixed categorical and numerical features.

**Output**: Predicted class probabilities or scores for classification; continuous target values for regression.

## Scoring recipe

```python
def compute_auc(y_true, y_pred):
    fpr, tpr, _ = roc_curve(y_true, y_pred)
    return auc(fpr, tpr) * 100

def compute_r2(y_true, y_pred):
    ss_res = np.sum((y_true - y_pred) ** 2)
    ss_tot = np.sum((y_true - np.mean(y_true)) ** 2)
    return 1 - (ss_res / ss_tot)
```

## Common pitfalls

- Averaging AUC scores across all datasets masks per-dataset performance variations.
- Data rotation experiments show that BiSHop's performance drops significantly when both directions and datasets are rotated, highlighting sensitivity to orientation.
- Ablation studies must use default hyperparameters for remaining components to ensure fair comparison.

## Evidence (verbatim from paper)

> We report the average AUC score (in %) over 10 runs in Table 18. The results indicate the superior performance of our proposed generalized sparse modern Hopfield model across datasets.

## Citation

```bibtex
@misc{xu2024bishep,
  title={BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model},
  author={Chenwei Xu et al.},
  year={2024},
  note={arXiv:2404.03830}
}
```

- arXiv: 2404.03830