Sst2 Imdb Eval

---
name: sst2-imdb-eval
description: Evaluates the binary sentiment classification capability of hybrid quantum-classical language models on both short and long text sequences. It probes whether adaptive quantum routing and attention mechanisms provide measurable accuracy gains over purely classical or purely quantum baselines on standard NLP benchmarks. Use when the user wants to benchmark on SST-2, IMDB, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.07026
  bibtex_key: pan2025bridging
  confidence: high
---

# sst2-imdb-eval

> Bridging Classical and Quantum Computing for Next-Generation Language Models — Pan et al. (2025) (arXiv:2508.07026, 2025)

## What this evaluates

Evaluates the binary sentiment classification capability of hybrid quantum-classical language models on both short and long text sequences. It probes whether adaptive quantum routing and attention mechanisms provide measurable accuracy gains over purely classical or purely quantum baselines on standard NLP benchmarks.

## Datasets

- **SST-2** — total ?; splits: test (-1)
- **IMDB** — total ?; splits: test (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly classified instances out of the total number of instances in the evaluation set.
- `Precision` — range: percent
  - Ratio of true positive predictions to the total number of positive predictions.
- `Recall` — range: percent
  - Ratio of true positive predictions to the total number of actual positive instances.
- `F1-Score` — range: percent
  - Harmonic mean of Precision and Recall.

## Input / output format

**Input**: Raw text sequences labeled with binary sentiment (positive/negative).

**Output**: Predicted sentiment class label (positive or negative).

## Scoring recipe

```python
def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Pure quantum models (e.g., VQC, Quantum LSTM) fail catastrophically on these tasks without classical integration, yielding near-random accuracy.
- Performance gains are highly dependent on sequence length and semantic complexity; quantum advantage scales exponentially with task difficulty rather than uniformly.
- Reported training/inference times are based on 20-qubit classical simulators, not real quantum hardware, which drastically underestimates future latency.

## Evidence (verbatim from paper)

> On SST-2, AQCF achieves 81.88% accuracy, outperforming the classical Transformer baseline by 2.52 percentage points and the standard Quantum Transformer by 2.29 percentage points. ... The performance gap widens on the more challenging IMDB dataset, where AQCF attains 86.30% accuracy—a 4.12 percentage point improvement over the classical baseline.

## Citation

```bibtex
@misc{pan2025bridging,
  title={Bridging Classical and Quantum Computing for Next-Generation Language Models},
  author={Pan et al. (2025)},
  year={2025},
  note={arXiv:2508.07026}
}
```

- arXiv: 2508.07026