Blas Performance Benchmark

---
name: blas-performance-benchmark
description: Evaluates the runtime performance and throughput of a C++ expression template library (SALT) against optimized BLAS implementations (Intel MKL) and other template libraries (Eigen) for standard vector operations. Use when the user has predictions and gold and needs to compute performance (GFLOPS).
metadata:
  skill_kind: metric
  source_arxiv: 1109.1264
  bibtex_key: progsch2011salt
  confidence: high
---

# blas-performance-benchmark

> A New Vectorization Technique for Expression Templates in C++ — Progsch et al. (2011) (arXiv:1109.1264, 2011)

## What this evaluates

Evaluates the runtime performance and throughput of a C++ expression template library (SALT) against optimized BLAS implementations (Intel MKL) and other template libraries (Eigen) for standard vector operations.

## Datasets

- **Synthetic Vector Operations Benchmark** — total ?; splits: test (-1)

## Metrics

- `performance (GFLOPS)` **(primary)** — range: GFLOPS
  - Measured in Giga-Floating Point Operations per Second. Calculated as (2*N)/time for dot product and AXPY, and N/time for scaling. Normalized against theoretical CPU peak.

## Input / output format

**Input**: Single-precision floating-point vectors of varying length N, and scalar alpha for AXPY.

**Output**: Throughput in GFLOPS for each vector size and operation type.

## Scoring recipe

```python
def compute_gflops(operation, N, time_seconds):
    if operation in ['dot', 'axpy']:
        ops = 2 * N
    elif operation == 'scal':
        ops = N
    else:
        ops = 0
    return (ops / time_seconds) / 1e9
```

## Common pitfalls

- Results are highly sensitive to CPU cache hierarchy and memory bandwidth, which differ across hardware generations.
- Compiler flags (-O3 -msse3) and specific versions drastically affect auto-vectorization and loop unrolling, making cross-version comparisons difficult.
- Out-of-place operations require separate copy and scale steps in BLAS, inflating their baseline time compared to expression templates that fuse them.

## Evidence (verbatim from paper)

> Measurements were taken for three common vector operations. These are the dot product (xDOT), vector scaling (xSCAL) and operations of the form y=y+alpha x (xAXPY). The results were compared to Intel’s MKL library (BLAS interface) and to Eigen (expression templates). The results for a Intel Core i5-580M CPU (2.66 GHz, 3.33 GHz TurboBoost) are shown in figures [3], [4] and [5]. The grey lines indicate the sizes of the different caches and the 17.1 GB/s memory bandwidth and 13.3 Gflop/s theoretical peak performance of the used processor.

## Citation

```bibtex
@misc{progsch2011salt,
  title={A New Vectorization Technique for Expression Templates in C++},
  author={Progsch et al. (2011)},
  year={2011},
  note={arXiv:1109.1264}
}
```

- arXiv: 1109.1264