Vega Hardware Bench

---
name: vega-hardware-bench
description: Measures the performance, energy efficiency, and latency of the Vega SoC on floating-point near-sensor analytic applications (NSAA) and deep neural network (DNN) inference workloads. Use when the user has predictions and gold and needs to compute Energy Efficiency.
metadata:
  skill_kind: metric
  source_arxiv: 2110.09101
  bibtex_key: rossi2021vega
  confidence: high
---

# vega-hardware-bench

> Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode — Rossi et al. (2021) (arXiv:2110.09101, 2021)

## What this evaluates

Measures the performance, energy efficiency, and latency of the Vega SoC on floating-point near-sensor analytic applications (NSAA) and deep neural network (DNN) inference workloads.

## Datasets

- **FP NSAA Kernels** — total ?; splits: test (-1)
- **MobileNetV2** — total ?; splits: test (-1)
- **RepVGG-A0/A1/A2** — total ?; splits: test (-1)

## Metrics

- `Energy Efficiency` **(primary)** — range: other
  - Total operations (MACs or FLOPs) divided by total energy consumption in joules, reported as GOPS/W or GFLOPS/W.
- `Latency` — range: other
  - Total execution time for a workload or inference pass, measured in milliseconds or microseconds.
- `Energy per Inference` — range: other
  - Total energy consumed during a single forward pass of a DNN, measured in millijoules (mJ).
- `ImageNet Top-1 Accuracy` — range: percent
  - Percentage of correctly classified images in the ImageNet validation set.

## Input / output format

**Input**: C/C++ source code for FP kernels; DNN model definitions (MobileNetV2, RepVGG) compiled to 8-bit integer tensors using PULP-NN library.

**Output**: Execution latency (ms/μs), energy consumption (mJ/pJ), throughput (MFLOPS/MACs), and classification accuracy (%).

## Scoring recipe

```python
# Calculate energy efficiency for DNN inference
total_macs = sum(layer_macs for layer in network)
total_energy_j = energy_compute + energy_mram_access + energy_hyperram_access
efficiency_gops_w = total_macs / (total_energy_j * soc_clock_hz)

# Calculate accuracy
correct_preds = sum(1 for p, g in zip(predictions, ground_truth) if p == g)
accuracy_pct = (correct_preds / len(ground_truth)) * 100
```

## Common pitfalls

- Confusing software-only execution with hardware accelerator (HWCE) results, which differ by ~3x speedup.
- Overlooking that MRAM energy savings only apply to weight storage, not activation memory.
- Assuming uniform compute-bound layers; early network layers are often memory-bound due to large activation sizes.

## Evidence (verbatim from paper)

> The results are presented for both HWCE-based and SW-based computation, using the same PULP-NN layers of the MobileNetV2 case study in the latter case. Almost all layers are compute-dominated, except for the final fully connected layer. In such conditions, HWCE-based execution delivers a 3x speedup over SW-based; and a 60-90% boost in system-level energy efficiency, depending on how much is the energy impact of HyperRAM traffic.

## Citation

```bibtex
@misc{rossi2021vega,
  title={Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode},
  author={Rossi et al. (2021)},
  year={2021},
  note={arXiv:2110.09101}
}
```

- arXiv: 2110.09101