Odin Mnist Eval

---
name: odin-mnist-eval
description: Evaluates the classification accuracy and energy efficiency of a 256-neuron spiking neuromorphic processor (ODIN) on the MNIST handwritten digit dataset. It compares offline gradient-based weight training against online spike-driven synaptic plasticity (SDSP) learning, while characterizing hardware power consumption and energy per spike operation. Use when the user wants to benchmark on MNIST, or asks about evaluating this task. Reports classification accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1804.07858
  bibtex_key: frenkel2018odin
  confidence: high
---

# odin-mnist-eval

> A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS — Frenkel et al. (2018) (arXiv:1804.07858, 2018)

## What this evaluates

Evaluates the classification accuracy and energy efficiency of a 256-neuron spiking neuromorphic processor (ODIN) on the MNIST handwritten digit dataset. It compares offline gradient-based weight training against online spike-driven synaptic plasticity (SDSP) learning, while characterizing hardware power consumption and energy per spike operation.

## Datasets

- **MNIST** — total ?; splits: train (-1), test (-1)

## Metrics

- `classification accuracy` **(primary)** — range: percent
  - Percentage of correctly classified MNIST digits by the 10-neuron SNN output.
- `energy per SOP (E_SOP)` — range: pJ
  - Incremental energy per spike operation calculated as (P - P_leak - P_idle * f_clk) / r_SOP, excluding static leakage and idle power.
- `global energy per SOP (E_tot,SOP)` — range: pJ
  - Total chip power divided by SOP rate (P / r_SOP), including leakage and idle power contributions.

## Input / output format

**Input**: 16x16 downsampled MNIST images converted to rate-based Poisson-distributed spike trains, fed into a single-layer fully-connected SNN of 10 LIF neurons.

**Output**: Spike trains from 10 output neurons, one per digit class. Classification determined by the neuron with the highest firing rate or spike count.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return (correct / len(gold)) * 100

def compute_energy_metrics(P, P_leak, P_idle, f_clk, r_SOP):
    E_SOP = (P - P_leak - P_idle * f_clk) / r_SOP
    E_tot_SOP = P / r_SOP
    return E_SOP, E_tot_SOP
```

## Common pitfalls

- Confusing incremental energy per SOP (E_SOP) with global energy per SOP (E_tot,SOP), which differ significantly due to leakage and idle power contributions.
- Assuming the MNIST accuracy represents state-of-the-art performance; the paper explicitly states the goal is to compare learning strategies, not to break accuracy records.
- Overlooking that spike coding uses rate-based Poisson trains during training, which may differ from inference coding schemes.

## Evidence (verbatim from paper)

> Benchmark for testing accuracy on image classification: pre-processing steps of the MNIST dataset of handwritten digits and the two considered setups for training the weights of a LIF-based 10-neuron spiking neural network implemented in the ODIN chip. (a) Off-chip offline weight training is carried out with quantization-aware stochastic gradient descent on a 10-neuron single-layer artificial neural network (ANN) with softmax units, implemented using Keras with a TensorFlow backend. The chosen optimizer and loss function are Adam with categorical cross-entropy. (b) On-chip online teacher-based weight training with the local SDSP learning rule.

## Citation

```bibtex
@misc{frenkel2018odin,
  title={A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS},
  author={Frenkel et al. (2018)},
  year={2018},
  note={arXiv:1804.07858}
}
```

- arXiv: 1804.07858