Bit Flip Resilience Eval

---
name: bit-flip-resilience-eval
description: Evaluates the robustness of neural network architectures (MLPs, CNNs, and Differentiable Weightless Networks) to parameter bit-flips under varying corruption rates. It measures how task accuracy degrades as a function of bit error rate (BER) and isolates the impact of architectural hyperparameters like precision, width, depth, activation functions, and sparsity. Use when the user wants to benchmark on MLPerf Tiny, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.22770
  bibtex_key: bacellar2026bitflip
  confidence: high
---

# bit-flip-resilience-eval

> From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips — Bacellar et al. (2026) (arXiv:2603.22770, 2026)

## What this evaluates

Evaluates the robustness of neural network architectures (MLPs, CNNs, and Differentiable Weightless Networks) to parameter bit-flips under varying corruption rates. It measures how task accuracy degrades as a function of bit error rate (BER) and isolates the impact of architectural hyperparameters like precision, width, depth, activation functions, and sparsity.

## Datasets

- **MLPerf Tiny** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - The proportion of correctly predicted labels or anomaly classifications out of the total number of samples. Computed as correct predictions divided by total samples for each corruption rate.

## Input / output format

**Input**: Standard task inputs (images, audio clips, or tabular data) passed through a neural network whose parameters (weights or LUT entries) have been corrupted by independent Bernoulli bit-flips with probability p.

**Output**: Predicted class labels or anomaly scores.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

# Evaluated across a spectrum of BERs (p) from 10^-8 to 1.0
# Results are averaged across datasets and Monte Carlo trials (100 per point)
```

## Common pitfalls

- Assuming pure integer quantization was tested; the paper explicitly notes it was excluded due to insufficient baseline accuracy, and only layer-wise affine quantization is reported.
- Confusing parameter bit-flips with activation or input corruption; faults are injected exclusively into network parameters (weights/LUTs).
- Expecting accuracy to drop to zero; models collapse to random-guess baselines (e.g., ~10-20% depending on class count) rather than absolute zero.

## Evidence (verbatim from paper)

> Figure 2 illustrates the degradation of accuracy as the bit error rate increases across FP32, FP16, FP8, INT8, INT4, INT2, and BNN formats. For clarity, all INT results reported here use layer-wise affine quantization, not pure integer quantization. Pure integer quantization was not included because, while it is theoretically more resilient to bit flips, it did not provide sufficient baseline task accuracy in our experiments.

## Citation

```bibtex
@misc{bacellar2026bitflip,
  title={From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips},
  author={Bacellar et al. (2026)},
  year={2026},
  note={arXiv:2603.22770}
}
```

- arXiv: 2603.22770