Adseq Delay Aware Autograd Snn

---
name: adseq-delay-aware-autograd-snn
description: "ADSEQ: delay-aware autograd-compatible framework for spike-event delivery in SNNs — memory-efficient autodifferentiable spike event queues with delay support, benchmarked across CPU/GPU/TPU/LPU platforms. arXiv:2512.05906v2"
tags: ["spiking-neural-networks", "autograd", "spike-delivery", "delay", "memory-efficient", "hardware-benchmark", "SNN-simulation", "queue-design"]
activation_words: [ADSEQ, spike event queue, delay-aware SNN, autograd SNN, spike delivery, ring buffer SNN, SNN accelerator benchmark, CPU GPU TPU LPU, selective spike dropping]
arxiv_id: "2512.05906"
---

# ADSEQ: Delay-Aware Autograd-Compatible Framework for Spike-Event Delivery in SNNs

## Paper Info
- **Title**: ADSEQ: A delay-aware autograd-compatible framework for spike-event delivery in SNNs
- **arXiv**: 2512.05906v2 (updated July 7, 2026)
- **Authors**: Lennart P. L. Landsmeer, Amirreza Movahedin, Said Hamdioui, Christos Strydis
- **Categories**: cs.NE

## Core Problem

Spiking Neural Networks (SNNs) require:
1. **Efficient simulation** — sparse spike events, not dense activations
2. **Gradient-based training** — backpropagation through spike times
3. **Spike delays** — biologically realistic and computationally important

**Gap**: Current SNN frameworks either:
- Use dense, memory-heavy data structures (defeating spike sparsity)
- Lack exact gradient methods with generality
- Omit or inefficiently handle delayed spikes

## Key Innovation

**Gradient computation through spike event queues** — including delays — implemented as memory-efficient, autodifferentiable data structures (ADSEQ).

### Queue Design Findings (Platform-Specific)

| Platform | Best Data Structure | Notes |
|----------|-------------------|-------|
| **CPU** | Tree-based or FIFO | Traditional implementations work well |
| **GPU** (small sim) | Ring buffers | Excel with contiguous memory access |
| **GPU** (high memory pressure) | Sparse data structures | Prefer sparsity under pressure |
| **TPU** | Sorting intrinsics | Leverage TPU's sorting hardware |
| **LPU** | (benchmarked) | Novel neuromorphic platform |

### Key Insight
> Queue design strongly shapes performance — there is no one-size-fits-all solution.

## Methodology

### 1. Spike Event Queue Formulation
- Represent spikes as events in a queue (not dense tensors)
- Each event: (timestamp, neuron_id, delay)
- Queue operations: insert, pop, delay propagation

### 2. Autodifferentiable Implementation
- Derive gradient computation through queue operations
- Make queue operations compatible with autograd (PyTorch/JAX)
- Handle delays explicitly in forward and backward passes

### 3. Memory Efficiency
- Sparse representation: only store actual spikes
- Avoid dense time × neuron matrices
- Scale to large networks without memory explosion

### 4. Selective Spike Dropping
- Trade-off: drop low-importance spikes for speed
- Provides simple performance-accuracy knob
- Future work: adaptive dropping based on gradient magnitude

## Design Principles

1. **Event-driven, not time-stepped** — process only when spikes occur
2. **Delay-aware** — synaptic and axonal delays are first-class citizens
3. **Platform-adaptive** — different data structures for different hardware
4. **Autograd-native** — gradients flow through queue operations naturally

## Applications

1. **Large-scale SNN simulation** — memory-efficient training of networks with millions of neurons
2. **Neuromorphic hardware compilation** — queue-based representation maps to neuromorphic cores
3. **Biologically realistic modeling** — delays are essential for realistic neural dynamics
4. **Multi-platform deployment** — same code runs on CPU/GPU/TPU/LPU with optimal data structures

## Implementation Notes

- Built on top of existing autograd frameworks (PyTorch-compatible)
- Queue operations are differentiable — gradients flow through insert/pop/delay
- Selective spike dropping is a runtime knob, not a training-time approximation
- Platform detection can auto-select optimal data structure

## Limitations

- Selective spike dropping is a heuristic — optimal dropping strategy unknown
- Platform-specific optimizations require separate implementations
- Delay handling adds complexity to gradient computation
- Not yet validated on very large-scale (>1M neuron) networks

## Related Work

- **SNN simulators**: BindsNET, Norse, SpikingJelly, Brian2
- **Event-driven simulation**: NEST, Neuron
- **Gradient methods**: surrogate gradients, e-prop, backprop through time
- **Hardware**: SpiNNaker, Loihi, TrueNorth

## Future Directions

The paper suggests:
> Future autograd frameworks could adapt diverging primal/tangent data-structures — using different representations for forward pass (speed) vs backward pass (gradient accuracy).

This is a profound insight: the forward simulation might benefit from one data structure (e.g., ring buffer for GPU), while the backward pass might need another (e.g., sparse for memory efficiency).

## Activation Keywords
ADSEQ, spike event queue, delay-aware SNN, autograd SNN, spike delivery, ring buffer SNN, SNN accelerator benchmark, CPU GPU TPU LPU, selective spike dropping, memory-efficient SNN, differentiable queue