Etide Eval

---
name: etide-eval
description: Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes. Use when the user wants to benchmark on ETram, E-3DTrack, or asks about evaluating this task. Reports aIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.27757
  bibtex_key: sen2026etide
  confidence: high
---

# etide-eval

> E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences — Biswadeep Sen et al. (arXiv:2603.27757, 2026)

## What this evaluates

Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes.

## Datasets

- **ETram** — total ?; splits: train (-1), val (-1), test (-1)
- **E-3DTrack** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `MSE` — range: other
  - Mean Squared Error between predicted logits and binary ground-truth occurrence maps. Lower is better.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index measuring luminance, contrast, and structure similarity between predicted and ground-truth frames. Higher is better.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity computed using a pretrained feature network to measure perceptual distance. Lower is better.
- `mIoU` — range: [0, 1]
  - Mean Intersection over Union averaged across the ON and OFF polarity channels. Computed after binarizing logits with Otsu's threshold. Higher is better.
- `aIoU` **(primary)** — range: [0, 1]
  - Intersection over Union on the polarity-agnostic occupancy mask formed by OR-ing the ON and OFF channels. Computed after binarizing logits with Otsu's threshold. Higher is better.

## Input / output format

**Input**: Sequence of $T_{in}=10$ frames, each represented as a two-channel binary occurrence map (ON/OFF) at 30 Hz. Spatial resolution is 128×128 (ETram uses random 512×512 activity crops downsampled; E-3DTrack uses full frame downsampled).

**Output**: Sequence of $T_{out}=10$ predicted frames as logits, which are sigmoided and binarized per-frame using Otsu's thresholding to yield binary ON/OFF event maps.

## Scoring recipe

```python
def score(predictions, ground_truth):
    # predictions: logits (T, 2, H, W), ground_truth: binary maps (T, 2, H, W)
    mse = np.mean((predictions - ground_truth)**2)
    ssim = compute_ssim(predictions, ground_truth)
    lpips = compute_lpips(predictions, ground_truth)
    
    all_on_ious, all_off_ious, all_a_ious = [], [], []
    for t in range(T):
        thresh = otsu_maximize_variance(predictions[t])
        pred_bin = (predictions[t] > thresh).astype(int)
        gt_bin = ground_truth[t]
        
        on_iou = jaccard_index(pred_bin[0], gt_bin[0])
        off_iou = jaccard_index(pred_bin[1], gt_bin[1])
        a_iou = jaccard_index(pred_bin[0] | pred_bin[1], gt_bin[0] | gt_bin[1])
        
        all_on_ious.append(on_iou)
        all_off_ious.append(off_iou)
        all_a_ious.append(a_iou)
        
    mIoU = np.mean(all_on_ious + all_off_ious)
    aIoU = np.mean(all_a_ious)
    return mse, ssim, lpips, mIoU, aIoU
```

## Common pitfalls

- Thresholding protocol: The paper uses automatic Otsu thresholding per frame to binarize logits before computing IoU, rather than a fixed global threshold. Using a fixed threshold will yield different rankings.
- Input preprocessing mismatch: ETram requires random cropping of 512×512 activity regions before downsampling to 128×128, whereas E-3DTrack uses the full frame. Failing to replicate this exact preprocessing pipeline breaks comparability.

## Evidence (verbatim from paper)

> We evaluate prediction quality using pixel-level fidelity metrics (MSE↓, SSIM↑, LPIPS↓), as well as overlap metrics (mIoU (mean ON/OFF IoU), and aIoU (IoU on the polarity-agnostic occupancy mask formed by OR-ing both channels)). These overlap metrics also indicate downstream usability, as they reflect whether predicted event structure is suitable for segmentation-style evaluation. We apply a sigmoid to the predicted logits and binarize with a threshold τ, then compute IoU separately for ON and OFF channels against ground-truth occurrence maps. We report mIoU (mean of ON/OFF IoU) and aIoU. To avoid manual tuning, τ is selected automatically per frame via a thresholding rule that maximizes inter-class variance, and overlap metrics are accumulated globally over the test set.

## Citation

```bibtex
@misc{sen2026etide,
  title={E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences},
  author={Biswadeep Sen et al.},
  year={2026},
  note={arXiv:2603.27757}
}
```

- arXiv: 2603.27757