Browser Inference Eval

---
name: browser_inference_eval
description: Evaluates the performance overhead and latency characteristics of running deep learning inference directly in web browsers compared to native environments. It probes the impact of WebAssembly runtime inefficiencies, SIMD limitations, and WebGL GPU abstraction on prediction, warmup, and setup phases across various models and hardware configurations. Use when the user wants to benchmark on Inference Benchmark (ResNet50, VGG16, MobileNetV2), or asks about evaluating this task. Reports prediction_latency.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2402.05981
  bibtex_key: wang2024anatomizing
  confidence: high
---

# browser_inference_eval

> Anatomizing Deep Learning Inference in Web Browsers — Wang et al. (2024) (arXiv:2402.05981, 2024)

## What this evaluates

Evaluates the performance overhead and latency characteristics of running deep learning inference directly in web browsers compared to native environments. It probes the impact of WebAssembly runtime inefficiencies, SIMD limitations, and WebGL GPU abstraction on prediction, warmup, and setup phases across various models and hardware configurations.

## Datasets

- **Inference Benchmark (ResNet50, VGG16, MobileNetV2)** — total ?; splits: (unstated)

## Metrics

- `prediction_latency` **(primary)** — range: ms
  - Mean execution time in milliseconds for the model inference step, measured after framework and model loading. Calculated by averaging prediction times across multiple runs per device/backend configuration.

## Input / output format

**Input**: Pre-trained deep learning models (e.g., ResNet50, VGG16, MobileNetV2) executed via TF.js or ORT.js backends (Wasm/CPU or WebGL/GPU) within a web browser, compared against native TF or ORT implementations.

**Output**: Latency values in milliseconds for prediction, warmup, and setup phases, reported as averages and quartiles/variance across different devices and backend configurations.

## Scoring recipe

```python
def measure_latency(model, backend, device):
    setup_start = time()
    load_framework_and_model(model, backend)
    setup_latency = time() - setup_start
    warmup_start = time()
    run_inference(model, backend, steps=1)
    warmup_latency = time() - warmup_start
    pred_latencies = []
    for _ in range(num_runs):
        pred_start = time()
        run_inference(model, backend)
        pred_latencies.append(time() - pred_start)
    prediction_latency = mean(pred_latencies)
    return {'setup': setup_latency, 'warmup': warmup_latency, 'prediction': prediction_latency}
```

## Common pitfalls

- Confusing warmup latency (shader compilation/memory allocation) with prediction latency, which can skew performance comparisons.
- Ignoring network transfer time when measuring setup latency for in-browser inference, as model files must be downloaded to the browser.
- Overlooking GPU resource contention in WebGL, which causes high latency variance and degrades smoothness despite lower average prediction times.

## Evidence (verbatim from paper)

> As for prediction latency, the in-browser prediction latency of TF.js is 3.7-18.4× higher than the latency of TF (Table[5] vs. [4]); the gap is 2.1-36.4× between ORT.js and ORT (Table[6] vs. [4]). The average gap of both frameworks is 16.9×.

## Citation

```bibtex
@misc{wang2024anatomizing,
  title={Anatomizing Deep Learning Inference in Web Browsers},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2402.05981}
}
```

- arXiv: 2402.05981