Secure Inference Latency

---
name: secure-inference-latency
description: Measures the online and offline computation latency and communication bandwidth for cryptographic primitives and neural network operations under secure two-party computation. It evaluates how efficiently packed homomorphic encryption and garbled circuits handle matrix-vector products, convolutions, and activation functions without revealing inputs or model parameters. Use when the user has predictions and gold and needs to compute t_online.
metadata:
  skill_kind: metric
  source_arxiv: 1801.05507
  bibtex_key: juvekar2018gazelle
  confidence: high
---

# secure-inference-latency

> Gazelle: A Low Latency Framework for Secure Neural Network Inference — Juvekar et al. (2018) (arXiv:1801.05507, 2018)

## What this evaluates

Measures the online and offline computation latency and communication bandwidth for cryptographic primitives and neural network operations under secure two-party computation. It evaluates how efficiently packed homomorphic encryption and garbled circuits handle matrix-vector products, convolutions, and activation functions without revealing inputs or model parameters.

## Datasets

- (no dataset; pure metric skill)

## Metrics

- `t_online` **(primary)** — range: milliseconds
  - Wall-clock time in milliseconds for the online phase of the secure computation protocol, which runs per inference and depends on client input.
- `t_setup` — range: milliseconds
  - Wall-clock time in milliseconds for the one-time offline setup phase (e.g., key generation, matrix preprocessing) that is amortized across multiple inferences.
- `BW_online` — range: megabytes
  - Communication bandwidth in megabytes transferred between the client and server during the online phase.
- `BW_offline` — range: megabytes
  - Communication bandwidth in megabytes transferred during the offline setup phase.

## Input / output format

**Input**: Plaintext matrices/vectors, encrypted ciphertexts, and neural network layer configurations (e.g., filter size, stride, window size, number of SIMD slots).

**Output**: Execution time (ms) and communication bandwidth (MB) for online and offline phases, along with operation counts (e.g., #in_rot, #out_rot, #mac) and speedup ratios compared to baseline implementations.

## Scoring recipe

```python
import time

def evaluate_gazelle(operation_func, num_runs=10):
    start_setup = time.time()
    operation_func.setup()
    t_setup = (time.time() - start_setup) * 1000
    
    start_online = time.time()
    operation_func.run_online()
    t_online = (time.time() - start_online) * 1000
    
    return {
        't_setup_ms': t_setup,
        't_online_ms': t_online,
        'BW_offline_MB': setup_bytes / (1024*1024),
        'BW_online_MB': online_bytes / (1024*1024)
    }
```

## Common pitfalls

- Offline setup time is a one-time cost and should not be averaged or reported as per-inference latency.
- Experiments are conducted in a LAN setting; results will not reflect real-world wide-area network latency or packet loss.
- Ciphertext size is fixed at 32 kB per parameter for the reported bandwidth calculations, so bandwidth scales linearly with the number of packed slots.

## Evidence (verbatim from paper)

> All benchmarks were generated using c4.xlarge AWS instances which provide a 4-threaded execution environment (on an Intel Xeon E5-2666 v3 2.90GHz CPU) with 7.5GB of system memory. Our experiments were conducted using Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-1041-aws) and our library was compiled using GCC 5.4.0 using the ’-O3’ optimization setting and enabling support for the AES-NI instruction set. Our schemes are evaluated in the LAN setting similar to previous work with both instances in the us-east-1a availability zone. Table [VIII] shows the online computation time and the time required to setup the scheme in milliseconds. | Algorithm | Outputs | t_offline | t_online | BW_offline | BW_online |

## Citation

```bibtex
@misc{juvekar2018gazelle,
  title={Gazelle: A Low Latency Framework for Secure Neural Network Inference},
  author={Juvekar et al. (2018)},
  year={2018},
  note={arXiv:1801.05507}
}
```

- arXiv: 1801.05507