Execution Time

---
name: execution_time
description: Measures the wall-clock execution time of four representative Fully Homomorphic Encryption (CKKS) workloads—bootstrapping, logistic regression training, RNN inference, and ResNet-20 inference—across different GPU architectures to evaluate library performance and memory constraints. Use when the user has predictions and gold and needs to compute execution_time.
metadata:
  skill_kind: metric
  source_arxiv: 2407.13055
  bibtex_key: kim2024cheddar
  confidence: high
---

# execution_time

> Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures — Kim et al. (2024) (arXiv:2407.13055, 2024)

## What this evaluates

Measures the wall-clock execution time of four representative Fully Homomorphic Encryption (CKKS) workloads—bootstrapping, logistic regression training, RNN inference, and ResNet-20 inference—across different GPU architectures to evaluate library performance and memory constraints.

## Datasets

- **Boot** — total ?; splits: (unstated)
- **HELR** — total ?; splits: (unstated)
- **RNN** — total ?; splits: (unstated)
- **ResNet** — total ?; splits: (unstated)

## Metrics

- `execution_time` **(primary)** — range: other
  - Wall-clock time measured in milliseconds (ms) for bootstrapping, milliseconds per iteration (ms/it) for logistic regression training, and seconds (s) for RNN and ResNet-20 inference.

## Input / output format

**Input**: Ciphertexts encrypting specific data structures (e.g., length-2^15 complex vectors, 1024-batch of 14×14 grayscale images, 32×128-long embeddings, single ResNet-20 image) under CKKS parameters with specified max L values.

**Output**: Execution time in ms or s per workload/iteration.

## Scoring recipe

```python
def compute_metric(workload, params):
    start = time.perf_counter()
    run_workload(workload, params)
    end = time.perf_counter()
    elapsed = end - start
    if workload == 'HELR':
        return elapsed / params['iterations'] * 1000  # ms/it
    elif workload == 'Boot':
        return elapsed * 1000  # ms
    else:
        return elapsed  # s
```

## Common pitfalls

- Out-of-memory (OoM) errors on GPUs with limited VRAM (e.g., V100, RTX 4090) prevent running larger workloads like RNN or ResNet.
- Execution times vary significantly based on GPU hardware specifications (e.g., A100 40GB vs 80GB DRAM bandwidth/capacity) and must be reported with exact hardware details.

## Evidence (verbatim from paper)

> We use execution time per iteration (ms/it) for comparison. Max L is 48. Table 4. Execution time of FHE CKKS workloads using Cheddar compared to prior acceleration studies.

## Citation

```bibtex
@misc{kim2024cheddar,
  title={Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures},
  author={Kim et al. (2024)},
  year={2024},
  note={arXiv:2407.13055}
}
```

- arXiv: 2407.13055