Parma Performance Eval

---
name: parma-performance-eval
description: Measures the computational, network, and database performance overhead of running containerized workloads inside an AMD SEV-SNP enclave with Parma's attested execution policies compared to a baseline outside the enclave. Use when the user wants to benchmark on nginx (wrk2), redis (redis-benchmark), SPEC2017 intspeed, NVIDIA Triton Inference Server (perf-analyzer), or asks about evaluating this task. Reports performance_overhead.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2302.03976
  bibtex_key: johnson2023parma
  confidence: high
---

# parma-performance-eval

> Parma: Confidential Containers via Attested Execution Policies — Johnson et al. (2023) (arXiv:2302.03976, 2023)

## What this evaluates

Measures the computational, network, and database performance overhead of running containerized workloads inside an AMD SEV-SNP enclave with Parma's attested execution policies compared to a baseline outside the enclave.

## Datasets

- **nginx (wrk2)** — total ?; splits: (unstated)
- **redis (redis-benchmark)** — total ?; splits: (unstated)
- **SPEC2017 intspeed** — total ?; splits: (unstated)
- **NVIDIA Triton Inference Server (perf-analyzer)** — total ?; splits: (unstated)

## Metrics

- `performance_overhead` **(primary)** — range: percent
  - Calculated as the percentage reduction in throughput or increase in latency relative to the baseline configuration. For rate-based metrics: (Baseline_Rate - TEE_Rate) / Baseline_Rate * 100%. For latency/time: (TEE_Value - Baseline_Value) / Baseline_Value * 100%. Reported as mean or median overhead across benchmark runs.

## Input / output format

**Input**: Containerized workloads (web server, in-memory database, CPU benchmarks, ML inference server) deployed on a host machine configured with three setups: baseline (no TEE), SEV-SNP enclave, and SEV-SNP + Parma execution policy.

**Output**: Raw performance measurements (latency in ms, request/inference rate in req/s, SPEC base ratio) and computed overhead percentages per configuration.

## Scoring recipe

```python
def compute_overhead(baseline_val, tee_val, is_rate=True):
    if is_rate:
        return (baseline_val - tee_val) / baseline_val * 100
    else:
        return (tee_val - baseline_val) / baseline_val * 100
# Aggregate median/mean across trials per benchmark.
```

## Common pitfalls

- Overhead varies significantly by workload type (e.g., memory-intensive vs IO-intensive) due to TLB pressure and page table walk metadata checks in SEV-SNP.
- Statistical significance is often low; median latency curves frequently fall within the first quartiles of each other, requiring careful interpretation of aggregate results.
- Baseline configurations must be strictly matched (bare metal vs container vs enclave) to accurately isolate the TEE and policy overhead from containerization overhead.

## Evidence (verbatim from paper)

> Looking at the geometric mean over all actions, we see a performance overhead of 18% added by operating within the AMD SEV-SNP enclave, and a further 1% when using Parma.

## Citation

```bibtex
@misc{johnson2023parma,
  title={Parma: Confidential Containers via Attested Execution Policies},
  author={Johnson et al. (2023)},
  year={2023},
  note={arXiv:2302.03976}
}
```

- arXiv: 2302.03976