Gpu Memory Co Optimization Eval

---
name: gpu-memory-co-optimization-eval
description: Evaluates the trade-off between system memory footprint and task latency when co-executing multiple workloads under different integrated CPU/GPU memory management policies on embedded platforms. It measures how strategically assigning Device, Managed, or Host-Pinned memory policies affects peak memory consumption, average GPU execution time, and overall GPU utilization during multitasking. Use when the user wants to benchmark on Rodinia Benchmark Suite (subset), DJI Drone Object Detection, Autoware Perception Module, or asks about evaluating this task. Reports GPU time.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2003.07945
  bibtex_key: bateni2020gpumemory
  confidence: high
---

# gpu-memory-co-optimization-eval

> Co-Optimizing Performance and Memory FootprintVia Integrated CPU/GPU Memory Management, anImplementation on Autonomous Driving Platform — Bateni et al. (2020) (arXiv:2003.07945, 2020)

## What this evaluates

Evaluates the trade-off between system memory footprint and task latency when co-executing multiple workloads under different integrated CPU/GPU memory management policies on embedded platforms. It measures how strategically assigning Device, Managed, or Host-Pinned memory policies affects peak memory consumption, average GPU execution time, and overall GPU utilization during multitasking.

## Datasets

- **Rodinia Benchmark Suite (subset)** — total ?; splits: test (7)
- **DJI Drone Object Detection** — total ?; splits: test (1)
- **Autoware Perception Module** — total ?; splits: test (1)

## Metrics

- `memory usage` — range: other
  - Peak GPU memory consumption measured during concurrent execution of workloads under a given MM policy.
- `GPU time` **(primary)** — range: other
  - Average execution time across multiple input data sizes for each benchmark or application.
- `GPU utilization` — range: percent
  - Percentage of GPU compute capacity actively used during multitasking workloads.
- `prediction error` — range: percent
  - Geometric mean of the absolute percentage difference between measured and model-predicted GPU times.

## Input / output format

**Input**: Workloads (benchmarks or real applications) with specific input data sizes, scheduled concurrently on embedded SoCs (TX2, AGX, PX2) under three GPU memory management configurations: Default (D), Memory Optimized (MO), and Co-Optimized (CO).

**Output**: Measured peak memory usage, average GPU execution time, GPU utilization percentage, and model prediction error rates.

## Scoring recipe

```python
def evaluate(workloads, config, hardware):
    results = []
    for wl in workloads:
        mem = measure_peak_memory(wl, config)
        time = measure_avg_gpu_time(wl, config)
        util = measure_gpu_utilization(wl, config)
        results.append({'memory': mem, 'time': time, 'util': util})
    errors = []
    for r in results:
        pred = predict_time(r['input_size'], config)
        errors.append(abs(r['time'] - pred) / r['time'])
    error_rate = geometric_mean(errors)
    return results, error_rate
```

## Common pitfalls

- Confusing the three GPU MM policies (Default/Device, Managed, Host-Pinned) and their distinct memory/latency trade-offs.
- Assuming the Memory Optimized (MO) configuration always yields the lowest memory footprint in multitasking, ignoring that the Co-Optimized (CO) policy strategically mixes policies to balance memory and latency.
- Overlooking hardware architecture differences (Parker/TX2 vs. Xavier/AGX), which significantly alter policy selection and performance due to varying memory access and transfer speeds.

## Evidence (verbatim from paper)

> We report the results of the memory usage and GPU time under three GPU MM setups (D, MO, CO) and two hardware configurations (TX2 and AGX). ... in worst case, the geometric mean of the error is 14.9% for D policy, 16.4% for M policy and 23.1 % for H policy; in average case, the error is 9.6 % for D, 11.8% for M and 6.5% for H policy.

## Citation

```bibtex
@misc{bateni2020gpumemory,
  title={Co-Optimizing Performance and Memory FootprintVia Integrated CPU/GPU Memory Management, anImplementation on Autonomous Driving Platform},
  author={Bateni et al. (2020)},
  year={2020},
  note={arXiv:2003.07945}
}
```

- arXiv: 2003.07945