Ecosched Hpc Scheduling Eval

---
name: ecosched-hpc-scheduling-eval
description: Evaluates an online co-scheduling framework's ability to jointly optimize GPU count selection and job packing to minimize energy consumption while maintaining performance across diverse multi-GPU HPC workloads. It probes the scheduler's capacity to handle non-linear scaling, NUMA-aware placement, and dynamic workload packing without prior knowledge of exact runtimes. Use when the user wants to benchmark on Multi-GPU Benchmark Suite, or asks about evaluating this task. Reports Energy Saving.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.17640
  bibtex_key: zheng2026ecosched
  confidence: high
---

# ecosched-hpc-scheduling-eval

> Towards Energy Efficient Co-Scheduling in HPC — Zheng et al. (2026) (arXiv:2604.17640, 2026)

## What this evaluates

Evaluates an online co-scheduling framework's ability to jointly optimize GPU count selection and job packing to minimize energy consumption while maintaining performance across diverse multi-GPU HPC workloads. It probes the scheduler's capacity to handle non-linear scaling, NUMA-aware placement, and dynamic workload packing without prior knowledge of exact runtimes.

## Datasets

- **Multi-GPU Benchmark Suite** — total 17; splits: test (17)

## Metrics

- `Energy Saving` **(primary)** — range: percent
  - Percentage reduction in total energy relative to a baseline scheduler. Calculated as (E_baseline - E_method) / E_baseline.
- `Makespan Improvement` — range: percent
  - Percentage reduction in total completion time relative to a baseline. Calculated as (T_baseline - T_method) / T_baseline.
- `EDP Saving` — range: percent
  - Percentage reduction in end-to-end Energy-Delay Product (EDP = Energy × Makespan) relative to a baseline. Calculated as (EDP_baseline - EDP_method) / EDP_baseline.
- `Performance Loss` — range: percent
  - Percentage increase in runtime relative to solo execution with the performance-optimal GPU count. Calculated as (T_method - T_optimal_solo) / T_optimal_solo.

## Input / output format

**Input**: A finite window of multi-GPU jobs specifying application type, required GPU count, and NUMA constraints, along with system topology (CPU/GPU count, NUMA nodes).

**Output**: A schedule assigning each job a GPU count, NUMA node placement, and execution start time.

## Scoring recipe

```python
def score(gold, predictions):
    energy_saving = (gold['baseline_energy'] - predictions['method_energy']) / gold['baseline_energy']
    makespan_improvement = (gold['baseline_makespan'] - predictions['method_makespan']) / gold['baseline_makespan']
    edp_saving = (gold['baseline_edp'] - predictions['method_edp']) / gold['baseline_edp']
    performance_loss = (predictions['method_runtime'] - gold['baseline_runtime']) / gold['baseline_runtime']
    return energy_saving, makespan_improvement, edp_saving, performance_loss
```

## Common pitfalls

- Confusing the offline Oracle baseline (which assumes perfect knowledge of runtime and power profiles) with a realistic online scheduler, leading to overly optimistic performance gap expectations.
- Evaluating only on a single workload type instead of the full 17-workload suite, which masks cross-job interference and non-linear scaling effects.
- Ignoring NUMA-aware placement constraints when measuring energy or performance, which inflates baseline metrics and invalidates the scheduling comparison.

## Evidence (verbatim from paper)

> EcoSched’s objective is to minimize total energy to completion of all workloads while meeting the performance loss bound, hence we use the following metrics: Energy Saving: percentage reduction in total energy relative to a baseline. Makespan Improvement: the percentage reduction in total completion time. EDP Saving: the percentage reduction in end-to-end EDP. Performance Loss: percentage increase in runtime relative to solo execution with the performance-optimal GPU count.

## Citation

```bibtex
@misc{zheng2026ecosched,
  title={Towards Energy Efficient Co-Scheduling in HPC},
  author={Zheng et al. (2026)},
  year={2026},
  note={arXiv:2604.17640}
}
```

- arXiv: 2604.17640