Edge Llm Energy Accuracy Eval

---
name: edge-llm-energy-accuracy-eval
description: Evaluates the trade-offs between quantization levels, model architectures, and task characteristics on energy efficiency and output accuracy for LLMs deployed on edge hardware. Use when the user wants to benchmark on bigbenchhard, commonsenseqa, gsm8k, humaneval, truthfulqa, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.03360
  bibtex_key: husom2025sustainable
  confidence: high
---

# edge-llm-energy-accuracy-eval

> Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency — Husom et al. (2025) (arXiv:2504.03360, 2025)

## What this evaluates

Evaluates the trade-offs between quantization levels, model architectures, and task characteristics on energy efficiency and output accuracy for LLMs deployed on edge hardware.

## Datasets

- **bigbenchhard** — total ?; splits: test (-1)
- **commonsenseqa** — total ?; splits: test (-1)
- **gsm8k** — total ?; splits: test (-1)
- **humaneval** — total ?; splits: test (-1)
- **truthfulqa** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard dataset accuracy calculated as the proportion of correctly predicted instances out of the total number of instances.
- `energy_per_token` — range: Joules
  - Total hardware energy consumption in Joules divided by the number of tokens generated in the response.

## Input / output format

**Input**: Text prompt with task-specific instructions (e.g., math problems, code generation, multiple-choice questions) fed to the LLM.

**Output**: Model-generated text response (variable length in tokens).

## Scoring recipe

```python
def compute_metrics(predictions, golds, response_tokens, total_joules):
    correct = sum(1 for p, g in zip(predictions, golds) if p == g)
    accuracy = correct / len(golds)
    energy_per_token = total_joules / response_tokens
    return {'accuracy': accuracy, 'energy_per_token': energy_per_token}
```

## Common pitfalls

- Aggressive quantization (e.g., q3) does not always yield lower energy due to computational overhead; q4 variants can be more efficient.
- Response length heavily skews per-token energy metrics; longer outputs distribute fixed inference overhead, making direct per-token comparisons misleading.
- Highly quantized models exhibit high variance in energy consumption, requiring multiple runs or careful averaging for reliable profiling.

## Evidence (verbatim from paper)

> To address RQ2, we analyzed the accuracy-energy tradeoff across quantization levels and model families. Table[8] reports accuracy scores for all models on the five benchmark datasets, while Figure[5] visualizes accuracy comparisons across variants.

## Citation

```bibtex
@misc{husom2025sustainable,
  title={Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency},
  author={Husom et al. (2025)},
  year={2025},
  note={arXiv:2504.03360}
}
```

- arXiv: 2504.03360