Pals Percentile Aware Layerwise Sparsity For Llm Pruning
ASecurity
One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwi. Based on arXiv:2607.07557.
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Pals Percentile Aware Layerwise Sparsity For Llm Pruning?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-pals-percentile-aware-layerwise-sparsity-for-llm-p)
---
name: pals-percentile-aware-layerwise-sparsity-for-llm-pruning
description: 'One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwi. Based on arXiv:2607.07557.'
---
# PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning
**arXiv**: 2607.07557 | **Authors**: Yazdan Jamshidi, Alexey Shvets | **Utility**: 0.85
## Overview
One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$). The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none. We also find that gradient-based allocation -- the seemingly more principled approach -- produces results worse than random, suggesting that gradient magnitude does not predict the impact of discrete weight removal. PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.
## Key Contributions
1. One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance.
2. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio.
3. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$).
4. The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none.
## Implementation Notes
- **Keywords**: llm, transformer, model-pruning
- **Categories**: cs.CL, cs.LG
- **Published**: 2026-07-08
## Activation Criteria
Use this skill when working on tasks involving: llm, transformer, model-pruning.