Skip to content
Back to skills

Pals Percentile Aware Layerwise Sparsity For Llm Pruning

ASecurity

One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwi. Based on arXiv:2607.07557.

  • 3 stars
  • 0 votes
  • 0 copies
  • 3 views
  • Added September 11, 2026
toolsgo

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill pals-percentile-aware-layerwise-sparsity-for-llm-pruning --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Pals Percentile Aware Layerwise Sparsity For Llm Pruning?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Pals Percentile Aware Layerwise Sparsity For Llm Pruning
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-pals-percentile-aware-layerwise-sparsity-for-llm-p/badge)](https://www.skillsdirectory.com/skills/hiyenwong-pals-percentile-aware-layerwise-sparsity-for-llm-p)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: pals-percentile-aware-layerwise-sparsity-for-llm-pruning
description: 'One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwi. Based on arXiv:2607.07557.'
---

# PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

**arXiv**: 2607.07557 | **Authors**: Yazdan Jamshidi, Alexey Shvets | **Utility**: 0.85

## Overview

One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$). The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none. We also find that gradient-based allocation -- the seemingly more principled approach -- produces results worse than random, suggesting that gradient magnitude does not predict the impact of discrete weight removal. PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.

## Key Contributions

1. One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance.
2. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio.
3. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$).
4. The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none.

## Implementation Notes

- **Keywords**: llm, transformer, model-pruning
- **Categories**: cs.CL, cs.LG
- **Published**: 2026-07-08

## Activation Criteria

Use this skill when working on tasks involving: llm, transformer, model-pruning.

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.