Skip to content
Back to skills

Sparsity In Deep Learning Pruning And Growth For Efficient Inference And Training In Neural Networks

ASecurity

**arXiv ID:** 2102.00554 **Authors:** Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste **Published:** 2021-01-31T22:48:50Z **Abstract:** The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biological counterparts, sparse networks generalize just as well, if not better than, the original dense networks. Sparsity can reduce the memory footprint of...

  • 3 stars
  • 0 votes
  • 0 copies
  • 5 views
  • Added September 11, 2026
toolsgoperformance

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill sparsity-in-deep-learning-pruning-and-growth-for-efficient-inference-and-training-in-neural-networks --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Sparsity In Deep Learning Pruning And Growth For Efficient Inference And Training In Neural Networks?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Sparsity In Deep Learning Pruning And Growth For Efficient Inference And Training In Neural Networks
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-sparsity-in-deep-learning-pruning-and-growth-for-e/badge)](https://www.skillsdirectory.com/skills/hiyenwong-sparsity-in-deep-learning-pruning-and-growth-for-e)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
# Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

**arXiv ID:** 2102.00554
**Authors:** Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste
**Published:** 2021-01-31T22:48:50Z
**Abstract:**
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biological counterparts, sparse networks generalize just as well, if not better than, the original dense networks. Sparsity can reduce the memory footprint of regular networks to fit mobile devices, as well as shorten training time for ever growing networks. In this paper, we survey prior work on sparsity in deep learning and provide an extensive tutorial of sparsification for both inference and training. We describe approaches to remove and add elements of neural networks, different training strategies to achieve model sparsity, and mechanisms to exploit sparsity in practice. Our work distills ideas from more than 300 research papers and provides guidance to practitioners who wish to utilize sparsity today, as well as to researchers whose goal is to push the frontier forward. We include the necessary background on mathematical methods in sparsification, describe phenomena such as early structure adaptation, the intricate relations between sparsity and the training process, and show techniques for achieving acceleration on real hardware. We also define a metric of pruned parameter efficiency that could serve as a baseline for comparison of different sparse networks. We close by speculating on how sparsity can improve future workloads and outline major open problems in the field.

## Skill Description

This skill is generated from the arXiv paper: Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks (2102.00554).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2102.00554](http://arxiv.org/abs/2102.00554v1)

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.