Skip to content
Back to skills

Learning Intrinsic Sparse Structures Within Long Shortterm Memory

ASecurity

**arXiv ID:** 1709.05027 **Authors:** Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li **Published:** 2017-09-15T01:10:23Z **Abstract:** Model compression is significant for the wide adoption of Recurrent Neural Networks (RNNs) in both user devices possessing limited resources and business clusters requiring quick responses to large-scale service requests. This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reduc...

  • 3 stars
  • 0 votes
  • 0 copies
  • 5 views
  • Added September 11, 2026
toolsgit

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill learning-intrinsic-sparse-structures-within-long-shortterm-memory --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Learning Intrinsic Sparse Structures Within Long Shortterm Memory?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Learning Intrinsic Sparse Structures Within Long Shortterm Memory
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-learning-intrinsic-sparse-structures-within-long-s/badge)](https://www.skillsdirectory.com/skills/hiyenwong-learning-intrinsic-sparse-structures-within-long-s)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
# Learning Intrinsic Sparse Structures within Long Short-Term Memory

**arXiv ID:** 1709.05027
**Authors:** Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li
**Published:** 2017-09-15T01:10:23Z
**Abstract:**
Model compression is significant for the wide adoption of Recurrent Neural Networks (RNNs) in both user devices possessing limited resources and business clusters requiring quick responses to large-scale service requests. This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs. Independently reducing the sizes of basic structures can result in inconsistent dimensions among them, and consequently, end up with invalid LSTM units. To overcome the problem, we propose Intrinsic Sparse Structures (ISS) in LSTMs. Removing a component of ISS will simultaneously decrease the sizes of all basic structures by one and thereby always maintain the dimension consistency. By learning ISS within LSTM units, the obtained LSTMs remain regular while having much smaller basic structures. Based on group Lasso regularization, our method achieves 10.59x speedup without losing any perplexity of a language modeling of Penn TreeBank dataset. It is also successfully evaluated through a compact model with only 2.69M weights for machine Question Answering of SQuAD dataset. Our approach is successfully extended to non- LSTM RNNs, like Recurrent Highway Networks (RHNs). Our source code is publicly available at https://github.com/wenwei202/iss-rnns

## Skill Description

This skill is generated from the arXiv paper: Learning Intrinsic Sparse Structures within Long Short-Term Memory (1709.05027).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:1709.05027](http://arxiv.org/abs/1709.05027v7)

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.