Skip to content
Back to skills

Sageattention2 Efficient Attention With Thorough Outlier Smoothing And Perthread Int4 Quantization

ASecurity

**arXiv ID:** 2411.10958 **Authors:** Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, Jianfei Chen **Published:** 2024-11-17T04:35:49Z **Abstract:** Although quantization for linear layers has been widely used, its application to accelerate the attention process remains limited. To further enhance the efficiency of attention computation compared to SageAttention while maintaining precision, we propose SageAttention2, which utilizes significantly faster 4-bit matrix multiplication...

  • 3 stars
  • 0 votes
  • 0 copies
  • 6 views
  • Added September 11, 2026
toolsgit

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill sageattention2-efficient-attention-with-thorough-outlier-smoothing-and-perthread-int4-quantization --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Sageattention2 Efficient Attention With Thorough Outlier Smoothing And Perthread Int4 Quantization?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Sageattention2 Efficient Attention With Thorough Outlier Smoothing And Perthread Int4 Quantization
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-sageattention2-efficient-attention-with-thorough-o/badge)](https://www.skillsdirectory.com/skills/hiyenwong-sageattention2-efficient-attention-with-thorough-o)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
# SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization

**arXiv ID:** 2411.10958
**Authors:** Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, Jianfei Chen
**Published:** 2024-11-17T04:35:49Z
**Abstract:**
Although quantization for linear layers has been widely used, its application to accelerate the attention process remains limited. To further enhance the efficiency of attention computation compared to SageAttention while maintaining precision, we propose SageAttention2, which utilizes significantly faster 4-bit matrix multiplication (Matmul) alongside additional precision-enhancing techniques. First, we propose to quantize matrices $(Q, K)$ to INT4 in a hardware-friendly thread-level granularity and quantize matrices $(\widetilde P, V)$ to FP8. Second, we propose a method to smooth $Q$, enhancing the accuracy of INT4 $QK^\top$. Third, we propose a two-level accumulation strategy for $\widetilde PV$ to enhance the accuracy of FP8 $\widetilde PV$. The operations per second (OPS) of SageAttention2 surpass FlashAttention2 and xformers by about 3x and 4.5x on RTX4090, respectively. Moreover, SageAttention2 matches the speed of FlashAttention3(fp8) on the Hopper GPUs, while delivering much higher accuracy. Comprehensive experiments confirm that our approach incurs negligible end-to-end metrics loss across diverse models, including those for language, image, and video generation. The code is available at https://github.com/thu-ml/SageAttention.

## Skill Description

This skill is generated from the arXiv paper: SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization (2411.10958).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2411.10958](http://arxiv.org/abs/2411.10958v7)

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.