Skip to content
Back to skills

Testtime Regression A Unifying Framework For Designing Sequence Models With Associative Memory

ASecurity

**arXiv ID:** 2501.12352 **Authors:** Ke Alexander Wang, Jiaxin Shi, Emily B. Fox **Published:** 2025-01-21T18:32:31Z **Abstract:** Sequence models lie at the heart of modern deep learning. However, rapid advancements have produced a diversity of seemingly unrelated architectures, such as Transformers and recurrent alternatives. In this paper, we introduce a unifying framework to understand and derive these sequence models, inspired by the empirical importance of associative recall, the capab...

  • 3 stars
  • 0 votes
  • 0 copies
  • 8 views
  • Added September 11, 2026
toolsgoapi

Works with

  • api

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill testtime-regression-a-unifying-framework-for-designing-sequence-models-with-associative-memory --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Testtime Regression A Unifying Framework For Designing Sequence Models With Associative Memory?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Testtime Regression A Unifying Framework For Designing Sequence Models With Associative Memory
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-testtime-regression-a-unifying-framework-for-desig/badge)](https://www.skillsdirectory.com/skills/hiyenwong-testtime-regression-a-unifying-framework-for-desig)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
# Test-time regression: a unifying framework for designing sequence models with associative memory

**arXiv ID:** 2501.12352
**Authors:** Ke Alexander Wang, Jiaxin Shi, Emily B. Fox
**Published:** 2025-01-21T18:32:31Z
**Abstract:**
Sequence models lie at the heart of modern deep learning. However, rapid advancements have produced a diversity of seemingly unrelated architectures, such as Transformers and recurrent alternatives. In this paper, we introduce a unifying framework to understand and derive these sequence models, inspired by the empirical importance of associative recall, the capability to retrieve contextually relevant tokens. We formalize associative recall as a two-step process, memorization and retrieval, casting memorization as a regression problem. Layers that combine these two steps perform associative recall via ``test-time regression'' over its input tokens. Prominent layers, including linear attention, state-space models, fast-weight programmers, online learners, and softmax attention, arise as special cases defined by three design choices: the regression weights, the regressor function class, and the test-time optimization algorithm. Our approach clarifies how linear attention fails to capture inter-token correlations and offers a mathematical justification for the empirical effectiveness of query-key normalization in softmax attention. Further, it illuminates unexplored regions within the design space, which we use to derive novel higher-order generalizations of softmax attention. Beyond unification, our work bridges sequence modeling with classic regression methods, a field with extensive literature, paving the way for developing more powerful and theoretically principled architectures.

## Skill Description

This skill is generated from the arXiv paper: Test-time regression: a unifying framework for designing sequence models with associative memory (2501.12352).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2501.12352](http://arxiv.org/abs/2501.12352v3)

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.