Skip to content
Back to skills

Implicit Variance Regularization In Noncontrastive Ssl

ASecurity

**arXiv ID:** 2212.04858 **Authors:** Manu Srinath Halvagal, Axel Laborieux, Friedemann Zenke **Published:** 2022-12-09T13:56:42Z **Abstract:** Non-contrastive SSL methods like BYOL and SimSiam rely on asymmetric predictor networks to avoid representational collapse without negative samples. Yet, how predictor networks facilitate stable learning is not fully understood. While previous theoretical analyses assumed Euclidean losses, most practical implementations rely on cosine similarity. To g...

  • 3 stars
  • 0 votes
  • 0 copies
  • 5 views
  • Added September 11, 2026
tools

Works with

  • cli

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill implicit-variance-regularization-in-noncontrastive-ssl --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Implicit Variance Regularization In Noncontrastive Ssl?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Implicit Variance Regularization In Noncontrastive Ssl
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-implicit-variance-regularization-in-noncontrastive/badge)](https://www.skillsdirectory.com/skills/hiyenwong-implicit-variance-regularization-in-noncontrastive)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
# Implicit variance regularization in non-contrastive SSL

**arXiv ID:** 2212.04858
**Authors:** Manu Srinath Halvagal, Axel Laborieux, Friedemann Zenke
**Published:** 2022-12-09T13:56:42Z
**Abstract:**
Non-contrastive SSL methods like BYOL and SimSiam rely on asymmetric predictor networks to avoid representational collapse without negative samples. Yet, how predictor networks facilitate stable learning is not fully understood. While previous theoretical analyses assumed Euclidean losses, most practical implementations rely on cosine similarity. To gain further theoretical insight into non-contrastive SSL, we analytically study learning dynamics in conjunction with Euclidean and cosine similarity in the eigenspace of closed-form linear predictor networks. We show that both avoid collapse through implicit variance regularization albeit through different dynamical mechanisms. Moreover, we find that the eigenvalues act as effective learning rate multipliers and propose a family of isotropic loss functions (IsoLoss) that equalize convergence rates across eigenmodes. Empirically, IsoLoss speeds up the initial learning dynamics and increases robustness, thereby allowing us to dispense with the EMA target network typically used with non-contrastive methods. Our analysis sheds light on the variance regularization mechanisms of non-contrastive SSL and lays the theoretical grounds for crafting novel loss functions that shape the learning dynamics of the predictor's spectrum.

## Skill Description

This skill is generated from the arXiv paper: Implicit variance regularization in non-contrastive SSL (2212.04858).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2212.04858](http://arxiv.org/abs/2212.04858v2)

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.