Back to skills
SKILL.md
Kat Kl Agreement Trap Termination
ASecurityKL Agreement Trap Termination (KAT) for on-policy distillation. Detects persistent low-KL agreement traps and terminates early to improve training efficiency.
- 3 stars
- 0 votes
- 0 copies
- 5 views
- Added September 11, 2026
Works with
Security analysis
100/100npx -y skills add hiyenwong/ai_collection --skill kat-kl-agreement-trap-termination --agent claude-codeAre you the author of Kat Kl Agreement Trap Termination?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-kat-kl-agreement-trap-termination)---
name: kat-kl-agreement-trap-termination
description: KL Agreement Trap Termination (KAT) for on-policy distillation. Detects persistent low-KL agreement traps and terminates early to improve training efficiency.
version: 1.0
created: 2026-06-10
source: arXiv 2606.09471v1
tags: [distillation, OPD, KL-divergence, termination, LLM, reasoning]
---
# KAT: KL Agreement Trap Termination
Online termination rule for On-Policy Distillation (OPD) that detects when student enters unrecoverable "KL agreement trap" states.
## Problem Statement
In on-policy distillation, when the student drifts into an unrecoverable prefix:
- Teacher may locally agree with degraded state
- Produces low reverse KL but little corrective training signal
- Tokens during and after traps produce less useful supervision
## KAT Detection Algorithm
### Dynamic Threshold Detection
```python
class KLAgreementTrapDetector:
def __init__(self, threshold_alpha=0.05, window_size=10):
self.threshold_alpha = threshold_alpha
self.window_size = window_size
self.kl_history = []
def detect_trap(self, kl_divergence, training_step):
# Dynamic threshold adapts to training progress
dynamic_threshold = self.compute_adaptive_threshold(training_step)
# Track persistent low-KL
self.kl_history.append(kl_divergence)
if len(self.kl_history) > self.window_size:
self.kl_history.pop(0)
# Check for persistent agreement
avg_kl = np.mean(self.kl_history)
if avg_kl < dynamic_threshold:
return True # Trap detected
return False
def compute_adaptive_threshold(self, step):
# Threshold increases as model improves
base_threshold = 0.1
return base_threshold * (1 + self.threshold_alpha * step)
```
### Termination Rule
```python
def kat_termination_rule(rollout, teacher_logits, detector):
kl_values = []
for position, (student_token, teacher_dist) in enumerate(rollout):
kl = compute_reverse_kl(student_token, teacher_dist)
kl_values.append(kl)
# Check for trap at each position
if detector.detect_trap(kl, position):
# Terminate early, discard weak supervision
return rollout[:position], kl_values[:position]
return rollout, kl_values # Full rollout if no trap
```
## Results on Math Benchmarks
- Avg@k accuracy: +2.66%
- Pass@k: +3.43%
- Average rollout length: -59.73% (significant efficiency gain)
## Integration with OPD
```python
def opd_with_kat(student, teacher, problems, detector):
for problem in problems:
# Generate student rollout
rollout = student.generate(problem)
# Get teacher logits
teacher_logits = teacher.score(rollout)
# Apply KAT termination
filtered_rollout, filtered_kl = kat_termination_rule(
rollout, teacher_logits, detector
)
# Train only on non-trapped portions
student.update(filtered_rollout, filtered_kl)
```
## When to Use
- On-policy distillation for reasoning models
- Mathematical reasoning benchmarks
- When student trajectories can drift into unrecoverable states
- To reduce rollout length while improving accuracy
## Activation Triggers
- `KL agreement trap`, `OPD termination`, `on-policy distillation quality`, `KAT`, `distillation trap detection`
## References
- arXiv:2606.09471v1 - Xin et al., "Escaping the KL Agreement Trap in On-Policy Distillation"
- Reverse KL divergence for distillation
- On-policy vs off-policy distillation trade-offsAttribution
Related skills
View all in tools →Comments
Loading comments…
Newsletter
Get the weekly list of safe new skills, and the ones we pulled.