Skip to content
Back to skills

Tt Opd Medical Agent Training

ASecurity

Turn-level Truncated On-Policy Distillation (TT-OPD) methodology for training multi-turn medical AI agents via reinforcement learning. Addresses multi-turn collapse, response length explosion, and tool-use erosion in clinical dialogue. Use when training medical AI agents, RL-based dialogue systems, or multi-turn agentic systems where sparse terminal rewards cause training instability.

  • 3 stars
  • 0 votes
  • 0 copies
  • 11 views
  • Added September 11, 2026
toolsperformance

Works with

  • terminal
  • cli

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill tt-opd-medical-agent-training --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Tt Opd Medical Agent Training?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Tt Opd Medical Agent Training
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-tt-opd-medical-agent-training/badge)](https://www.skillsdirectory.com/skills/hiyenwong-tt-opd-medical-agent-training)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: tt-opd-medical-agent-training
description: "Turn-level Truncated On-Policy Distillation (TT-OPD) methodology for training multi-turn medical AI agents via reinforcement learning. Addresses multi-turn collapse, response length explosion, and tool-use erosion in clinical dialogue. Use when training medical AI agents, RL-based dialogue systems, or multi-turn agentic systems where sparse terminal rewards cause training instability."
---

# TT-OPD: Turn-level Truncated On-Policy Distillation

## Problem
Multi-turn agentic structures in RL training collapse into verbose single-turn monologues, with response length explosion and tool-use frequency erosion. Root cause: misalignment between sparse terminal rewards and sequential clinical trajectories.

## Core Mechanism

### Self-Distillation Architecture
1. Maintain a teacher model updated via gradient-free EMA (Exponential Moving Average)
2. Apply dense, outcome-aware KL regularization at every conversation turn
3. Leverage outcome-privileged information unavailable at inference time

### Key Steps
1. Initialize student and teacher models from same base
2. For each turn in multi-turn dialogue:
   - Compute turn-level reward signal (not just terminal)
   - Apply KL divergence between student output and teacher output
   - Weight KL by outcome quality (privileged signal)
   - Update student; update teacher via EMA of student weights
3. Monitor tool-use frequency as stability metric

## When to Use
- Training multi-turn dialogue agents with RL
- Clinical/medical AI agent training environments
- Any agentic system where GRPO/vanilla RL causes length explosion
- Systems requiring sustained tool-use across conversation turns

## Pitfalls
- Vanilla GRPO achieves strong final accuracy but suffers training instability
- Monitor response length oscillations as early warning of collapse
- Outcome-privileged information must be available during training only
- EMA update rate critically affects convergence speed

## Verification
- Tool-use frequency should remain stable across training
- Response length should not monotonically increase
- Performance on held-out benchmarks should improve steadily

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.