Claude Skills by ADu2021

Use dynamic temperature scheduling that explores early (high temperature) and exploits late (low temperature) during generation, improving sample efficiency in RL with verifiable rewards.

Accelerate LLM pretraining by leveraging small pretrained models as teachers, enabling large models to explicitly learn late-layer knowledge in early layers via alignment loss, achieving 1.6x convergence speedup and 5% downstream improvement even with 10x smaller teachers.

Concept-based explanations quantify how high-level concepts (e.g., gender or experience) influence model behavior, which is crucial for decision-makers in high-stakes domains. Recent work evaluates the faithfulness of such explanations by comparing them to reference causal effects estimated from counterfactuals. In practice, existing benchmarks rely on costly human-written counterfactuals that serve as an imperfect proxy. To address this, we introduce a framework for constructing datasets con...

Multi-stage training approach using entropy-preserving SFT and token-wise entropy-adaptive RL to improve instruction adherence. Combines data curation with reward-guided reasoning, outperforming larger models on IFEval.

Reduce inference tokens by up to 38× and API calls by 30× through three-stage memory system: sensory compression, short-term consolidation, and offline long-term storage.

Aligns reasoning traces with final decisions in preference models using an independent frozen VLM as listener. Achieves 67.4% accuracy on ImageReward by enforcing consistency between explanations and choices. Use when training reward models for image generation quality where both reasoning quality and accuracy matter.

Accelerate video diffusion generation by exploiting temporal attention sparsity—skip redundant attention tiles across denoising steps using persistent skip masks, achieving 40% speedup with quality retention.

Benchmark framework for evaluating LLM agents navigating large-scale Model Context Protocol ecosystems with multi-tool composition across 95 daily tasks.

Enable richer visual reasoning in multimodal models via learnable latent tokens trained with visual bottlenecking. Introduces special tokens that implicitly learn task-relevant visual abstractions without explicit supervision, through attention masking forcing visual information through latents—improving vision-heavy tasks without task-specific annotations.

LK Losses replace KL divergence with direct acceptance rate optimization for speculative decoding, achieving 8-10% improvements without computational overhead.

Convert pre-trained autoregressive models into large-scale diffusion language models via Warmup-Stable-Decay training strategy. Progressively increase block size during warmup, perform stable diffusion training, then decay to smaller blocks for inference. Achieve 535 tokens-per-second with document-level masking and confidence-aware training.

Identifies Lazy Likelihood Displacement (LLD) as GRPO failure mechanism in tool-integrated LLMs and proposes lightweight LLDS regularization that penalizes only tokens responsible for likelihood decrease, achieving up to 45.2% performance gains.

Learn to enhance LLM post-training for diverse creative outputs by weighting training pairs using deviation metrics (semantic and style diversity). Applies to models where standard alignment reduces diversity, enabling competitive quality with higher output variety.

Automatically generate high-quality domain-specific fine-tuning datasets from raw documents using adaptive processing and persona-driven synthesis, preserving general capabilities while specializing models.

Benchmark LLMs on real-world forecasting questions from Metaculus, comparing against human crowds and expert forecasters. Identifies which domains LLMs handle well and where they fall short relative to human intelligence.

Interpret LLM behavior as locally linear mappings between hidden representations, enabling mechanistic understanding of computation without examining individual weights or attention patterns.

Understand and mitigate unintended memorization of training data in LLMs by systematizing mechanisms, detection methods, and mitigation strategies across the model lifecycle.

Enable language models to explore code sandboxes to solve diverse tasks like mathematics and long-context reasoning without additional training, with reinforcement learning further enhancing capabilities. Use when you need LLMs to perform tasks requiring exploration and code execution in isolated environments.

Learn to encode LLM-generated responses rather than raw inputs by training special tokens and projection layers while keeping the backbone frozen. Bridges the input-output gap to transfer LLM capabilities like reasoning directly into embedding space.

Accelerate image generation by generating multiple patches in parallel instead of sequentially. Uses locality-aware ordering that generates spatially-close tokens while keeping concurrent tokens far apart, reducing steps from 256 to 20 (12× speedup) while maintaining quality.

Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat MI as an observational science, summarizing analytical insights while lacking a systematic framework for actionable intervention. To bridge this gap, we present a practical survey structured around the pipeline: 'Locate, Steer, and Improve.' We formally categorize Localizing (diagnosis) and Steering (intervention) ...

Reduce self-attention complexity from O(N²) to O(N log N) through hierarchical token selection and enrichment. Perform hierarchical Top-K selection progressively adopting sparse Top-K at each level. Implement sparse index transpose algorithm avoiding dense mask construction. Achieve 28.27× faster inference and 6.09× faster training.

Train LLMs for long-context reasoning using KeyChain synthesis: convert short multi-hop QA into long-context tasks by embedding UUID chains in distractor documents, enabling 16K→128K generalization.

Navigate Long CoT Degradation phenomenon when training small models on extended reasoning, understanding recovery dynamics and implementing strategies to maintain performance.

Implements Controllable Memory Usage from arXiv:2601.05107

Generate animations longer than 500 frames with consistent coloring. Uses dynamic global-local memory to compress long-term history and intelligently fuse global color features with local generation, enabling 49% quality improvement over previous methods.

Implement techniques from LongCat-Flash-Thinking-2601 Technical Report. LongCat-Flash-Thinking-2601 achieves state-of-the-art performance among open-source models on a wide range of agentic benchmarks, including agentic search, agentic tool use, and tool-integrated reasoning

Train reward models to evaluate long-context responses by introducing faithfulness as a criterion and using consistency-based alignment to maintain judgment-explanation coherence.

Training-free method extending diffusion LLMs to 6x context length using NTK-based RoPE scaling, exploiting bidirectional attention stability.

Generate consistent ultra-long videos (up to one minute) using depth and trajectory controls with autoregressive segment generation and unified noise initialization.

Train vision-language models on hour-long videos using a two-stage pipeline combining supervised fine-tuning with GRPO, reaching 71% accuracy on VideoMME while supporting 8,192 frames through efficient multi-modal sequence parallelism.

Evict unnecessary KV cache entries using trainable lookahead tokens and LoRA modules that predict token importance without drafting responses. Achieve 14.5× speedup over draft-based methods with 2% eviction overhead.

Enable budget-conditioned reasoning by repeatedly applying a shared transformer block stack with trajectory-based conditioning on time and step size. Train via shortcut-consistency loss to align shorter and full-length trajectories, enabling variable-depth inference without retraining.

Improve LLM tool-use through automated, closed-loop data curation combining capability probing, error correction, and targeted data expansion—enabling self-refining training pipelines that surpass larger models without expensive APIs.

Maximize parallelism in diffusion language models by intelligently exploring token filling orders. Spawn multiple candidate branches predicting high-confidence positions, evaluate all branches in one forward pass, and select the branch enabling most future parallelism—increasing tokens-per-forward-pass 4.4× without accuracy loss.

Preserve reasoning diversity in RL by protecting low-probability tokens that represent novel thinking paths. Trigger: maintain exploration capability during RL training while avoiding entropy collapse on reasoning tasks.

Filter training samples by response length to identify high-confidence correct solutions and complex problems, improving sample efficiency in LLM reasoning RL without additional metrics.

Improves instruction-following by using differentiated reward mechanisms that recognize logical dependencies (sequential, conditional, parallel) in instructions, enabling better model reasoning about task structure.

Multimodal system for joint audio-video understanding and generation, enhancing agent perception and communication in environments with both audio and visual information.

Build generalist agents for 3D open worlds using vision-language models with human-like interaction patterns and adaptive reasoning—achieving zero-shot transfer across games without fine-tuning through pixel-level perception and high-frequency action generation.

Generate videos autoregressively by extending LLM architectures to spatiotemporal data. MM-RoPE balances frequency spectra across temporal and spatial dimensions, while Autoregressive Diffusion Forcing enables efficient parallel decoding. Lumos-1 (0.5B-3B variants) matches or exceeds Show-o2 and COSMOS on text-to-video with training on 48 GPUs.

Fix length bias in sequence-level policy optimization by scaling each sequence's loss by its token count, eliminating gradient imbalances that cause models to shorten responses during training.

Classify attention heads into retrieval (full attention) and sparse (token-selected) roles using HardKuma distribution for differentiable discrete optimization. Sparse heads reuse KV pairs from retrieval heads, reducing cache by 90% while maintaining quality through joint training.

Enable models to stop generating when confident through lightweight hidden-state probes with distributional guarantees. LYNX achieves cross-domain transferability without retraining—ideal when you need confidence-controlled reasoning efficiency.

Design and evaluate multimodal agents for tool use with M3-Bench: assess three interconnected dimensions (multi-modal grounding, multi-hop causality, multi-threaded parallelism) using similarity-bucketed Hungarian alignment for transparent tool call evaluation without LLM judges.

Generates molecules meeting precise numeric property constraints across multiple dimensions through two-stage multi-agent framework with fragment-level edits and Group Relative Policy Optimization, improving validity and property satisfaction.

Decomposes document processing into specialized agents (planning, execution, judgment, answer) with agent-wise adaptive test-time scaling. Achieves 9.9-11.5% performance gain with smaller models while maintaining reasoning.

Build reasoning capabilities through pure RL without distilled traces, achieving 50% AIME accuracy improvement via scalable asynchronous training with novel reward shaping for multilingual consistency.

Memory management architecture using multi-graph representations for autonomous agents, enabling efficient knowledge organization, contextual retrieval, and dynamic memory expansion to support complex agent decision-making.

Scale GUI agents to real-world complexity via extended action space (user interaction, tool calls) and device-cloud collaboration. Online RL supports 500+ parallel environments with asynchronous handling; local agent monitors trajectory alignment and handoffs to cloud when drift detected—achieving 41.7% MobileWorld success with privacy-preserving delegation.