Skip to content
Back to skills

Rl Post Training Builds Compositional Reasoning Strategies

ASecurity

Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable. Based on arXiv:2607.07646.

  • 3 stars
  • 0 votes
  • 0 copies
  • 6 views
  • Added September 11, 2026
toolsgo

Security analysis

A100/100

Scanned September 11, 2026

npx -y skills add hiyenwong/ai_collection --skill rl-post-training-builds-compositional-reasoning-strategies --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Rl Post Training Builds Compositional Reasoning Strategies?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Rl Post Training Builds Compositional Reasoning Strategies
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/hiyenwong-rl-post-training-builds-compositional-reasoning-st/badge)](https://www.skillsdirectory.com/skills/hiyenwong-rl-post-training-builds-compositional-reasoning-st)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: rl-post-training-builds-compositional-reasoning-strategies
description: 'Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable. Based on arXiv:2607.07646.'
---

# RL Post-Training Builds Compositional Reasoning Strategies

**arXiv**: 2607.07646 | **Authors**: Azwar Abdulsalam, Nishil Patel, Andrew Saxe | **Utility**: 0.87

## Overview

Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures. These include sequential compositions, which collapse ordered chains of primitive contractions, and parallel compositions, which combine independent primitive contractions in a single step. The composed procedures are not isolated samples; they are reused and consolidated into a stable repertoire. Comparing RL with rejection fine-tuning shows that the key difference is not exploration volume but selectivity: RFT produces many shortcut-like rewrites, much of them invalid, whereas RL concentrates exploration into valid reusable structure. Pretraining ablations show that the emergence of compositional strategies is gated not by primitive exposure alone, but by whether pretraining organizes primitive competence into reduction procedures that RL can later compress. The base model provides weak procedural ingredients; RL builds them into reliable higher-level strategies.

## Key Contributions

1. Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited.
2. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward.
3. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus.
4. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures.

## Implementation Notes

- **Keywords**: transformer, skill-library, reasoning
- **Categories**: cs.AI, cs.CL
- **Published**: 2026-07-08

## Activation Criteria

Use this skill when working on tasks involving: transformer, skill-library, reasoning.

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in tools →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.