Training And Configs

---
name: training-and-configs
description: "Assemble verl PPO, GRPO, SFT, and on-policy distillation training
  commands, Hydra overrides, backend strategy switches, and config
  troubleshooting guidance."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# training-and-configs

Use this sub-skill when a coding agent needs to build or debug verl training commands and Hydra config overrides for PPO-like RL, GRPO, SFT, or on-policy distillation.

## Route by Task

- **PPO / critic training**: Use `references/training-workflows.md` for the `python -m verl.trainer.main_ppo` command shape, required actor/critic/model/data/logger overrides, and FSDP or Megatron examples.
- **GRPO / critic-less RL**: Use `references/training-workflows.md` for `algorithm.adv_estimator=grpo`, grouped rollout `n`, KL-loss settings, and no-critic command patterns.
- **SFT**: Use `references/training-workflows.md` for `torchrun -m verl.trainer.sft_trainer`, `engine=fsdp`, dataset message keys, LoRA/PEFT, and sequence-parallel switches.
- **On-policy distillation**: Use `references/training-workflows.md` for `distillation.enabled=True`, teacher resource sizing, teacher model overrides, and distillation loss modes.
- **Hydra and backend switches**: Use `references/configuration.md` before changing `model_engine`, FSDP/FSDP2, Megatron, VeOmni, TorchTitan, rollout backend, batch, checkpoint, logger, or generated-reference config fields.
- **Failures and validation errors**: Use `references/troubleshooting.md` for overlong prompts, OOM, Hydra override syntax, config-doc CI, rollout backend mismatch, Ray/GPU resource, and distillation teacher sizing problems.

## Bundled Tool

- `scripts/build_ppo_command.py` assembles a dry-run `python -m verl.trainer.main_ppo` command from common data, model, rollout, backend, batch, and logger options. It prints only; it never imports verl, starts Ray, downloads models, or runs training.

## Boundaries

- Dataset schema details, reward/tool internals, backend installation, and checkpoint merge/export operations belong to other verl sub-skills or repo-level references.
- Do not point future agents at source checkout examples or scripts as runtime dependencies; adapt their patterns into commands or use the bundled script here.