Lora Training Step

---
name: lora_training_step
description: Run a deterministic reduced LoRA training step that updates only low-rank factors and records loss evidence.
---

# LoRA Reduced Training Step

Use this skill when full pretrained model training is blocked but soft-mode recovery permits a mechanism-faithful reduced experiment. It executes an actual optimizer update on LoRA factors over supervised examples. Do not call this a full GLUE, GPT-2, or pretrained-model reproduction.

## Inputs
- Synthetic or resource-derived supervised examples.
- Frozen base weight matrix and LoRA A/B matrices.
- Learning rate, alpha scale, and number of bounded update steps.

## Outputs
- `loss_before` and `loss_after`.
- `params_before` and `params_after` with changed LoRA factors and unchanged base weights.
- Merge-equivalence diagnostics for the trained adapter.

## Workflow
1. Construct examples that expose a task update missing from the frozen base model.
2. Compute predictions with `W0 x + (alpha/r) B A x`.
3. Backpropagate mean-squared error analytically into A and B only.
4. Update A and B with a bounded deterministic gradient step and keep W0 unchanged.
5. Save validator-compatible traces with before/after losses and parameters.

## Validation
Run `python scripts/reduced_training.py --output /tmp/trace.json` or the included tests. The test requires loss reduction, changed LoRA factors, unchanged base weights, and merged inference equivalence.

## Limitations
This is a reduced standard-library proxy for the LoRA mechanism, not training of RoBERTa, DeBERTa, GPT-2, or GPT-3.