Training And Data Prep

---
name: training-and-data-prep
description: "Guide safe Stanza training, evaluation, and dataset preparation
  across model families."
disable-model-invocation: true
metadata:
  disco-role: operating
license: NOASSERTION
---

# Training and Data Prep

Use this sub-skill when you need to prepare corpora or build training and evaluation command plans for Stanza model families.

It covers:
- tokenizer, MWT, POS, lemma, dependency parsing, NER, constituency, charlm, classifier, lang_identifier, and wl_coref workflows;
- wrapper scripts under `stanza.utils.training`;
- dataset preparation helpers under `stanza.utils.datasets`;
- safe command templates that avoid accidental downloads, wandb side effects, or save-dir collisions.

These details were distilled from installed Stanza 1.14.0 training/data utilities, selected tests, CI setup, and command help.

## Start here
1. Read `references/cli-reference.md` to choose the right entry point.
2. Run `scripts/build_training_command.py --help` or `scripts/build_training_command.py <task> <name> --mode train` to print a safe command template.
3. Copy or source `scripts/config_template.sh` to set local data roots and output directories.
4. Read `references/data-preparation.md` before moving any corpus files.
5. Use `references/training-workflows.md` for direct-module versus wrapper choices.
6. Use `references/troubleshooting.md` when a corpus, schema, pretrain, charlm, GPU, or logging step fails.

## Boundaries
- Route pipeline inference and resource loading to `pipelines-and-resources`.
- Route raw `Document`/CoNLL-U manipulation to `documents-and-conllu`.
- Route demos and browser-facing rendering to `visualization-and-demos`.
- Do not start training or large downloads from this sub-skill.

## Bundled helpers
- `scripts/build_training_command.py` prints representative `python -m` command templates, validates task names, and never launches a job.
- `scripts/config_template.sh` mirrors the common training environment variables with safe defaults and placeholders.