Installs into .claude/skills of the current project.
Are you the author of Fine Tuning?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/vectorspacelab-fine-tuning-arex-skill)
---
name: fine-tuning
description: "Prepare FlagEmbedding fine-tuning data and safe training commands
for embedders and rerankers."
disable-model-invocation: true
metadata:
disco-role: operating
license: MIT
---
# Fine-Tuning
Use this sub-skill when the task is to prepare or validate FlagEmbedding fine-tuning data, choose the correct training module for an embedder or reranker, or build a bounded `torchrun` command without launching training automatically.
Read first:
- `references/data-formats.md` for JSONL schemas, hard negatives, teacher scores, `prompt`, and embedder ICL `type` handling.
- `references/training-commands.md` for module entry points, command patterns, DeepSpeed config shapes, LoRA flags, flash-attn caveats, and bge-m3 unified fine-tuning.
- `references/troubleshooting.md` when JSONL validation, optional finetune extras, CUDA, DeepSpeed, flash-attn, cache/token, OOM, or distributed launch issues appear.
Bundled helpers:
- `scripts/validate_train_jsonl.py` validates embedder or reranker JSONL before training. Use `--knowledge-distillation` only when score fields are intended for KD.
- `scripts/split_jsonl_by_text_length.py` splits JSONL by safe character or token-estimate length without downloading tokenizers.
Route away:
- Inference-only model loading, encoding, reranking, and score computation belongs to sibling `inference`.
- Post-training retrieval, reranking, MTEB, BEIR, or custom benchmark work belongs to sibling `evaluation`.
- Broad package installation, import failures, and backend setup outside fine-tuning belongs to the root troubleshooting reference.
Operational constraints:
- Do not run full training unless the user explicitly asks for it and model, data, devices, caches, output directory, and runtime budget are clear.
- Treat hard-negative mining and teacher-score generation as model-loading workflows with network/cache/device dependencies unless every model artifact is already local.
- The optional `FlagEmbedding[finetune]` extra adds DeepSpeed and flash-attn, but full GPU training and flash-attn compatibility were not verified by this sub-skill.
