Supervised Fine-Tuning (SFT)

Full definition

After pretraining on raw text, base models are fluent but not helpful. SFT teaches the model the response format users want — answer questions, follow instructions, refuse unsafe requests. The dataset is usually 10k–1M high-quality demonstrations. SFT is the first of three post-training stages in the InstructGPT/ChatGPT recipe: SFT, then reward modeling, then RLHF. SFT alone delivers much of the improvement at a fraction of RLHF's compute cost, but RLHF still adds a clear gain: in the InstructGPT study, labelers preferred the RLHF model's outputs to the SFT model's 73% of the time.

Why it matters

SFT is the cheapest, highest-ROI customization step for enterprises. Most "fine-tuned for our domain" claims are SFT, not RLHF. Buyers should ask vendors how many SFT examples were used and how they were curated — quality dominates quantity above ~10k examples.

Example

A pharma company runs SFT on Llama 3 70B with 25k pairs of (clinical question, regulator-approved answer); the resulting model passes internal compliance review at 3x the rate of the base model.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Reinforcement Learning from Human Feedback (RLHF)
  Reinforcement Learning from Human Feedback (RLHF) is a training technique that aligns a language model's outputs with human preferences by training a reward model on human comparisons of candidate responses, then optimizing the LLM against that reward.
• Fine-Tuning
  Fine-tuning is the process of further training a pretrained model on a smaller, task-specific dataset so it specializes in a particular style, domain, or behavior — without retraining from scratch.
• Pretraining
  Pretraining is the first, most compute-intensive stage of training a foundation model — typically self-supervised next-token prediction over trillions of tokens of text, code, and other modalities — that produces the base model later fine-tuned for specific tasks.
• LoRA (Low-Rank Adaptation)
  LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pretrained model's weights and injects small trainable low-rank matrices into selected layers — cutting trainable parameters by up to 10,000x and GPU memory by about 3x with negligible quality loss on most tasks.

More training & optimization terms

• Data Labeling
  Data labeling is the process of attaching ground-truth annotations to raw data — text, images, audio — so a supervised model can learn from it, ranging from yes/no classification to structured extraction to multi-turn preference comparisons.
• Active Learning
  Active learning is a labeling strategy in which the model selects which unlabeled examples a human should label next — typically the cases the current model is least certain about — focusing labeling effort on the most informative examples.
• Weak Supervision
  Weak supervision is a paradigm for training models on labels generated programmatically by labeling functions, rules, regex, knowledge bases, or distant heuristics — rather than by hand — and then learning to denoise them.