Instruction Tuning

Full definition

Introduced at scale in Google's FLAN (Wei et al., 2021) and T0 (Sanh et al., 2022), instruction tuning is the technical reason ChatGPT, Claude, and Gemini feel "useful" out of the box rather than just predicting next tokens. Datasets include FLAN, Super-NaturalInstructions, Dolly, OpenAssistant, Tulu. Instruction tuning is the SFT stage that precedes preference-tuning (RLHF/DPO) in modern post-training pipelines.

Why it matters

For enterprises building domain models, instruction-tuning with a few thousand high-quality instruction-response pairs produces 80% of the benefit of full RLHF at 5% of the cost. It is the most cost-effective customization step.

Example

An insurance company instruction-tunes Llama 3 70B on 8,000 (claims-question, approved-answer) pairs from senior adjusters; the model handles 60% of L1 inquiries autonomously, escalating only edge cases.

Related terms

• Supervised Fine-Tuning (SFT)
  Supervised Fine-Tuning (SFT) is the stage of LLM post-training where a pretrained model is fine-tuned on curated input-output pairs, typically instruction-response demonstrations written by humans.
• Fine-tuning
  Fine-tuning is the process of further training a pretrained model on a smaller, task-specific dataset so it specializes in a particular style, domain, or behavior — without retraining from scratch.
• Reinforcement Learning from Human Feedback (RLHF)
  Reinforcement Learning from Human Feedback (RLHF) is a training technique that aligns a language model's outputs with human preferences by training a reward model on human comparisons of candidate responses, then optimizing the LLM against that reward.
• LoRA (Low-Rank Adaptation)
  LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pretrained model's weights and injects small trainable low-rank matrices into selected layers — reducing fine-tuning cost by 100-1000x with negligible quality loss on most tasks.