Few-shot / Zero-shot Learning

Full definition

GPT-3 popularized in-context few-shot learning: rather than fine-tuning, you provide 1-10 examples in the prompt and the model infers the pattern. Modern frontier LLMs are often "good enough" zero-shot, but few-shot prompting still adds 5-20 percentage points of accuracy on harder tasks. Both are forms of in-context learning, where the model uses its frozen weights but adapts behavior based on the context window.

Why it matters

Few-shot and zero-shot learning are the reason LLM adoption is fast. New tasks ship in hours, not the weeks or months a fine-tuning project would require. Engineering organizations should default to prompting first, fine-tune only when data and stability requirements justify it.

Example

A product team builds a sentiment classifier in 30 minutes by prompting GPT-4o with 3 example reviews per class — no labels, no training, no MLOps pipeline.

Related terms

• Prompt Engineering
  Prompt engineering is the practice of designing the inputs to a language model — instructions, examples, role definitions, output formats, and constraints — to reliably produce a desired output without changing the model's weights.
• Fine-tuning
  Fine-tuning is the process of further training a pretrained model on a smaller, task-specific dataset so it specializes in a particular style, domain, or behavior — without retraining from scratch.
• Transfer Learning
  Transfer learning is the practice of using a model trained on one task as the starting point for a related task, dramatically reducing the data and compute required to reach high accuracy on the new task.
• Large Language Model (LLM)
  A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.