Few-Shot Learning

What Is Few-Shot Learning

Few-shot learning is the technique of teaching an LLM a new task by example — at inference time, not by retraining. You write the task description, then paste a handful of correctly-solved input-output pairs, then ask the model to handle a new input the same way. The model picks up the pattern from context. The term was popularized by the 2020 GPT-3 paper “Language Models are Few-Shot Learners” by Brown et al. Use it when zero-shot output isn’t reliable enough and you have a few examples lying around.

How It Works

A typical few-shot prompt looks like:

Translate English to French.

English: Hello, how are you?
French: Bonjour, comment allez-vous?

English: I would like a coffee.
French: Je voudrais un café.

English: Where is the train station?
French:

The model completes the last line by analogy. Three to five examples usually capture most of the gain — past that, returns diminish and you start eating your context window for little benefit.

When to Use It

• Output format is unusual or strict (custom JSON, DSL, niche markup)
• Tone or style matters and is hard to describe in words
• The task is rare in training data
• You don’t have enough labeled data to fine-tune

Few-Shot vs Fine-Tuning

Few-shot lives entirely in the prompt. Cheap to set up, easy to iterate, costs tokens on every request. Fine-tuning bakes the behavior into the weights — higher upfront cost, smaller prompts at inference, harder to change. Start with few-shot. Move to fine-tuning only when prompts get expensive or examples stop fitting in the window.