Fine-Tuning LLMs: When to Do It and When Not To

Note: AI Weekender has moved. New posts are published at ai-weekender.com, and this Substack is now an archive.

To keep receiving weekly issues, please subscribe at ai-weekender.com instead of here.

If you’ve been following AI engineering, you’ll have seen fine-tuning showing up everywhere, from job postings to Hugging Face repos to conference talks.

Fine-tuning is worth knowing if you’re serious about AI engineering or building an AI product. It’s a strong lever when you need stable behavior (i.e. tone, labels, and output habits) that prompts alone won’t hold in production.

What Fine-tuning Actually Changes

A base model such as Llama, Mistral, Qwen, or Gemma is trained on large text corpora during pre-training. These pre-trained models already handle general language and reasoning.

Fine-tuning is post-training on your dataset so the model’s default behavior shifts toward your task. There are two ways to update weights during supervised fine-tuning (SFT):

• Full fine-tuning: update all weights. This is expensive in both memory and compute, so it’s uncommon for most product teams.

• Parameter Efficient Fine-Tuning (PEFT): update only a small set of added parameters on top of a frozen base model. This keeps memory and compute low enough to run on one GPU or a laptop, rather than needing a multi-node cluster.

What Fine-tuning is Useful For

In production, fine-tuning usually shows up when you need the model to adapt to specific behaviors more reliably. Below are the three use cases I see most often with fine-tuning.

1. Customize to your Domain Categories

Fine-tuning is useful when you need the model to predict your taxonomy, not a generic label. You train on input–output pairs where the output is the category specific to your organization.

For example, a common use case in payments and risk ops is to fine-tune on alerts your team has already labeled. It would take:

Input = alert description

Output = one of monitor, escalate, false_positive

Through fine-tuning, SFT learns to map text to your defined labels on new cases.

2. Write in your style and voice

Fine-tuning is useful when you need a stable voice and register that holds across long conversations and varied user inputs. Here are some real-world examples:

• Legal drafting: Fine-tune on your firm’s approved agreements so outputs match house style and standard clause patterns.

• Customer Support: Write consistent replies in a customer-friendly voice.

• Dialects: Adapt to colloquial or spoken dialects, which is more relevant in certain languages where regional differences are larger than in English.

3. Standardize output format

Fine-tuning is useful when a downstream service depends on a strict output contract on every completion, such as fixed JSON keys, valid enums, or a specific layout.

What to take away

Fine-tuning is post-training on your labeled dataset so the model’s defaults adapt to your task or output style.

It remains relevant in 2026 when you need the model outputs to adapt to:

• Your categories rather than generic labels.

• A certain voice or register at scale that prompts cannot maintain

• Certain formats that downstream parsers require.

Next week, we’ll dive deeper into fine-tuning methodologies and which to use in different scenarios.

Note: AI Weekender has moved. New posts are published at ai-weekender.com, and this Substack is now an archive.

To keep receiving weekly issues, please subscribe at ai-weekender.com instead of here.