Fine-tuning: how AI is taught special skills

Fine-tuning is a process where an already trained large model is trained a bit more on a small, specialised dataset so that it does a specific task better or writes in the right style.

Fine-tuning as a short internship for an already trained model

In plain words

Imagine a school graduate who already knows languages, maths and history. A café hires them and spends two weeks teaching them its own drinks and rules. Nothing has to be learned from scratch - just a bit of sharpening for the job. Fine-tuning is those two weeks of on-the-job training for a neural network.

How it works

You take a ready-made base model that can already do a lot.
You prepare a set of examples: a request and the desired answer in the right style or topic.
The model is trained a little on these examples, changing some of its parameters.
To save resources, often only small add-on "adapters" are changed (the LoRA method).
The result is a model that handles a narrow task better than the base one.

A small adapter changes the style of the base model

Where you'll see it

Companies use fine-tuning so a support bot answers "their way", so a model understands medical or legal terms, or so an image generator draws in a particular style or a specific character. Fine-tuned open models are popular among enthusiasts.

Does it matter when buying

As a regular user, you will rarely need it. But it is good to know that many tasks can be solved with a good prompt with examples, no fine-tuning needed. And if you see a service that says "train AI on your photos", check what happens to your photos after training.

Myths

Myth: Fine-tuning gives the model new knowledge about recent events. Reality: For knowledge, retrieval-augmented generation (RAG) works better. Fine-tuning mostly affects style and skills.
Myth: Fine-tuning needs a supercomputer. Reality: Small models can be fine-tuned on a single powerful graphics card or in an inexpensive cloud service.

FAQ

How is fine-tuning different from a prompt?

A prompt is an in-the-moment hint; fine-tuning changes the model itself.

What is LoRA?

A way to fine-tune only a small add-on part of a model - faster and cheaper.

Can I fine-tune a model on my own photos?

Yes, image generators support this. But think about your privacy.