What is Fine-Tuning?

Fine-tuning takes a pre-trained model and continues training it on a smaller, curated dataset to specialize it — for example, on medical Q&A, a company's support tone, or a new language.

Full fine-tuning updates all parameters and is expensive. Parameter-efficient methods like LoRA update only a small set of adapter weights, making fine-tuning feasible on a single GPU.

Fine-tuning changes behavior and style well, but it's a poor way to inject factual knowledge (use RAG for that), and it can degrade the model's general abilities if done carelessly.