Supervised fine-tuning (SFT)
Show the model example conversations and it learns to answer the same way. This is where almost every fine-tune starts.
When to use it
Use SFT when you can write down good answers: a support assistant that knows your product, a model that always replies in your JSON schema, a writer in your house style. If you can only say which of two answers is better, use preference tuning after SFT.
Data
One conversation per line, in the OpenAI chat format. The model learns to produce the assistant turns. A system message is optional and is learned as part of the context.
python
1{"messages": [{"role": "system", "content": "You are Acme's support assistant."}, {"role": "user", "content": "How do I reset my password?"}, {"role": "assistant", "content": "Open Settings → Security and choose Reset password. We'll email you a link."}]}
Train
SFT is what FastLanguageModel.train runs for the adapter methods (qlora, lora, dora, ia3, prefix).
python
1from datasets import load_dataset2from langtrain import FastLanguageModel34model, tokenizer = FastLanguageModel.from_pretrained(5 "Qwen/Qwen2.5-7B-Instruct",6 load_in_4bit=True,7 max_seq_length=2048,8)9model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32, method="qlora")1011dataset = load_dataset("json", data_files="train.jsonl", split="train")12FastLanguageModel.train(13 model, tokenizer, dataset,14 output_dir="./output",15 num_train_epochs=3,16 learning_rate=2e-4,17)
Settings that matter
- Base model - start from an instruct or chat model of the family you want; it already follows instructions
- num_train_epochs - 1 to 3. More often memorises the examples instead of learning the pattern
- learning_rate - 2e-4 is the default for adapters; lower it if loss jumps around
- max_seq_length - long enough for your longest conversation; longer costs memory
- r - adapter rank, 16 by default. Raise it for harder tasks with plenty of data
Check the result
Keep some conversations out of training and read the model's answers to them before you ship. See Evaluation.