Tutorial: a support assistant
Fine-tune an open chat model on your support conversations, talk to it, check it, and serve it behind an OpenAI-compatible API.
1. Collect conversations
Export a few hundred real support conversations and keep the ones where the answer was right. Write them as chat JSONL, with the same system message on every line so the model learns its role. Keep 10% back for step 4.
python
1{"messages": [{"role": "system", "content": "You are Acme's support assistant. Be brief and friendly."}, {"role": "user", "content": "My order hasn't arrived."}, {"role": "assistant", "content": "Sorry about that! Send me your order number and I'll check where it is."}]}
2. Train
Fine-tune an instruct model with QLoRA. On a 16 GB GPU, a 7B or 8B model works.
python
1from datasets import load_dataset2from langtrain import FastLanguageModel34model, tokenizer = FastLanguageModel.from_pretrained(5 "Qwen/Qwen2.5-7B-Instruct",6 load_in_4bit=True,7 max_seq_length=2048,8)9model = FastLanguageModel.get_peft_model(model, r=16, method="qlora")1011data = load_dataset("json", data_files="support.jsonl", split="train").train_test_split(test_size=0.1, seed=42)12FastLanguageModel.train(model, tokenizer, data["train"], output_dir="./output", num_train_epochs=2)
3. Talk to it
Format the conversation with the model's own chat template, then generate the next answer.
python
1FastLanguageModel.for_inference(model, tokenizer)23messages = [4 {"role": "system", "content": "You are Acme's support assistant. Be brief and friendly."},5 {"role": "user", "content": "Can I change my delivery address?"},6]7prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)8print(FastLanguageModel.generate(model, tokenizer, prompt, max_new_tokens=200))
4. Check it
Run the held-back conversations and read the answers next to the real ones. Look for wrong facts, the wrong tone, and promises your team wouldn't make. If it goes wrong in a pattern, fix the data and train again. See Evaluation.
python
1for row in data["test"].select(range(20)):2 question = row["messages"][:-1]3 prompt = tokenizer.apply_chat_template(question, tokenize=False, add_generation_prompt=True)4 print("MODEL:", FastLanguageModel.generate(model, tokenizer, prompt, max_new_tokens=200))5 print("REAL: ", row["messages"][-1]["content"], "\n")
5. Save and serve
Merge the adapter and save standard weights, then serve them with vLLM. Any OpenAI SDK client can call it by pointing base_url at your server.
python
1FastLanguageModel.save_pretrained(model, tokenizer, "./support-assistant", merge_before_save=True)23# in a shell4pip install vllm5vllm serve ./support-assistant
6. Make it better
Once it's in use, collect pairs of a better and a worse answer for the same question, and run preference tuning with DPO to sharpen tone and judgement.