Langtrain Logo
Langtrain Docs
GuidesAPI referenceSDKs
GitHubOpen the app

Search documentation

Jump to any docs page

  • What is Langtrain
  • Quick start
  • Installation
GitHub

Quick start

Install the SDK, write a small dataset, fine-tune an open model on your own GPU and save weights you can run anywhere.

  • Python 3.8+
  • NVIDIA GPU

  1. 1. Install

    Install the SDK with its training extras and Langtune, inside a virtual environment, on a machine with an NVIDIA GPU.

    bash
    1pip install "langtrain-ai[train]" langtune

  2. 2. Prepare your data

    Create a .jsonl file. Each line is one conversation: a messages array in the same format as the OpenAI chat API.

    json
    1{"messages": [{"role": "user", "content": "What is Langtrain?"}, {"role": "assistant", "content": "A platform for fine-tuning open models."}]}
    2{"messages": [{"role": "user", "content": "How do I start?"}, {"role": "assistant", "content": "Write a JSONL file like this one, then train."}]}

  3. 3. Train

    Load the model in 4-bit, add a QLoRA adapter and train. Only the adapter learns; the base model stays frozen, so an 8B model fits on a single 24 GB GPU.

    python
    1from datasets import load_dataset
    2from langtrain import FastLanguageModel
    3

    4model, tokenizer = FastLanguageModel.from_pretrained(
    5    "meta-llama/Llama-3.1-8B-Instruct",
    6    load_in_4bit=True,
    7)
    8model = FastLanguageModel.get_peft_model(model, r=16, method="qlora")
    9

    10dataset = load_dataset("json", data_files="training_data.jsonl", split="train")
    11FastLanguageModel.train(model, tokenizer, dataset, output_dir="./output")

  4. 4. Try it and keep the weights

    Ask it something, then merge the adapter into the base model. The result is a standard Hugging Face model folder that llama.cpp, vLLM and Ollama can run.

    python
    1FastLanguageModel.for_inference(model, tokenizer)
    2print(FastLanguageModel.generate(model, tokenizer, "How do I start?"))
    3

    4FastLanguageModel.save_pretrained(model, tokenizer, "./my-model", merge_before_save=True)

    Next steps

    • Run your model anywhere
    • Tune LoRA and QLoRA settings