Langtrain Logo
Langtrain Docs
GuidesAPI referenceSDKs
GitHubOpen the app

Search documentation

Jump to any docs page

  • REST API reference
  • Python SDK
  • Node.js and TypeScript
GitHub

Python SDK

langtrain-ai is the Python package for Langtrain: fine-tune open models on your own GPU, check a dataset before you train, and run AdaptiveRank, our LoRA recipe.

  • Python 3.8+
  • Runs locally
  • MIT

Install

Install the package with its training extras and Langtune, on a machine with an NVIDIA GPU. For vision models, add langtrain-ai[vision].
python
1pip install "langtrain-ai[train]" langtune
2

3python -c "import langtrain; print(langtrain.__version__)"

Fine-tune a text model

FastLanguageModel loads an open model (4-bit by default), adds an adapter and trains it with TRL. Chat-format JSONL, a messages array per line, works as is.
python
1from datasets import load_dataset
2from langtrain import FastLanguageModel
3

4model, tokenizer = FastLanguageModel.from_pretrained(
5    "meta-llama/Llama-3.1-8B-Instruct",
6    load_in_4bit=True,
7    max_seq_length=2048,
8)
9model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32, method="qlora")
10

11dataset = load_dataset("json", data_files="train.jsonl", split="train")
12FastLanguageModel.train(model, tokenizer, dataset, output_dir="./output", num_train_epochs=3)

Methods

Pass method= to get_peft_model:
  • –lora, qlora (the default), dora, ia3, prefix - adapter methods on a frozen model
  • –dpo, orpo, simpo - learn from pairs of a better and a worse answer (prompt, chosen, rejected columns)
  • –kto - learn from single answers marked good or bad
See Preference tuning for the data each one needs.

Try it and save it

Generate from the trained model, then merge the adapter into the base weights. The folder is a standard Hugging Face model that llama.cpp, vLLM and Ollama can run.
python
1FastLanguageModel.for_inference(model, tokenizer)
2print(FastLanguageModel.generate(model, tokenizer, "Summarise our refund policy.", max_new_tokens=200))
3

4FastLanguageModel.save_pretrained(model, tokenizer, "./my-model", merge_before_save=True)

Check a dataset before training

DatasetIntelligence reads a JSONL or CSV file and reports what kind of task it holds, how many examples there are, how many are duplicates, and a suggested model and LoRA rank. Without an API key set, it runs entirely on your machine.
python
1from langtrain import DatasetIntelligence
2

3report = DatasetIntelligence.analyze("train.jsonl")
4print(report.task_type, report.sample_count, report.dedup_ratio)
5print(report.recommended_model, report.lora_rank)
6print(report.enhancement_suggestions)

AdaptiveRank

Our own LoRA trainer. Adapters start from the strongest directions of the base weights (an SVD) instead of random noise, and similar examples are batched together. See Research.
python
1from langtrain import AdaptiveRankTrainer, AdaptiveRankConfig
2

3config = AdaptiveRankConfig(initial_rank=16, num_epochs=3, output_dir="./output")
4trainer = AdaptiveRankTrainer(model_name="meta-llama/Llama-3.1-8B-Instruct", config=config)
5trainer.train(dataset)

Vision models

FastVisionModel does the same for vision-language models. See Fine-tune vision models.
PreviousREST API referenceNextNode.js and TypeScript