LLM fine-tuning
LlamaFactory is a unified, efficient framework for fine-tuning 100+ large language and vision models via CLI/Web UI, supporting PEFT/QLoRA/RLHF.
Same problem, same approach. Swapping one for another is a config change, not a rewrite.
choose when you need even broader model coverage (600+ LLMs, 300+ MLLMs) with tight ModelScope ecosystem integration.
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).
choose when you want a simple open-source fine-tuning library with dataset preprocessing and a direct CLI interface.
Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
choose when you want a full train/evaluate/deploy orchestration framework instead of fine-tuning only.
Easily fine-tune, evaluate and deploy Qwen, Gemma, or any open weight LLM!
Solves the same problem with a different architecture or at a different layer. Expect to rewrite the integration.
choose when you want a lightweight, self-contained PyTorch codebase with pretrain/finetune recipes for a curated set of 20+ open models.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
choose when you need NVIDIA-native distributed training with HuggingFace models and can bypass friendly CLI/UI abstractions.
🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
choose when you want extremely fast LoRA/QLoRA memory efficiency via custom kernels rather than a broad multi-method trainer.
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.
choose when you need memory-efficient layer streaming to train larger models on severely limited GPUs.
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
An earlier or more limited way to do the job. Still the right call when you need something small, proven, or CPU-only.
a legacy research codebase for instruction-tuning/CoT-style tuning of early LLaMA models; pick it for reproducing paper experiments.
We unified the interfaces of instruction-tuning data (e.g., CoT data), multiple LLMs and parameter-efficient methods (e.g., lora, p-tuning) together for easy use. We welcome open-source enthusiasts to initiate any meaningful PR on this repo and integrate as many LLM related technologies as possible. 我们打造了方便研究人员上手和使用大模型等微调平台，我们欢迎开源爱好者发起任何有意义的pr！
narrow, paper-specific code for tuning LLaMA with the LLaMA-Adapter method; use it only for reproducing that approach.
[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters
a legacy minimal LLaMA implementation with LoRA/adapter training; use it only if you need LLaMA-specific bare-metal research code without HuggingFace.
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Does the same job, but ships as an app. Useful to a person, not swappable into a codebase.
choose when you prefer a no-code desktop GUI for fine-tuning rather than a library/CLI you integrate into your code.
H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/
choose when you want a simple, user-facing UI for fine-tuning models instead of coding with a library.
Simple UI for LLM Model Finetuning
These projects were analysed and named LlamaFactory among their alternatives. The relationship is not symmetric — how LlamaFactory rates them is a separate judgement, made when LlamaFactory is analysed in its own right.
calls LlamaFactory “Choose when you want a HuggingFace-centric unified SFT/DPO/GRPO harness with a broader plug-in ecosystem and a different UI workflow.”
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).
calls LlamaFactory “Choose it for a battle-tested, model-heavy fine-tuning library that is simpler and more mature for SFT/DPO/GRPO if you do not need Oumi's eval/deploy layer.”
Easily fine-tune, evaluate and deploy Qwen, Gemma, or any open weight LLM!
calls LlamaFactory “Choose for web/YAML-based PEFT and full-parameter training of 100+ models if you need broad coverage and don't require Unsloth's optimized kernels.”
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.
calls LlamaFactory “PyTorch/HF fine-tuning framework for 100+ LLMs; choose for PEFT/fine-tuning workflows outside Paddle.”
PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.