LLM training from scratch / educational
Train a small 64M-parameter LLM from scratch in PyTorch, with the complete training pipeline (pretraining, SFT, LoRA, RLHF) and educational explanations.
Same problem, same approach. Swapping one for another is a config change, not a rewrite.
Choose this if you want a focused step-by-step reasoning-model implementation with recent updates, rather than the broader minimind pipeline.
Implement a reasoning LLM in PyTorch from scratch, step by step
Choose this if you want an extremely beginner-friendly, fully commented from-scratch GPT with an MIT license.
Build a modern LLM from scratch. Every line commented. Explained like we are five.
Choose this if you want a concise end-to-end pipeline that includes data download and text generation with minimal dependencies.
A straightforward method for training your LLM, from downloading data to generating text.
Solves the same problem with a different architecture or at a different layer. Expect to rewrite the integration.
Choose this if you prefer a Chinese-language interactive notebook tutorial over a single cohesive training project.
π δ»ιΆεΌε§ζ建倧樑ε
Choose this if you want a production-grade framework to pretrain/finetune larger LLMs at scale instead of educational from-scratch code.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
An earlier or more limited way to do the job. Still the right call when you need something small, proven, or CPU-only.
Choose this if you want Karpathy's classic from-scratch course, but note it is archived and outdated.
LLM101n: Let's build a Storyteller
These projects were analysed and named minimind among their alternatives. The relationship is not symmetric β how minimind rates them is a separate judgement, made when minimind is analysed in its own right.
calls minimind βA minimal Apache-2.0 64M LLM from scratch that trains in hours, for a quick simple codebase.β
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
calls minimind βNarrow hands-on training of a tiny LLM; choose for a quick small-scale training exercise.β
Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
calls minimind βlightweight code to train a tiny LLM on a single GPU, not for large-scale useβ
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
calls minimind βEducational from-scratch training for a single tiny 64M model; valid only for small/learning scenarios.β
PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.