diy-llm
Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐,6 项渐进代码作业,掌握 LLM 全栈知识
Open the source (opens github.com) https://github.com/datawhalechina/diy-llm
What it is
AI repositories from datawhalechina.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/datawhalechina/diy-llm.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
VLM-R1
AI repositoriesSolve Visual Understanding with Reinforced VLMs
git clone https://github.com/om-ai-lab/VLM-R1.git -
ms-swift
AI repositoriesUse PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
git clone https://github.com/modelscope/ms-swift.git -
FineEnvs
AI repositoriesFineEnvs — RL Environments 101: building and scaling RL environments in the age of LLMs
git clone https://github.com/adithya-s-k/FineEnvs.git -
oumi
AI repositoriesEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
git clone https://github.com/oumi-ai/oumi.git -
reasoning-from-scratch
AI repositoriesImplement a reasoning LLM in PyTorch from scratch, step by step
git clone https://github.com/rasbt/reasoning-from-scratch.git -
MedicalGPT
AI repositoriesMedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO、GRPO。
git clone https://github.com/shibing624/MedicalGPT.git