InnerVault

diy-llm

Stale GitHub, datawhalechina

Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐,6 项渐进代码作业,掌握 LLM 全栈知识

Open the source (opens github.com) https://github.com/datawhalechina/diy-llm

What it is

AI repositories from datawhalechina.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/datawhalechina/diy-llm.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. VLM-R1

    AI repositories

    Solve Visual Understanding with Reinforced VLMs

    Stale GitHub, om-ai-lab 6,034 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/om-ai-lab/VLM-R1.git
  2. ms-swift

    AI repositories

    Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

    Indexed GitHub, modelscope 16.1k ★Apache-2.0Updated 3 days ago

    git clone https://github.com/modelscope/ms-swift.git
  3. FineEnvs

    AI repositories

    FineEnvs — RL Environments 101: building and scaling RL environments in the age of LLMs

    Stale GitHub, adithya-s-k 526 ★Apache-2.0Updated 25 days ago

    git clone https://github.com/adithya-s-k/FineEnvs.git
  4. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/oumi-ai/oumi.git
  5. reasoning-from-scratch

    AI repositories

    Implement a reasoning LLM in PyTorch from scratch, step by step

    Stale GitHub, rasbt 5,404 ★Apache-2.0Updated 45 days ago

    git clone https://github.com/rasbt/reasoning-from-scratch.git
  6. MedicalGPT

    AI repositories

    MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO、GRPO。

    Stale GitHub, shibing624 5,861 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/shibing624/MedicalGPT.git