chitu
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
Open the source (opens github.com) https://github.com/thu-pacman/chitu
What it is
AI repositories from thu-pacman.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/thu-pacman/chitu.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
ai-infra-book
AI repositories《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
git clone https://github.com/bojieli/ai-infra-book.git -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
ome
AI repositoriesOpen Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
git clone https://github.com/ome-projects/ome.git -
kvcached
AI repositoriesVirtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
git clone https://github.com/ovg-project/kvcached.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git