rtp-llm
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Open the source (opens github.com) https://github.com/alibaba/rtp-llm
What it is
AI repositories from alibaba.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/alibaba/rtp-llm.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
airllm
AI repositoriesAirLLM 2.8T LLM inference with single 4GB GPU
git clone https://github.com/lyogavin/airllm.git -
sglang
AI repositoriesSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.git -
aikit
AI repositories🏗️ Fine-tune, build, and deploy open-source LLMs easily!
git clone https://github.com/kaito-project/aikit.git -
oumi
AI repositoriesEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
git clone https://github.com/oumi-ai/oumi.git -
LightLLM
AI repositoriesLightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
git clone https://github.com/ModelTC/LightLLM.git