InnerVault

rtp-llm

IndexedGitHub, alibaba

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

Open the source (opens github.com) https://github.com/alibaba/rtp-llm

What it is

AI repositories from alibaba.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/alibaba/rtp-llm.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  2. airllm

    AI repositories

    AirLLM 2.8T LLM inference with single 4GB GPU

    Indexed GitHub, lyogavin 42.5k ★Apache-2.0Updated today

    git clone https://github.com/lyogavin/airllm.git
  3. sglang

    AI repositories

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Indexed GitHub, sgl-project 38.9k ★Apache-2.0Updated today

    git clone https://github.com/sgl-project/sglang.git
  4. aikit

    AI repositories

    🏗️ Fine-tune, build, and deploy open-source LLMs easily!

    Indexed GitHub, kaito-project 544 ★MITUpdated 2 days ago

    git clone https://github.com/kaito-project/aikit.git
  5. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/oumi-ai/oumi.git
  6. LightLLM

    AI repositories

    LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

    Indexed GitHub, ModelTC 4,362 ★Apache-2.0Updated today

    git clone https://github.com/ModelTC/LightLLM.git