InnerVault

lmdeploy

Stale GitHub, InternLM

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Open the source (opens github.com) https://github.com/internlm/lmdeploy

What it is

AI repositories from InternLM.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/InternLM/lmdeploy.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  2. llama-gpt

    AI repositories

    A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

    Stale GitHub, getumbrel 10.9k ★MITUpdated 2 years ago

    git clone https://github.com/getumbrel/llama-gpt.git
  3. Streamer-Sales

    AI repositories

    Streamer-Sales 销冠 —— 卖货主播 LLM 大模型🛒🎁,一个能够根据给定的商品特点从激发用户购买意愿角度出发进行商品解说的卖货主播大模型。🚀⭐内含详细的数据生成流程❗ 📦另外还集成了 LMDeploy 加速推理🚀、RAG检索增强生成 📚、TTS文字转语音🔊、数字人生成 🦸、 Agent 使用网络查询实时信息🌐、ASR 语音转文字🎙️、Vue 生态搭建前端🍍、FastAPI 搭建后端🗝️、Docker-compose 打包部署🐋

    Stale GitHub, PeterH0323 3,788 ★AGPL-3.0Updated 20 months ago

    git clone https://github.com/PeterH0323/Streamer-Sales.git
  4. ipex-llm

    AI repositories

    Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

    Stale Archived GitHub, intel 8,846 ★Apache-2.0Updated 9 months ago 1 note

    git clone https://github.com/intel/ipex-llm.git