vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
Open the source (opens github.com) https://github.com/vllm-project/vllm-ascend
What it is
AI repositories from vllm-project.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/vllm-project/vllm-ascend.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
llm-action
AI repositories本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)
git clone https://github.com/liguodongiot/llm-action.git -
rtp-llm
AI repositoriesRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
git clone https://github.com/alibaba/rtp-llm.git -
BentoML
AI repositoriesThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
git clone https://github.com/bentoml/BentoML.git -
superduper
AI repositoriesSuperduper: End-to-end framework for building custom AI applications and agents.
git clone https://github.com/superduper-io/superduper.git -
OpenLLM
AI repositoriesRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
git clone https://github.com/bentoml/OpenLLM.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git