nano-vllm
Nano vLLM
Open the source (opens github.com) https://github.com/geeeekexplorer/nano-vllm
What it is
AI repositories from GeeeekExplorer.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/GeeeekExplorer/nano-vllm.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
openvino
AI repositoriesOpenVINO™ is an open source toolkit for optimizing and deploying AI inference
git clone https://github.com/openvinotoolkit/openvino.git -
sie
AI repositoriesOpen-source inference server and production cluster for all the models your agent needs.
git clone https://github.com/superlinked/sie.git -
LightLLM
AI repositoriesLightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
git clone https://github.com/ModelTC/LightLLM.git -
nncf
AI repositoriesNeural Network Compression Framework for enhanced OpenVINO™ inference
git clone https://github.com/openvinotoolkit/nncf.git -
transformers
AI repositories🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
git clone https://github.com/huggingface/transformers.git -
llm-foundry
AI repositoriesLLM training code for Databricks foundation models
git clone https://github.com/mosaicml/llm-foundry.git