llm-d
State of the art inference performance with modern accelerators on Kubernetes
Open the source (opens github.com) https://github.com/llm-d/llm-d
What it is
AI repositories from llm-d.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/llm-d/llm-d.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
awsome-distributed-ai
AI repositoriesBest practices, reference architectures, and examples for distributed AI training and inference on AWS.
git clone https://github.com/awslabs/awsome-distributed-ai.git -
paddler
AI repositoriesOpen-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
git clone https://github.com/intentee/paddler.git -
pooled
AI repositoriesPeer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
git clone https://github.com/Nehanth/pooled.git -
serve
AI repositories☁️ Build multimodal AI applications with cloud-native stack
git clone https://github.com/jina-ai/serve.git -
llm-datasets
AI repositoriesCurated list of datasets and tools for post-training.
git clone https://github.com/mlabonne/llm-datasets.git