InnerVault

llm-d

IndexedGitHub, llm-d

State of the art inference performance with modern accelerators on Kubernetes

Open the source (opens github.com) https://github.com/llm-d/llm-d

What it is

AI repositories from llm-d.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/llm-d/llm-d.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  2. awsome-distributed-ai

    AI repositories

    Best practices, reference architectures, and examples for distributed AI training and inference on AWS.

    Indexed GitHub, awslabs 498 ★MIT-0Updated 2 days ago

    git clone https://github.com/awslabs/awsome-distributed-ai.git
  3. paddler

    AI repositories

    Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

    Stale GitHub, intentee 1,690 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/intentee/paddler.git
  4. pooled

    AI repositories

    Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.

    Stale GitHub, Nehanth 589 ★MITUpdated 31 days ago

    git clone https://github.com/Nehanth/pooled.git
  5. serve

    AI repositories

    ☁️ Build multimodal AI applications with cloud-native stack

    Stale GitHub, jina-ai 21.9k ★Apache-2.0Updated 20 months ago

    git clone https://github.com/jina-ai/serve.git
  6. llm-datasets

    AI repositories

    Curated list of datasets and tools for post-training.

    Stale GitHub, mlabonne 4,809 ★Updated 6 months ago

    git clone https://github.com/mlabonne/llm-datasets.git