InnerVault

inference

IndexedGitHub, xorbitsai

Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.

Open the source (opens github.com) https://github.com/xorbitsai/inference

What it is

AI repositories from xorbitsai.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/xorbitsai/inference.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. sglang

    AI repositories

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Indexed GitHub, sgl-project 38.9k ★Apache-2.0Updated today

    git clone https://github.com/sgl-project/sglang.git
  2. transformers

    AI repositories

    🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

    Indexed GitHub, huggingface 169.0k ★Apache-2.0Updated today

    git clone https://github.com/huggingface/transformers.git
  3. PolyStrata

    AI repositories

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    Stale GitHub, VecSzn 722 ★MITUpdated 16 days ago

    git clone https://github.com/VecSzn/PolyStrata.git
  4. TensorSharp

    AI repositories

    A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

    Stale GitHub, zhongkaifu 675 ★BSD-3-ClauseUpdated 4 days ago

    git clone https://github.com/zhongkaifu/TensorSharp.git
  5. self-llm

    AI repositories

    《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调(全参数/Lora)、部署国内外开源大模型(LLM)/多模态大模型(MLLM)教程

    Stale GitHub, datawhalechina 32.5k ★Apache-2.0Updated 2 months ago

    git clone https://github.com/datawhalechina/self-llm.git
  6. unsloth

    AI repositories

    Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

    Indexed GitHub, unslothai 79.6k ★Apache-2.0Updated today

    git clone https://github.com/unslothai/unsloth.git