pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Open the source (opens github.com) https://github.com/pegainfer-project/pegainfer
What it is
AI repositories from pegainfer-project.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/pegainfer-project/pegainfer.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
tt-metal
AI repositories:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
git clone https://github.com/tenstorrent/tt-metal.git -
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
project-maya
AI repositoriesProject Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
git clone https://github.com/mw00/project-maya.git -
TensorSharp
AI repositoriesA native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability
git clone https://github.com/zhongkaifu/TensorSharp.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git