tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
Open the source (opens github.com) https://github.com/lightseekorg/tokenspeed
What it is
AI repositories from lightseekorg.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/lightseekorg/tokenspeed.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
sglang
AI repositoriesSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.git -
Mooncake
AI repositoriesMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
git clone https://github.com/kvcache-ai/Mooncake.git -
smg
AI repositoriesEngine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
git clone https://github.com/smg-project/smg.git -
openlake
AI repositoriesOpenLake is a high performance storage engine for efficient LLM inference and GPU Training
git clone https://github.com/openlake-project/openlake.git -
dsh-Go
AI repositoriesdsh-go — DeepSeek Harness 装进口袋:手机本机运行,也能遥控你的电脑。
git clone https://github.com/ccch1mneyyy/dsh-Go.git