PowerInfer
High-speed Large Language Model Serving for Local Deployment
Open the source (opens github.com) https://github.com/tiiny-ai/powerinfer
What it is
AI repositories from Tiiny-AI.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/Tiiny-AI/PowerInfer.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
picollm
AI repositoriesOn-device LLM Inference Powered by X-Bit Quantization
git clone https://github.com/Picovoice/picollm.git -
litgpt
AI repositories20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
git clone https://github.com/Lightning-AI/litgpt.git -
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
wllama
AI repositoriesWebAssembly binding for llama.cpp - Enabling on-browser LLM inference
git clone https://github.com/ngxson/wllama.git -
GenerativeAIExamples
AI repositoriesGenerative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
git clone https://github.com/NVIDIA/GenerativeAIExamples.git -
Chinese-LLaMA-Alpaca
AI repositories中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca.git