InnerVault

PowerInfer

Stale GitHub, Tiiny-AI

High-speed Large Language Model Serving for Local Deployment

Open the source (opens github.com) https://github.com/tiiny-ai/powerinfer

What it is

AI repositories from Tiiny-AI.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/Tiiny-AI/PowerInfer.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. picollm

    AI repositories

    On-device LLM Inference Powered by X-Bit Quantization

    Indexed GitHub, Picovoice 320 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Picovoice/picollm.git
  2. litgpt

    AI repositories

    20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

    Stale GitHub, Lightning-AI 13.7k ★Apache-2.0Updated 34 days ago

    git clone https://github.com/Lightning-AI/litgpt.git
  3. lmdeploy

    AI repositories

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    Stale GitHub, InternLM 8,136 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/InternLM/lmdeploy.git
  4. wllama

    AI repositories

    WebAssembly binding for llama.cpp - Enabling on-browser LLM inference

    Stale GitHub, ngxson 1,363 ★MITUpdated 10 days ago

    git clone https://github.com/ngxson/wllama.git
  5. GenerativeAIExamples

    AI repositories

    Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

    Stale GitHub, NVIDIA 4,210 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/NVIDIA/GenerativeAIExamples.git
  6. Chinese-LLaMA-Alpaca

    AI repositories

    中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

    Stale GitHub, ymcui 18.9k ★Apache-2.0Updated 7 months ago

    git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca.git