InnerVault

SwiftLM

IndexedGitHub, SharpAI

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

Open the source (opens github.com) https://github.com/sharpai/swiftlm

What it is

AI repositories from SharpAI.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/SharpAI/SwiftLM.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. llm-action

    AI repositories

    本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)

    Stale GitHub, liguodongiot 25.2k ★Apache-2.0Updated 33 days ago

    git clone https://github.com/liguodongiot/llm-action.git
  2. BentoML

    AI repositories

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Stale GitHub, bentoml 8,935 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/bentoml/BentoML.git
  3. llm-d

    AI repositories

    State of the art inference performance with modern accelerators on Kubernetes

    Indexed GitHub, llm-d 5,274 ★Apache-2.0Updated today

    git clone https://github.com/llm-d/llm-d.git
  4. PowerInfer

    AI repositories

    High-speed Large Language Model Serving for Local Deployment

    Stale GitHub, Tiiny-AI 9,828 ★MITUpdated 6 months ago

    git clone https://github.com/Tiiny-AI/PowerInfer.git
  5. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  6. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git