InnerVault

omlx

IndexedGitHub, jundot

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Open the source (opens github.com) https://github.com/jundot/omlx

What it is

AI repositories from jundot.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/jundot/omlx.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. vllm-mlx

    AI repositories

    High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

    Stale GitHub, waybarrios 1,646 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/waybarrios/vllm-mlx.git
  2. mlx-serve

    AI repositories

    Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

    Indexed GitHub, ddalcu 2,475 ★Custom licence (see repository)Updated today

    git clone https://github.com/ddalcu/mlx-serve.git
  3. SwiftLM

    AI repositories

    ⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

    Indexed GitHub, SharpAI 799 ★MITUpdated today

    git clone https://github.com/SharpAI/SwiftLM.git
  4. vllm-metal

    AI repositories

    Community maintained hardware plugin for vLLM on Apple Silicon

    Stale GitHub, vllm-project 1,987 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/vllm-project/vllm-metal.git
  5. SiliconScope

    AI repositories

    Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

    Stale GitHub, kennss 1,013 ★MITUpdated 14 days ago

    git clone https://github.com/kennss/SiliconScope.git
  6. turbo-fieldfare

    AI repositories

    Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

    Stale GitHub, drumih 6,930 ★Apache-2.0Updated 33 days ago

    git clone https://github.com/drumih/turbo-fieldfare.git