omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Open the source (opens github.com) https://github.com/jundot/omlx
What it is
AI repositories from jundot.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/jundot/omlx.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
vllm-mlx
AI repositoriesHigh-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
git clone https://github.com/waybarrios/vllm-mlx.git -
mlx-serve
AI repositoriesNative LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
git clone https://github.com/ddalcu/mlx-serve.git -
SwiftLM
AI repositories⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
git clone https://github.com/SharpAI/SwiftLM.git -
vllm-metal
AI repositoriesCommunity maintained hardware plugin for vLLM on Apple Silicon
git clone https://github.com/vllm-project/vllm-metal.git -
SiliconScope
AI repositoriesSudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking
git clone https://github.com/kennss/SiliconScope.git -
turbo-fieldfare
AI repositoriesGemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
git clone https://github.com/drumih/turbo-fieldfare.git