halogen-flash-server
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
Open the source (opens github.com) https://github.com/peonist-ai/halogen-flash-server
What it is
AI repositories from peonist-ai.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/peonist-ai/halogen-flash-server.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
gufo
AI repositoriesThe open-source AI engine for AMD Strix Halo. Text, embeddings, speech, image and video models in one binary, with OpenAI-compatible APIs, caching for coding agents and model extensions. Linux and Windows.
git clone https://github.com/gufo-org/gufo.git -
ml-engineering
AI repositoriesMachine Learning Engineering Open Book
git clone https://github.com/stas00/ml-engineering.git -
FastFlowLM
AI repositoriesRun LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.
git clone https://github.com/ROCm/FastFlowLM.git -
llm-d
AI repositoriesState of the art inference performance with modern accelerators on Kubernetes
git clone https://github.com/llm-d/llm-d.git -
LMCache
AI repositoriesLMCache: Supercharge Your LLM with the Fastest KV Cache Layer
git clone https://github.com/LMCache/LMCache.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git