InnerVault

halogen-flash-server

Stale GitHub, peonist-ai

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

Open the source (opens github.com) https://github.com/peonist-ai/halogen-flash-server

What it is

AI repositories from peonist-ai.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/peonist-ai/halogen-flash-server.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. gufo

    AI repositories

    The open-source AI engine for AMD Strix Halo. Text, embeddings, speech, image and video models in one binary, with OpenAI-compatible APIs, caching for coding agents and model extensions. Linux and Windows.

    Indexed GitHub, gufo-org 1,167 ★MITUpdated 2 days ago

    git clone https://github.com/gufo-org/gufo.git
  2. ml-engineering

    AI repositories

    Machine Learning Engineering Open Book

    Stale GitHub, stas00 19.7k ★CC-BY-SA-4.0Updated 12 days ago

    git clone https://github.com/stas00/ml-engineering.git
  3. FastFlowLM

    AI repositories

    Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

    Indexed GitHub, ROCm 2,053 ★Custom licence (see repository)Updated today

    git clone https://github.com/ROCm/FastFlowLM.git
  4. llm-d

    AI repositories

    State of the art inference performance with modern accelerators on Kubernetes

    Indexed GitHub, llm-d 5,274 ★Apache-2.0Updated today

    git clone https://github.com/llm-d/llm-d.git
  5. LMCache

    AI repositories

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    Indexed GitHub, LMCache 12.4k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/LMCache/LMCache.git
  6. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git