InnerVault

OlliteRT

Stale GitHub, NightMean

Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source

Open the source (opens github.com) https://github.com/nightmean/ollitert

What it is

AI repositories from NightMean.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/NightMean/OlliteRT.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. PolyStrata

    AI repositories

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    Stale GitHub, VecSzn 722 ★MITUpdated 16 days ago

    git clone https://github.com/VecSzn/PolyStrata.git
  2. litert-samples

    AI repositories

    LiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.

    Indexed GitHub, google-ai-edge 496 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/google-ai-edge/litert-samples.git
  3. flutter_edge_ai

    AI repositories

    On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.

    Indexed GitHub, DenisovAV 665 ★MITUpdated today

    git clone https://github.com/DenisovAV/flutter_edge_ai.git
  4. BigMoeOnEdge

    AI repositories

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    Indexed GitHub, Helldez 924 ★Apache-2.0Updated today

    git clone https://github.com/Helldez/BigMoeOnEdge.git
  5. hermes-official-plugins

    AI repositories

    ☤ The official plugins for Hermes Agent, built and maintained by Nous Research: Home Assistant, Spotify, Claude Subscription DirectSDK and more, one install away from the Hermes plugin catalog.

    Indexed GitHub, NousResearch 726 ★MITUpdated yesterday

    git clone https://github.com/NousResearch/hermes-official-plugins.git
  6. mlx-serve

    AI repositories

    Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

    Indexed GitHub, ddalcu 2,475 ★Custom licence (see repository)Updated today

    git clone https://github.com/ddalcu/mlx-serve.git