InnerVault

engine

Stale GitHub, cactus-compute

2bit quants, kernels and runtime for consumer devices.

Open the source (opens github.com) https://github.com/cactus-compute/engine

What it is

AI repositories from cactus-compute.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/cactus-compute/engine.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. MNN

    AI repositories

    MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

    Indexed GitHub, alibaba 16.5k ★Apache-2.0Updated today

    git clone https://github.com/alibaba/MNN.git
  2. runanywhere-sdks

    AI repositories

    Production ready toolkit to run AI locally

    Stale GitHub, RunanywhereAI 10.3k ★Custom licence (see repository)Updated 12 days ago

    git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
  3. litert-samples

    AI repositories

    LiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.

    Indexed GitHub, google-ai-edge 496 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/google-ai-edge/litert-samples.git
  4. flutter_edge_ai

    AI repositories

    On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.

    Indexed GitHub, DenisovAV 665 ★MITUpdated today

    git clone https://github.com/DenisovAV/flutter_edge_ai.git
  5. BigMoeOnEdge

    AI repositories

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    Indexed GitHub, Helldez 924 ★Apache-2.0Updated today

    git clone https://github.com/Helldez/BigMoeOnEdge.git
  6. Llamatik

    AI repositories

    True on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM inference, Speech-to-Text, Image Generation, Agent Framework and RAG — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.

    Indexed GitHub, llamatik 220 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/llamatik/Llamatik.git