engine
2bit quants, kernels and runtime for consumer devices.
Open the source (opens github.com) https://github.com/cactus-compute/engine
What it is
AI repositories from cactus-compute.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/cactus-compute/engine.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
MNN
AI repositoriesMNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
git clone https://github.com/alibaba/MNN.git -
runanywhere-sdks
AI repositoriesProduction ready toolkit to run AI locally
git clone https://github.com/RunanywhereAI/runanywhere-sdks.git -
litert-samples
AI repositoriesLiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.
git clone https://github.com/google-ai-edge/litert-samples.git -
flutter_edge_ai
AI repositoriesOn-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.
git clone https://github.com/DenisovAV/flutter_edge_ai.git -
BigMoeOnEdge
AI repositoriesRun MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
git clone https://github.com/Helldez/BigMoeOnEdge.git -
Llamatik
AI repositoriesTrue on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM inference, Speech-to-Text, Image Generation, Agent Framework and RAG — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.
git clone https://github.com/llamatik/Llamatik.git