apple-silicon-llm-bench
Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.
Open the source (opens github.com) https://github.com/john-rocky/apple-silicon-llm-bench
What it is
Datasets & benchmarks from john-rocky.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/john-rocky/apple-silicon-llm-bench.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
mac-ml-speed-test
Datasets & benchmarksA few quick scripts focused on testing TensorFlow/PyTorch/Llama 2 on macOS.
git clone https://github.com/mrdbourke/mac-ml-speed-test.git -
ollama-benchmark
Datasets & benchmarksLLM Benchmark for Throughput via Ollama (Local LLMs)
git clone https://github.com/aidatatools/ollama-benchmark.git -
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
AI_Diplomacy
Datasets & benchmarksFrontier Models playing the board game Diplomacy.
git clone https://github.com/GoodStartLabs/AI_Diplomacy.git -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git