InnerVault

apple-silicon-llm-bench

IndexedGitHub, john-rocky

Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

Open the source (opens github.com) https://github.com/john-rocky/apple-silicon-llm-bench

What it is

Datasets & benchmarks from john-rocky.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/john-rocky/apple-silicon-llm-bench.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. mac-ml-speed-test

    Datasets & benchmarks

    A few quick scripts focused on testing TensorFlow/PyTorch/Llama 2 on macOS.

    Stale GitHub, mrdbourke 203 ★Updated 2 years ago

    git clone https://github.com/mrdbourke/mac-ml-speed-test.git
  2. ollama-benchmark

    Datasets & benchmarks

    LLM Benchmark for Throughput via Ollama (Local LLMs)

    Stale GitHub, aidatatools 397 ★MITUpdated 20 days ago

    git clone https://github.com/aidatatools/ollama-benchmark.git
  3. TheAgentCompany

    Datasets & benchmarks

    An agent benchmark with tasks in a simulated software company.

    Stale GitHub, TheAgentCompany 795 ★MITUpdated 12 months ago

    git clone https://github.com/TheAgentCompany/TheAgentCompany.git
  4. AI_Diplomacy

    Datasets & benchmarks

    Frontier Models playing the board game Diplomacy.

    Stale GitHub, GoodStartLabs 708 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/GoodStartLabs/AI_Diplomacy.git
  5. ClawBench

    Datasets & benchmarks

    Open-source benchmark for browser AI agents on daily tasks.

    Stale GitHub, TIGER-AI-Lab 1,229 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/TIGER-AI-Lab/ClawBench.git
  6. swe-sweep

    Datasets & benchmarks

    How many bugs can LMs find & fix in large codebases?

    Indexed GitHub, facebookresearch 240 ★Custom licence (see repository)Updated 4 days ago

    git clone https://github.com/facebookresearch/swe-sweep.git