Library
85,337 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
llmspeed
Datasets & benchmarksFeel how fast a local LLM runs before you buy the hardware
git clone https://github.com/nicorobinreal/llmspeed.git -
ai-hardware-fit
Datasets & benchmarksCan my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.
git clone https://github.com/jaeseok614/ai-hardware-fit.git -
agent-duel
Datasets & benchmarksBlinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.
git clone https://github.com/bottomless/agent-duel.git -
aa-agentperf-local
Datasets & benchmarksBenchmark local LLM serving by replaying real agent trajectories
git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git -
strix-halo-guide
Datasets & benchmarksEvidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results.
git clone https://github.com/hogeheer499-commits/strix-halo-guide.git -
turnkeyml
Datasets & benchmarksNo-code CLI designed for accelerating ONNX workflows
git clone https://github.com/onnx/turnkeyml.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
ollama-benchmark
Datasets & benchmarksLLM Benchmark for Throughput via Ollama (Local LLMs)
git clone https://github.com/aidatatools/ollama-benchmark.git -
typed-decisions
Datasets & benchmarksHugging Face dataset by LocalLLaMA
LocalLLaMA/typed-decisions -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
qwen38-strix-halo-harness
Datasets & benchmarksAI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NPU side-jobs, crash recovery, logged benchmarks.
git clone https://github.com/aic0d3r/qwen38-strix-halo-harness.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
multi-agents
Datasets & benchmarksMulti-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.
git clone https://github.com/BichengWang/multi-agents.git -
xorcise
Datasets & benchmarksRun cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.
git clone https://github.com/xorcise-ai/xorcise.git -
inferock-bench
Datasets & benchmarksLocal LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.
git clone https://github.com/inferock/inferock-bench.git -
sloptotal
Datasets & benchmarksOpen-source AI text detector and ChatGPT detector: 23 engines, one calibrated score, every number measured on a public benchmark. Self-hosted, runs on CPU.
git clone https://github.com/pablocaeg/sloptotal.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.