Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
polymarket-paper-trader
Datasets & benchmarksPaper trading simulator for Polymarket
git clone https://github.com/agent-next/polymarket-paper-trader.git -
MobileWorld
Datasets & benchmarksBenchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)
git clone https://github.com/Tongyi-MAI/MobileWorld.git -
civ6-mcp
Datasets & benchmarksAn MCP server that lets LLM agents play Civilization VI.
git clone https://github.com/lmwilki/civ6-mcp.git -
LMeterX
Datasets & benchmarksA general-purpose API load testing platform that supports LLM services、A2A/MCP services and business HTTP interfaces, enabling one-click performance testing, result comparison, and AI-powered intelligent analysis and summarization. 一站式通用 API 压测平台,支持大模型推理业/务 HTTP 接口、A2A/MCP服务,一键完成性能测试、结果对比与 AI 智能分析总结
git clone https://github.com/MigoXLab/LMeterX.git -
awesome-ai-gateway
Datasets & benchmarks⚡ Awesome AI Gateway — pick an AI gateway from 240+ (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security & compliance, see what consolidated in 2026, learn how they actually work from an 8-chapter handbook, and verify every number yourself. Reproducible benchmarks, bilingual; stars & releases refreshed daily.
git clone https://github.com/cuihuan/awesome-ai-gateway.git -
primitive-bench
Datasets & benchmarksThe marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.
git clone https://github.com/primitive-bench/primitive-bench.git -
DishwasherBench
Datasets & benchmarksA dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
git clone https://github.com/composo-ai/DishwasherBench.git -
qwen38-strix-halo-harness
Datasets & benchmarksAI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NPU side-jobs, crash recovery, logged benchmarks.
git clone https://github.com/aic0d3r/qwen38-strix-halo-harness.git -
lasagna
Datasets & benchmarksSwap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augmented Generation Compositionality": 450 configurations, eight benchmarks, corpora from 5K to 200K documents.
git clone https://github.com/dolev31/lasagna.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.