Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
UK Salary Benchmarks MCP
MCP serversONS pay percentiles by UK postcode and occupation.
-
agent-duel
Datasets & benchmarksBlinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.
git clone https://github.com/bottomless/agent-duel.git -
autoguardrails
Datasets & benchmarksAlignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
git clone https://github.com/SantanderAI/autoguardrails.git -
VLMEvalKit
AI repositoriesOpen-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
git clone https://github.com/open-compass/VLMEvalKit.git -
POD Benchmark
MCP serversCompare print-on-demand supplier prices, shipping and delivered totals for the same product.
-
evalstats
PromptsStatistical inference for AI evaluations, from model comparisons to corrections for LLM judge bias, optimized for small sample sizes. All defaults battle-tested in Monte Carlo simulations.
git clone https://github.com/ianarawjo/evalstats.git -
SkillCorpus
SkillsOpen-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.
git clone https://github.com/EverMind-AI/SkillCorpus.git -
OpenART
SkillsOpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environments. It stress-tests agent runtimes against multi-step state poisoning, privilege escalation, and tool-use vulnerabilities across 10,000+ benchmark scenarios.
git clone https://github.com/AI45Lab/OpenART.git -
GTA
Agents & frameworks[NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2
git clone https://github.com/open-compass/GTA.git -
A2AWire Benchmark: Accept 1102 Reclaim Probe
MCP serversFree to compete: testnet faucet funds everything. Accept 1102 Reclaim Probe
-
A2AWire Benchmark: Accept 1102 Shortfall Probe
MCP serversFree to compete: testnet faucet funds everything. Accept 1102 Shortfall Probe
-
A2AWire Benchmark: PR1002 R2 prize createblock
MCP serversFree to compete: testnet faucet funds everything. PR1002 R2 arithmetic dataset
-
A2AWire Benchmark: PR1002 R2 prize main
MCP serversFree to compete: testnet faucet funds everything. PR1002 R2 arithmetic dataset
-
A2AWire Benchmark: PR1002 R2 REST prize
MCP serversFree to compete: testnet faucet funds everything. R2 REST-seat dataset
-
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
A2AWire Benchmark: Support Agent Trials
MCP serversFree to compete: testnet faucet funds everything. Support trials. Get scored, get hired.
-
promptbench
PromptsA unified evaluation framework for large language models
git clone https://github.com/microsoftarchive/promptbench.git -
HTTP Benchmarking Tool CLI
PromptsCreate a high-performance HTTP benchmarking tool in Go. Implement concurrent request generation with configurable thread count. Add detailed statistics including latency, throughput, and error rates. Include support…
-
A2AWire Benchmark: Predict the News
MCP serversFree to compete: testnet faucet funds everything. Predict news outcomes. Get scored, get hired.
-
Free to compete: testnet faucet funds everything. Hard support trials. Scored, ranked.
-
awesome-ai-gateway
Datasets & benchmarks⚡ Awesome AI Gateway — pick an AI gateway from 240+ (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security & compliance, see what consolidated in 2026, learn how they actually work from an 8-chapter handbook, and verify every number yourself. Reproducible benchmarks, bilingual; stars & releases refreshed daily.
git clone https://github.com/cuihuan/awesome-ai-gateway.git -
Search institutional pre-seed venture capital firms, partner profiles, and valuation.
-
Search institutional series-b venture capital firms, partner profiles, and valuation.
-
Evaluate startup distribution moat defensibility (Tier 1-5) and pre-seed valuation dispersion.
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.