Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
IA-QA Eval & RAG — 38 tools
MCP serversLLM & RAG evals: hallucination, eval contracts, semantic tests, tokens & model info. Free, no key.
-
LLM-RGB
Datasets & benchmarksLLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.
git clone https://github.com/babelcloud/LLM-RGB.git -
Execute clause-risk-score-evaluator-mcp
MCP serversPremium agentic endpoint for clause-risk-score-evaluator-mcp.
-
chess-eval
MCP serversStockfish chess eval: best move, score, principal variation, multipv. $0.005/call via x402.
-
evals-mcp-server
MCP serversAuthor verifiable eval records through a draft→review→revise→submit loop with enforced graders.
@cyanheads/evals-mcp-server@0.2.0 -
GenPark Safe Eval Expression AST Sandbox
MCP serversZero-dependency AST safe mathematical and boolean expression evaluator and validator
https://github.com/Alpha-Park/genpark-safe-eval-expression-ast-sandbox-skill/releases/download/v1.0.1/genpark-safe-eval-expression-ast-sandbox-skill-v1.0.1.mcpb -
In LangChain for LLM Application Development, you will gain essential skills in expanding the use cases and capabilities of language models in application development using the LangChain framework.
git clone https://github.com/Ryota-Kawamura/LangChain-for-LLM-Application-Development.git -
autoguardrails
Datasets & benchmarksAlignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
git clone https://github.com/SantanderAI/autoguardrails.git -
measured-type
SkillsAgent skill for web typography: Bringhurst applied to modern CSS, with claims measured in the browser and a baseline eval suite
git clone https://github.com/PetriLahdelma/measured-type.git -
d3-lite
Modelszero shot classification model by vllm-sr · transformers
vllm-sr/d3-lite -
hwatu
MCP serversVerification browser for coding agents: headless WebKit, DOM eval, screenshots, pixel-diff.
https://github.com/hongnoul/hwatu/releases/download/v0.6.0/hwatu-mcp-linux-x86_64.tar.gz@0.6.0 -
elimination_game
Datasets & benchmarksA multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other
git clone https://github.com/lechmazur/elimination_game.git -
Long-Horizon-Terminal-Bench
Datasets & benchmarksHugging Face dataset by IntelligenceLab
IntelligenceLab/Long-Horizon-Terminal-Bench -
MIRAI
Agents & frameworksCode and Data for "MIRAI: Evaluating LLM Agents for Event Forecasting"
git clone https://github.com/yecchen/MIRAI.git -
evalgate
MCP serversStatistical checks an agent runs before trusting an AI eval number (is #1 real, judge bias, more).
eval-integrity@0.4.1 -
evalpair
MCP serversRun LLM eval suites on local or hosted models; compare runs with intervals and a noise floor.
evalpair@0.1.1 -
Sage
Agents & frameworksAn RSI Agent Harness for running, evaluating, and evolving agents—with tools, skills, MCP, stateful execution, and multi-agent workflows.
git clone https://github.com/ZHangZHengEric/Sage.git -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
Privi Ledger — Recovery Evaluation
MCP serversEvaluation-only ledger MCP. Temporary storage; not durable audit evidence. Payments disabled.
-
Agent evaluation scorecard — Atom
MCP serversAgent evaluation scorecard, delivered per API call as an atom service on Conduit Atoms.
-
Claim evaluation review — Atom
MCP serversClaim evaluation review, delivered per API call as an atom service on Conduit Atoms.
-
Vendor evaluation three way match pack — Atom
MCP serversVendor evaluation three-way match: declared procurement record checked per call.
-
cs329z-chinese
Docs & tutorials斯坦福 Stanford CS329Z Engineering AI Agents 中文教程:面向 Python 开发者的 AI 智能体课程,19 讲与 4 项实践,涵盖 LLM、RAG、工具调用、记忆、多智能体、评测与安全。完整 Markdown、在线与离线 HTML。Chinese course notes.
git clone https://github.com/wanghao-SunSky/cs329z-chinese.git -
Awesome-CUA
Agents & frameworksOpen, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
git clone https://github.com/reacher-z/Awesome-CUA.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.