Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
EvalForge Lite
MCP serversCompare text LLMs across OpenRouter, Bedrock, Vertex AI and Foundry with automated grading.
evalforge-lite@1.0.1 -
evalsnap
MCP serversSnapshot diff for agent and tool outputs: changed cases, score regressions, pass/fail verdict.
@reneturcios/evalsnap@0.1.0 -
agentic-ai
AI repositoriesAgentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.
git clone https://github.com/mahmoudrabie/agentic-ai.git -
PhysGym
Datasets & benchmarksA benchmark suite for evaluating LLM-based interactive scientific reasoning.
git clone https://github.com/principia-ai/PhysGym.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git -
LightOnOCR-3-1B
Modelsimage text to text model by lightonai · transformers
lightonai/LightOnOCR-3-1B -
be-ambitious-harness
SkillsMake AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and an independent verifier that never sees the worker's reasoning. Plain markdown, EN + RU.
git clone https://github.com/alexpiar/be-ambitious-harness.git -
DM-Code-Agent
Agents & frameworksAn inspectable coding agent. Trace changes, diagnose failed runs, and share test evidence. Local-first Python, no-key demo, standalone HTML reports.
git clone https://github.com/hwfengcs/DM-Code-Agent.git -
Xiaomi-OCR-0
Modelsimage text to text model by SeerRay-Lab · transformers
SeerRay-Lab/Xiaomi-OCR-0 -
Act as a Senior Research Paper Evaluator. You are an experienced academic reviewer with expertise in evaluating scholarly work across multiple disciplines. Your task is to critically assess academic documents and…
-
humanities_data_benchmark
Datasets & benchmarksLLM Benchmark Suite for Humanities Data
git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git -
labelhub-platform
MCP servers面向 LLM 训练数据生产的可信工作台:AI 提供证据、人类裁决,append-only 账本、可复现导出、评测关卡与 MCP Server。
git clone https://github.com/Iceberg86-gb/labelhub-platform.git -
agent-eval
MCP serversStatistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.
agent-regress-cli@0.1.8 -
Hive Evaluator
MCP serversMCP server: NEED + YIELD + CLEAN-MONEY gates with EIP-3009 attestations · Hive Civilization
-
Act as a Business Idea Evaluator. You are an expert in assessing business concepts across various industries. Your task is to evaluate and score the given business idea based on specific criteria. You will: - Analyze…
-
Chipp Analytics and evals
MCP serversChipp: conversations, messages, usage and engagement analytics, end users and their memories, and.
-
Phonon-2.5-English
Modelsautomatic speech recognition model by FermionResearch · mlx
FermionResearch/Phonon-2.5-English -
Research-Assistant-Agent
Agents & frameworksMulti-agent research execution system: paper reproduction, benchmark evaluation, AutoResearch and claim-evidence graphs
git clone https://github.com/zzp1221/Research-Assistant-Agent.git -
verbatim-citation-gate
SkillsCatch fabricated RAG citations before they reach the user: a zero-token verbatim gate + a burden-of-proof judge, framework-agnostic.
git clone https://github.com/tonydzi/verbatim-citation-gate.git -
Repo2RLEnv
AI repositoriesTurn any repository into verifiable RL environments for coding agents - Harbor tasks you can train on, evaluate and share on the Hugging Face Hub
git clone https://github.com/huggingface/Repo2RLEnv.git -
beir
AI repositoriesA Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
git clone https://github.com/beir-cellar/beir.git -
Act as a PhD Thesis Evaluator for Computer Science. You are an expert in computer science with significant experience in reviewing doctoral dissertations. Your task is to evaluate the provided PhD thesis and offer…
-
GateMem
Agents & frameworksGateMem: a benchmark and evaluation toolkit for memory governance in multi-principal shared-memory LLM agents.
git clone https://github.com/rzhub/GateMem.git -
LLM Eval Harness
MCP serversRun a golden set and gate LLM model swaps on accuracy, with latency and cost per request.
ghcr.io/razilzul/llm-eval-harness:1.0.0
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.