InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. judge0

    Agents & frameworks

    Robust, fast, scalable, and sandboxed open-source online code execution system for humans and AI.

    Stale GitHub, judge0 4,474 ★GPL-3.0Updated 40 days ago

    git clone https://github.com/judge0/judge0.git
  2. judgeval

    Prompts

    The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

    Indexed GitHub, JudgmentLabs 1,074 ★Apache-2.0Updated yesterday

    git clone https://github.com/JudgmentLabs/judgeval.git
  3. Hugging Face space by bigcode

    Indexed Hugging Face Hub, bigcode 1,528 ♥Updated 2 years ago 1 note

    bigcode/bigcode-models-leaderboard
  4. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  5. mt_bench_human_judgments

    Datasets & benchmarks

    Hugging Face dataset by lmsys

    Stale Hugging Face Hub, lmsys 149 ♥cc-by-4.0Updated 3 years ago

    lmsys/mt_bench_human_judgments
  6. dingo

    AI repositories

    Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

    Stale GitHub, MigoXLab 764 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/MigoXLab/dingo.git
  7. jev-judge-mcp

    MCP servers

    Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

    Stale GitHub, PyModel 48 ★MITUpdated 33 days ago

    git clone https://github.com/PyModel/jev-judge-mcp.git
  8. xVerify

    Datasets & benchmarks

    xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

    Stale GitHub, IAAR-Shanghai 150 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/IAAR-Shanghai/xVerify.git
  9. Mica-v0.5-4B

    Models

    text classification model by sky7350

    Stale Hugging Face Hub, sky7350 42 ♥apache-2.0Updated 20 days ago

    sky7350/Mica-v0.5-4B
  10. xFinder

    Datasets & benchmarks

    [ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

    Stale GitHub, IAAR-Shanghai 178 ★Custom licence (see repository)Updated 12 months ago

    git clone https://github.com/IAAR-Shanghai/xFinder.git
  11. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  12. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 25 days ago

    git clone https://github.com/oumi-ai/oumi.git
  13. styleseed

    Skills

    Open-source design-method engine for Claude Code, Codex & Cursor. One agent skill with 22 internal workflows for fixed design judgment, multiple grammars, semantic palettes, reference compilation, and evidence-verified UI. MIT.

    Indexed GitHub, bitjaru 1,020 ★MITUpdated yesterday

    git clone https://github.com/bitjaru/styleseed.git
  14. Judge0 CE

    Public APIs

    Online code execution system

    Indexed public-apis (GitHub list)

  15. Fjc Judges

    MCP servers

    FJC Judges — the Federal Judicial Center's Biographical Directory of Article III federal judges…

    Stale Official MCP Registry, io.github.pipeworx-io Updated 16 days ago 3 notes

    @pipeworx/mcp-fjc-judges@0.2.1
  16. Search 160M+ Chinese court judgments; verify case numbers and quotes.

    Stale Official MCP Registry, com.wenshucha Updated 2 months ago 1 note

  17. judgevet

    MCP servers

    Ask TypeSafe Jev yes/no, choice and score questions and receive structured model judgments.

    Stale Official MCP Registry, io.github.Alberto-Codes Updated 23 days ago 2 notes

    judgevet@0.23.0
  18. simmer

    Skills

    Iterative artifact refinement with investigation-first judge board - constructs problem-specific judges that read the code, understand the problem, and propose evidence-based improvements

    Stale GitHub, 2389-research 16 ★MITUpdated 15 days ago

    git clone https://github.com/2389-research/simmer.git
  19. judgment-pack

    MCP servers

    Offline JPS validator, conformance tester, and experimental evaluator over stdio MCP; keyless.

    Stale Official MCP Registry, io.github.Judgment-Pack Updated 3 days ago 1 note

    ghcr.io/judgment-pack/judgment-pack:0.46.0
  20. Judges Panel

    MCP servers

    45 judges that evaluate AI-generated code for security, cost, and quality with built-in AST.

    Stale Official MCP Registry, io.github.KevinRabun Updated 7 months ago 1 note

    @kevinrabun/judges@3.129.9
  21. ZTL Judge

    MCP servers

    Zero-trust logic judge: ZFL claims, algebra derivation steps, sets defined by a property.

    Stale Official MCP Registry, io.github.inventor1975 Updated 12 days ago 1 note

  22. mcp-as-a-judge

    MCP servers

    MCP as a Judge: a behavioral MCP that strengthens AI coding assistants via explicit LLM evaluations

    Stale Official MCP Registry, io.github.OtherVibes Updated 14 months ago 1 note

    mcp-as-a-judge@0.3.20
  23. Statistical adjudication for agents: backtest judges, fact-checks. Pay per call in USDC via x402.

    Stale Official MCP Registry, io.github.Elimuaa Updated 46 days ago 1 note

  24. debate

    Datasets & benchmarks

    Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.

    Stale GitHub, lechmazur 38 ★Updated 22 days ago

    git clone https://github.com/lechmazur/debate.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.