InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. coding.sh

    MCP servers

    Open-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.

    Indexed GitHub, jackulau 30 ★MITUpdated today

    git clone https://github.com/jackulau/coding.sh.git
  2. LLM Eval Harness

    MCP servers

    Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.

    Stale Official MCP Registry, io.github.razilzul Updated 29 days ago 2 notes

    ghcr.io/razilzul/llm-eval-harness:1.0.0
  3. tuui

    MCP servers

    A desktop MCP client designed as a tool unitary utility integration, accelerating AI adoption through the Model Context Protocol (MCP) and enabling cross-vendor LLM API orchestration.

    Stale GitHub, AI-QL 1,157 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/AI-QL/tuui.git
  4. labelhub-platform

    MCP servers

    面向 LLM 训练数据生产的可信工作台:AI 提供证据、人类裁决,append-only 账本、可复现导出、评测关卡与 MCP Server。

    Stale GitHub, Iceberg86-gb 48 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/Iceberg86-gb/labelhub-platform.git
  5. cap-evolve

    MCP servers

    Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

    Indexed GitHub, skillberry-ai 70 ★Apache-2.0Updated today

    git clone https://github.com/skillberry-ai/cap-evolve.git
  6. agentenv-framework

    MCP servers

    Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

    Indexed GitHub, scaleapi 441 ★Apache-2.0Updated today

    git clone https://github.com/scaleapi/agentenv-framework.git
  7. retriEVAL

    MCP servers

    LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.

    Stale Official MCP Registry, io.github.hcarrillo001 Updated 2 months ago 2 notes

    ghcr.io/hcarrillo001/retrieval-mcp:1.0.0
  8. jev-judge-mcp

    MCP servers

    Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

    Stale GitHub, PyModel 48 ★MITUpdated 32 days ago

    git clone https://github.com/PyModel/jev-judge-mcp.git
  9. PuzzleTide

    MCP servers

    Word search, crossword, and sudoku generators, printable PDF worksheets, and verifiable LLM evals.

    Stale Official MCP Registry, io.github.catorch Updated 4 months ago 1 note

    puzzletide-mcp@0.2.1
  10. digline

    MCP servers

    Read, measure and explain LLM eval runs of a digline suite. Cannot promote a baseline.

    Stale Official MCP Registry, dev.digline Updated 6 days ago 1 note

    digline-mcp@0.9.1
  11. LLM Prover

    MCP servers

    LLM evaluation: compare models on cost and quality, score outputs, run benchmarks, analyse agents.

    Stale Official MCP Registry, com.pysolvr Updated 4 days ago 1 note

  12. evalpair

    MCP servers

    Run LLM eval suites on local or hosted models; compare runs with intervals and a noise floor.

    Stale Official MCP Registry, io.github.Srikarmk Updated 11 days ago 2 notes

    evalpair@0.1.1
  13. mcp-as-a-judge

    MCP servers

    MCP as a Judge: a behavioral MCP that strengthens AI coding assistants via explicit LLM evaluations

    Stale Official MCP Registry, io.github.OtherVibes Updated 14 months ago 1 note

    mcp-as-a-judge@0.3.20

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.