InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. Awesome-Agent-Harness

    Agents & frameworks

    The first survey dedicated to LLM agent harnesses — 168 references, 34 evidence-coded systems, and 30 evaluation entries.

    Stale GitHub, Gloriaameng 381 ★CC-BY-4.0Updated 7 days ago

    git clone https://github.com/Gloriaameng/Awesome-Agent-Harness.git
  2. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  3. bisheng

    AI repositories

    BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.

    Indexed GitHub, dataelement 12.3k ★Apache-2.0Updated today

    git clone https://github.com/dataelement/bisheng.git
  4. Agent-Skills

    Skills

    Skills SDK for Codex/AI coding agents: author, validate, evaluate, and sync runtime projections through ask.

    Stale GitHub, jscraik 9 ★Apache-2.0Updated 21 days ago

    git clone https://github.com/jscraik/Agent-Skills.git
  5. Japanese Order Eval

    MCP servers

    Regression checks for Japanese order automations: six free cases, strict JSON scoring and CI gates.

    Stale Official MCP Registry, app.netlify.shigoto-dogu Updated 2 months ago 1 note

  6. Web Search API Eval

    MCP servers

    Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.

    Stale Official MCP Registry, io.github.psinglap Updated 20 days ago 3 notes

    websearch-api-eval-mcp@1.1.0
  7. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  8. 28 eval-informed mental models and critical-thinking skills for Claude Code, GitHub Copilot, Codex, Cursor, and other Agent Skills-compatible tools

    Stale GitHub, tjboudreaux 1,637 ★MITUpdated 3 months ago

    git clone https://github.com/tjboudreaux/cc-thinking-skills.git
  9. Free to compete: testnet faucet funds everything. Classify a product blurb: category + sentiment

    Stale Official MCP Registry, com.a2awire Updated 8 days ago 1 note

  10. Remote MCP tools for 3-vendor AI workflow diagnostics, HAR sandboxes, 50v audits, and rubrics.

    Stale Official MCP Registry, io.github.jdurey Updated 2 days ago 1 note

  11. inferock-bench

    Datasets & benchmarks

    Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.

    Stale GitHub, inferock 107 ★Custom licence (see repository)Updated 3 months ago

    git clone https://github.com/inferock/inferock-bench.git
  12. gaik-toolkit

    AI repositories

    Generative AI knowledge management toolkit: strategy, guidance, evaluation and reusable Python components.

    Indexed GitHub, GAIK-project 108 ★MITUpdated yesterday

    git clone https://github.com/GAIK-project/gaik-toolkit.git
  13. AISafetyHot

    MCP servers

    AI Safety HOT: daily AI safety news, papers and incidents in one place — jailbreaks, prompt injection, alignment, evals, governance. Website, daily digest, RSS and MCP.

    Indexed GitHub, wuyoscar 2,127 ★Custom licence (see repository)Updated today

    git clone https://github.com/wuyoscar/AISafetyHot.git
  14. wtfibought

    Agents & frameworks

    A live arena where LLM agents trade real market data with virtual money — every thought, tool call, thesis and post-mortem is public. Bring your own model and key.

    Stale GitHub, mamawai 143 ★MITUpdated 18 days ago

    git clone https://github.com/mamawai/wtfibought.git
  15. chunky

    AI repositories

    A local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.

    Stale GitHub, GiovanniPasq 213 ★MITUpdated 7 days ago

    git clone https://github.com/GiovanniPasq/chunky.git
  16. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  17. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  18. Act as an AI Security and Compliance Expert. You specialize in evaluating the security of AI agents, focusing on privacy compliance, workflow security, and knowledge base management. Your task is to create a…

    Indexed prompts.chat (CC0 prompt collection), bluedog110776@gmail.com CC0-1.0

  19. # Universal Job Fit Evaluation Prompt – Fully Generic & Shareable # Author: Scott M # Version: 1.6 # Last Modified: 2026-03-06 ## Changelog - **v1.6 (2026-03-06):** Integrated "Read Between the Lines" (Vibe Check), ATS…

    Indexed prompts.chat (CC0 prompt collection), thanos0000@gmail.com CC0-1.0

  20. mnemiq

    MCP servers

    Open-source text-to-SQL engine you tune and measure on your own database

    Indexed GitHub, agenticfabriq 81 ★Apache-2.0Updated today

    git clone https://github.com/agenticfabriq/mnemiq.git
  21. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  22. Self-improvement feedback loop for AI agent skills — analyzes past sessions, drafts structured improvement proposals, and gates auto-apply behind an evaluation framework. Host-agnostic (Hermes, Claude Code, and any HostAdapter).

    Stale GitHub, Carlo1911 6 ★MITUpdated 29 days ago

    git clone https://github.com/Carlo1911/skill-evolution.git
  23. adk-js

    Agents & frameworks

    An open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

    Indexed GitHub, google 1,478 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/google/adk-js.git
  24. civic-ai-tools

    MCP servers

    Open-source platform connecting AI assistants to government open data — MCP server, curated civic MCP directory, and anti-hallucination framework for all 559 Socrata portals

    Indexed GitHub, npstorey 48 ★MITUpdated 3 days ago

    git clone https://github.com/npstorey/civic-ai-tools.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.