Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
Awesome-Agent-Harness
Agents & frameworksThe first survey dedicated to LLM agent harnesses — 168 references, 34 evidence-coded systems, and 30 evaluation entries.
git clone https://github.com/Gloriaameng/Awesome-Agent-Harness.git -
sec-code-bench
Datasets & benchmarksSecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).
git clone https://github.com/alibaba/sec-code-bench.git -
bisheng
AI repositoriesBISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.
git clone https://github.com/dataelement/bisheng.git -
Agent-Skills
SkillsSkills SDK for Codex/AI coding agents: author, validate, evaluate, and sync runtime projections through ask.
git clone https://github.com/jscraik/Agent-Skills.git -
Japanese Order Eval
MCP serversRegression checks for Japanese order automations: six free cases, strict JSON scoring and CI gates.
-
Web Search API Eval
MCP serversWeb search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
websearch-api-eval-mcp@1.1.0 -
DishwasherBench
Datasets & benchmarksA dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
git clone https://github.com/composo-ai/DishwasherBench.git -
cc-thinking-skills
Skills28 eval-informed mental models and critical-thinking skills for Claude Code, GitHub Copilot, Codex, Cursor, and other Agent Skills-compatible tools
git clone https://github.com/tjboudreaux/cc-thinking-skills.git -
Free to compete: testnet faucet funds everything. Classify a product blurb: category + sentiment
-
Human Creative AI Workflow & Eval Lab
MCP serversRemote MCP tools for 3-vendor AI workflow diagnostics, HAR sandboxes, 50v audits, and rubrics.
-
inferock-bench
Datasets & benchmarksLocal LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.
git clone https://github.com/inferock/inferock-bench.git -
gaik-toolkit
AI repositoriesGenerative AI knowledge management toolkit: strategy, guidance, evaluation and reusable Python components.
git clone https://github.com/GAIK-project/gaik-toolkit.git -
AISafetyHot
MCP serversAI Safety HOT: daily AI safety news, papers and incidents in one place — jailbreaks, prompt injection, alignment, evals, governance. Website, daily digest, RSS and MCP.
git clone https://github.com/wuyoscar/AISafetyHot.git -
wtfibought
Agents & frameworksA live arena where LLM agents trade real market data with virtual money — every thought, tool call, thesis and post-mortem is public. Bring your own model and key.
git clone https://github.com/mamawai/wtfibought.git -
chunky
AI repositoriesA local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.
git clone https://github.com/GiovanniPasq/chunky.git -
primitive-bench
Datasets & benchmarksThe marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.
git clone https://github.com/primitive-bench/primitive-bench.git -
fde-gym
Datasets & benchmarksThe online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
git clone https://github.com/deeplethe/fde-gym.git -
Act as an AI Security and Compliance Expert. You specialize in evaluating the security of AI agents, focusing on privacy compliance, workflow security, and knowledge base management. Your task is to create a…
-
# Universal Job Fit Evaluation Prompt – Fully Generic & Shareable # Author: Scott M # Version: 1.6 # Last Modified: 2026-03-06 ## Changelog - **v1.6 (2026-03-06):** Integrated "Read Between the Lines" (Vibe Check), ATS…
-
mnemiq
MCP serversOpen-source text-to-SQL engine you tune and measure on your own database
git clone https://github.com/agenticfabriq/mnemiq.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
skill-evolution
SkillsSelf-improvement feedback loop for AI agent skills — analyzes past sessions, drafts structured improvement proposals, and gates auto-apply behind an evaluation framework. Host-agnostic (Hermes, Claude Code, and any HostAdapter).
git clone https://github.com/Carlo1911/skill-evolution.git -
adk-js
Agents & frameworksAn open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
git clone https://github.com/google/adk-js.git -
civic-ai-tools
MCP serversOpen-source platform connecting AI assistants to government open data — MCP server, curated civic MCP directory, and anti-hallucination framework for all 559 Socrata portals
git clone https://github.com/npstorey/civic-ai-tools.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.