Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
gaik-toolkit
AI repositoriesGenerative AI knowledge management toolkit: strategy, guidance, evaluation and reusable Python components.
git clone https://github.com/GAIK-project/gaik-toolkit.git -
beir
AI repositoriesA Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
git clone https://github.com/beir-cellar/beir.git -
wtfibought
Agents & frameworksA live arena where LLM agents trade real market data with virtual money — every thought, tool call, thesis and post-mortem is public. Bring your own model and key.
git clone https://github.com/mamawai/wtfibought.git -
mnemiq
MCP serversOpen-source text-to-SQL engine you tune and measure on your own database
git clone https://github.com/agenticfabriq/mnemiq.git -
fde-gym
Datasets & benchmarksThe online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
git clone https://github.com/deeplethe/fde-gym.git -
pydantic-ai-skills
SkillsProduction-ready Claude Code skills for building AI agents with Pydantic AI. Includes dependency injection, tools, validators, streaming, multi-agent orchestration, and evaluation framework patterns.
git clone https://github.com/Fuenfgeld/pydantic-ai-skills.git -
primitive-bench
Datasets & benchmarksThe marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.
git clone https://github.com/primitive-bench/primitive-bench.git -
chunky
AI repositoriesA local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.
git clone https://github.com/GiovanniPasq/chunky.git -
You are a senior physician with 20+ years of clinical experience in preventive medicine and laboratory interpretation. Analyze the attached health report comprehensively and clinically. Provide output in the following…
-
ai-reliability-copilot
PromptsTurn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.
git clone https://github.com/YanpengQi7/ai-reliability-copilot.git -
civic-ai-tools
MCP serversOpen-source platform connecting AI assistants to government open data — MCP server, curated civic MCP directory, and anti-hallucination framework for all 559 Socrata portals
git clone https://github.com/npstorey/civic-ai-tools.git -
explicit-edit-benchmark
Datasets & benchmarksBenchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.
git clone https://github.com/alexshpunt/explicit-edit-benchmark.git -
seo-skill-bench
Datasets & benchmarksOpen benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.
git clone https://github.com/aleclindz/seo-skill-bench.git -
langfuse-docs
Skills🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
git clone https://github.com/langfuse/langfuse-docs.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
VerifyAX
MCP serversEvaluate, benchmark, and simulate AI agents on the VerifyAX agent-evaluation platform.
@verifyax/mcp-server@0.5.0 -
collaborative-gym
Agents & frameworksFramework and toolkits for building and evaluating collaborative agents that can work together with humans.
git clone https://github.com/SALT-NLP/collaborative-gym.git -
ROLE: Multi-Agent Fact-Checking System You will execute FOUR internal agents IN ORDER. Agents must not share prohibited information. Do not revise earlier outputs after moving to the next agent. AGENT ⊕ EXTRACTOR -…
-
bisheng
AI repositoriesBISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.
git clone https://github.com/dataelement/bisheng.git -
adk-js
Agents & frameworksAn open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
git clone https://github.com/google/adk-js.git -
llm-rl-environments-lil-course
Agents & frameworks🌱 A little course on Reinforcement Learning Environments for evaluating and training Language Models
git clone https://github.com/anakin87/llm-rl-environments-lil-course.git -
RAG-FiT
AI repositoriesFramework for enhancing LLMs for RAG tasks using fine-tuning.
git clone https://github.com/IntelLabs/RAG-FiT.git -
git clone https://github.com/microsoft/frontiertour27-ILL224-see-how-far-one-agent-goes-in-copilot-studio.git
-
agent-eval-loop
SkillsBuild evals from traces, code, and coding sessions, then measure and improve AI agent behavior, including your coding agents
git clone https://github.com/relai-ai/agent-eval-loop.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.