InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. beir

    AI repositories

    A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

    Stale GitHub, beir-cellar 2,310 ★Apache-2.0Updated 13 months ago

    git clone https://github.com/beir-cellar/beir.git
  2. wtfibought

    Agents & frameworks

    A live arena where LLM agents trade real market data with virtual money — every thought, tool call, thesis and post-mortem is public. Bring your own model and key.

    Stale GitHub, mamawai 143 ★MITUpdated 18 days ago

    git clone https://github.com/mamawai/wtfibought.git
  3. mnemiq

    MCP servers

    Open-source text-to-SQL engine you tune and measure on your own database

    Indexed GitHub, agenticfabriq 81 ★Apache-2.0Updated today

    git clone https://github.com/agenticfabriq/mnemiq.git
  4. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  5. Production-ready Claude Code skills for building AI agents with Pydantic AI. Includes dependency injection, tools, validators, streaming, multi-agent orchestration, and evaluation framework patterns.

    Stale GitHub, Fuenfgeld 10 ★MITUpdated 30 days ago

    git clone https://github.com/Fuenfgeld/pydantic-ai-skills.git
  6. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  7. chunky

    AI repositories

    A local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.

    Stale GitHub, GiovanniPasq 213 ★MITUpdated 7 days ago

    git clone https://github.com/GiovanniPasq/chunky.git
  8. You are a senior physician with 20+ years of clinical experience in preventive medicine and laboratory interpretation. Analyze the attached health report comprehensively and clinically. Provide output in the following…

    Indexed prompts.chat (CC0 prompt collection), Glitchrex CC0-1.0

  9. Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.

    Stale GitHub, YanpengQi7 83 ★Updated 29 days ago

    git clone https://github.com/YanpengQi7/ai-reliability-copilot.git
  10. civic-ai-tools

    MCP servers

    Open-source platform connecting AI assistants to government open data — MCP server, curated civic MCP directory, and anti-hallucination framework for all 559 Socrata portals

    Indexed GitHub, npstorey 48 ★MITUpdated 3 days ago

    git clone https://github.com/npstorey/civic-ai-tools.git
  11. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  12. seo-skill-bench

    Datasets & benchmarks

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    Stale GitHub, aleclindz 51 ★MITUpdated 32 days ago

    git clone https://github.com/aleclindz/seo-skill-bench.git
  13. langfuse-docs

    Skills

    🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

    Stale GitHub, langfuse 253 ★MITUpdated 4 days ago

    git clone https://github.com/langfuse/langfuse-docs.git
  14. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  15. VerifyAX

    MCP servers

    Evaluate, benchmark, and simulate AI agents on the VerifyAX agent-evaluation platform.

    Indexed Official MCP Registry, io.github.verifyax Updated today 3 notes

    @verifyax/mcp-server@0.5.0
  16. bisheng

    AI repositories

    BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.

    Indexed GitHub, dataelement 12.3k ★Apache-2.0Updated today

    git clone https://github.com/dataelement/bisheng.git
  17. collaborative-gym

    Agents & frameworks

    Framework and toolkits for building and evaluating collaborative agents that can work together with humans.

    Stale GitHub, SALT-NLP 163 ★MITUpdated 2 months ago

    git clone https://github.com/SALT-NLP/collaborative-gym.git
  18. ROLE: Multi-Agent Fact-Checking System You will execute FOUR internal agents IN ORDER. Agents must not share prohibited information. Do not revise earlier outputs after moving to the next agent. AGENT ⊕ EXTRACTOR -…

    Indexed prompts.chat (CC0 prompt collection), m727ichael@gmail.com CC0-1.0

  19. adk-js

    Agents & frameworks

    An open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

    Indexed GitHub, google 1,478 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/google/adk-js.git
  20. llm-rl-environments-lil-course

    Agents & frameworks

    🌱 A little course on Reinforcement Learning Environments for evaluating and training Language Models

    Stale GitHub, anakin87 234 ★Apache-2.0Updated 5 months ago

    git clone https://github.com/anakin87/llm-rl-environments-lil-course.git
  21. RAG-FiT

    AI repositories

    Framework for enhancing LLMs for RAG tasks using fine-tuning.

    Stale Archived GitHub, IntelLabs 766 ★Apache-2.0Updated 32 days ago 1 note

    git clone https://github.com/IntelLabs/RAG-FiT.git
  22. Stale GitHub, microsoft 11 ★MITUpdated 5 days ago

    git clone https://github.com/microsoft/frontiertour27-ILL224-see-how-far-one-agent-goes-in-copilot-studio.git
  23. Build evals from traces, code, and coding sessions, then measure and improve AI agent behavior, including your coding agents

    Indexed GitHub, relai-ai 14 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/relai-ai/agent-eval-loop.git
  24. metaharness

    Agents & frameworks

    🛠️ The meta-harness for AI agents — scaffold your own focused, branded agent harness with its own npx CLI, MCP server, memory, learning loop, and witness-signed releases. Works with Claude Code, Codex, pi.dev, Hermes, OpenClaw, and RVM (hardware-isolated sandbox).

    Indexed GitHub, ruvnet 747 ★MITUpdated 2 days ago

    git clone https://github.com/ruvnet/metaharness.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.