InnerVault

Library

85,336 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. promptfoo

    Prompts

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    Indexed GitHub, promptfoo 27.3k ★MITUpdated today

    git clone https://github.com/promptfoo/promptfoo.git
  2. ai-agents-the-definitive-guide

    Agents & frameworks

    Repo for AI Agents The Definitive Guide

    Stale GitHub, Nicolepcx 2,747 ★Updated 3 months ago

    git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git
  3. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  4. AgentLab

    Datasets & benchmarks

    AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.

    Stale GitHub, ServiceNow 653 ★Custom licence (see repository)Updated 41 days ago

    git clone https://github.com/ServiceNow/AgentLab.git
  5. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  6. agentic_security

    Agents & frameworks

    Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

    Indexed GitHub, msoedov 2,046 ★Apache-2.0Updated yesterday

    git clone https://github.com/msoedov/agentic_security.git
  7. Kiln

    Prompts

    Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

    Indexed GitHub, Kiln-AI 5,283 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/Kiln-AI/Kiln.git
  8. future-agi

    Agents & frameworks

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    Indexed GitHub, future-agi 2,267 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/future-agi/future-agi.git
  9. promptbench

    Prompts

    A unified evaluation framework for large language models

    Stale Archived GitHub, microsoftarchive 2,821 ★MITUpdated 8 months ago 1 note

    git clone https://github.com/microsoftarchive/promptbench.git
  10. coder_eval

    Skills

    Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.

    Indexed GitHub, UiPath 160 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/UiPath/coder_eval.git
  11. FairMind

    MCP servers

    Release gate for AI systems from FairMind: evaluations, framework evidence and ledger snapshots.

    Indexed Official MCP Registry, io.github.adhit-r Updated today 2 notes

    fairmind-mcp@0.4.0a0
  12. UHGEval

    Datasets & benchmarks

    [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

    Stale GitHub, IAAR-Shanghai 183 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/IAAR-Shanghai/UHGEval.git
  13. eval-genius

    Skills

    Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Deterministic-first, tool-agnostic.

    Stale GitHub, alexgreensh 24 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/alexgreensh/eval-genius.git
  14. skill-harness

    Skills

    How do you KNOW if a 'skill' is any good - deterministically? This runs controlled skill-vs-no-skill experiments. It doesn't just ask, “Did the skill work?” It asks “Did we actually give the skill to the model, did the comparison remain fair, what changed, what did it cost, and do we have enough evidence to say that confidently?”

    Stale GitHub, MrBinnacle 16 ★MITUpdated 14 days ago

    git clone https://github.com/MrBinnacle/skill-harness.git
  15. Self-improvement feedback loop for AI agent skills — analyzes past sessions, drafts structured improvement proposals, and gates auto-apply behind an evaluation framework. Host-agnostic (Hermes, Claude Code, and any HostAdapter).

    Stale GitHub, Carlo1911 6 ★MITUpdated 29 days ago

    git clone https://github.com/Carlo1911/skill-evolution.git
  16. Production-ready Claude Code skills for building AI agents with Pydantic AI. Includes dependency injection, tools, validators, streaming, multi-agent orchestration, and evaluation framework patterns.

    Stale GitHub, Fuenfgeld 10 ★MITUpdated 30 days ago

    git clone https://github.com/Fuenfgeld/pydantic-ai-skills.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.