InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. coder_eval

    Skills

    Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.

    Indexed GitHub, UiPath 160 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/UiPath/coder_eval.git
  2. eval-genius

    Skills

    Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Deterministic-first, tool-agnostic.

    Stale GitHub, alexgreensh 24 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/alexgreensh/eval-genius.git
  3. codex-keysmith

    Prompts

    Versioned Codex instruction deployment with preview, ownership manifests, hook isolation, scenario evaluation, and recovery.

    Stale GitHub, Jia-Ethan 4,740 ★MITUpdated 50 days ago

    git clone https://github.com/Jia-Ethan/codex-keysmith.git
  4. prism

    Datasets & benchmarks

    Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

    Stale GitHub, genlab-1c 55 ★MITUpdated 18 days ago

    git clone https://github.com/genlab-1c/prism.git
  5. nemo-helix

    Agents & frameworks

    NeMo Helix: Make the agents you ship faster, more accurate, and safer.

    Indexed GitHub, NVIDIA-NeMo 145 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/NVIDIA-NeMo/nemo-helix.git
  6. promptbench

    Prompts

    A unified evaluation framework for large language models

    Stale Archived GitHub, microsoftarchive 2,821 ★MITUpdated 8 months ago 1 note

    git clone https://github.com/microsoftarchive/promptbench.git
  7. nasde-toolkit

    Skills

    CLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys.

    Stale GitHub, NoesisVision 14 ★MITUpdated 32 days ago

    git clone https://github.com/NoesisVision/nasde-toolkit.git
  8. generative-ai

    Agents & frameworks

    Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

    Stale GitHub, genieincodebottle 2,669 ★MITUpdated 7 days ago

    git clone https://github.com/genieincodebottle/generative-ai.git
  9. llm-colosseum

    Datasets & benchmarks

    Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM

    Stale GitHub, OpenGenerativeAI 1,483 ★MITUpdated 20 months ago

    git clone https://github.com/OpenGenerativeAI/llm-colosseum.git
  10. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  11. SHP

    Datasets & benchmarks

    Hugging Face dataset by stanfordnlp

    Indexed Hugging Face Hub, stanfordnlp 333 ♥Updated 3 years ago

    stanfordnlp/SHP
  12. palico-ai

    Agents & frameworks

    Build, Improve Performance, and Productionize your AI Application

    Stale GitHub, palico-ai 343 ★MITUpdated 24 months ago

    git clone https://github.com/palico-ai/palico-ai.git
  13. LightCompress

    Datasets & benchmarks

    [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

    Stale GitHub, ModelTC 751 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/ModelTC/LightCompress.git
  14. deepresearch-agent

    Agents & frameworks

    一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。

    Stale GitHub, qiqihezh 165 ★Updated 6 months ago

    git clone https://github.com/qiqihezh/deepresearch-agent.git
  15. fast-agent

    Agents & frameworks

    Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support

    Indexed GitHub, evalstate 3,970 ★Apache-2.0Updated yesterday

    git clone https://github.com/evalstate/fast-agent.git
  16. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  17. Adaptive-rag

    AI repositories

    按问题复杂度自动分流的电商客服 RAG:简单问题直接回答;复杂问题经过多跳检索并自我纠错;查单、退改通过工具调用完成,且高风险写操作必须先经人工确认。基于 FastAPI 和 LangGraph,混合检索并附带引用,内置离线评测与 Prometheus 指标。涵盖简历写法和项目面试拷打

    Stale GitHub, JEen-wang 137 ★MITUpdated 25 days ago

    git clone https://github.com/JEen-wang/Adaptive-rag.git
  18. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  19. DreamLayer-Eval

    AI repositories

    DreamLayer Eval: open-source benchmarking for image and video diffusion models. Automate prompts, seeds, and metrics for reproducible results. A research project from DreamLayer AI (dreamlayer.io).

    Stale GitHub, TheDesignFounder 412 ★GPL-3.0Updated 17 days ago

    git clone https://github.com/TheDesignFounder/DreamLayer-Eval.git
  20. agents-last-exam

    Datasets & benchmarks

    Hugging Face dataset by agents-last-exam

    Stale Hugging Face Hub, agents-last-exam 225 ♥cc-by-4.0Updated 33 days ago

    agents-last-exam/agents-last-exam
  21. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  22. Awesome-Scientific-LLM-Benchmarks

    Datasets & benchmarks

    A curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.

    Stale GitHub, subinium 42 ★MITUpdated 14 days ago

    git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git
  23. WFGY

    AI repositories

    WFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.

    Indexed GitHub, onestardao 1,805 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/onestardao/WFGY.git
  24. dokimi

    Skills

    Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.

    Indexed GitHub, intel 37 ★Apache-2.0Updated today

    git clone https://github.com/intel/dokimi.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.