Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
OpenART
SkillsOpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environments. It stress-tests agent runtimes against multi-step state poisoning, privilege escalation, and tool-use vulnerabilities across 10,000+ benchmark scenarios.
git clone https://github.com/AI45Lab/OpenART.git -
ai-agents-tutorial
Agents & frameworksLearn AI Agents step by step, from scratch - from function calling to agent loops to multi-agent systems, orchestration, and evaluation.
git clone https://github.com/amitshekhariitbhu/ai-agents-tutorial.git -
WritingBench
Datasets & benchmarksWritingBench: A Comprehensive Benchmark for Generative Writing
git clone https://github.com/X-PLUG/WritingBench.git -
GTA
Agents & frameworks[NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2
git clone https://github.com/open-compass/GTA.git -
skill-reach
SkillsAn evaluation suite for AI agent skill triggering and collision detection
git clone https://github.com/google/skill-reach.git -
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
AgentLab
Datasets & benchmarksAgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
git clone https://github.com/ServiceNow/AgentLab.git -
agentops
AI repositoriesPython SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
git clone https://github.com/AgentOps-AI/agentops.git -
self-bench
Datasets & benchmarksBenchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
git clone https://github.com/mupt-ai/self-bench.git -
reforge-ai-evals
PromptsOpen-source AI evals toolkit: skills that guide AI coding agents to write eval rubrics, eval datasets (User Input Grid), trace codes, code evaluators and LLM-as-a-judge prompts. From the Reforge AI Evals course by Calibre.
git clone https://github.com/Calibre-Labs/reforge-ai-evals.git -
judgeval
PromptsThe Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
git clone https://github.com/JudgmentLabs/judgeval.git -
xorcise
Datasets & benchmarksRun cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.
git clone https://github.com/xorcise-ai/xorcise.git -
awesome-llm-observability
Skills50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.
git clone https://github.com/ContextJet-ai/awesome-llm-observability.git -
instructor-xl
Modelssentence similarity model by hkunlp · sentence-transformers
hkunlp/instructor-xl -
CivAgent
Agents & frameworksCivAgent is an LLM-based Human-like Agent acting as a Digital Player within the Strategy Game Unciv.
git clone https://github.com/fuxiAIlab/CivAgent.git -
folkbench
AI repositories帮你的 AI 编程工具找到不降智、不断连、高性价比的好线路 · Live LLM relay rankings, model integrity probes & Tauri/Rust switcher for Claude Code, Codex & Gemini CLI
git clone https://github.com/Folkbench/folkbench.git -
t2v_metrics
AI repositoriesEvaluating text-to-image/video/3D models with VQAScore
git clone https://github.com/linzhiqiu/t2v_metrics.git -
tweakidea
SkillsEvaluate your startup idea from 1 to 5
git clone https://github.com/eph5xx/tweakidea.git -
rashomon
SkillsMeasure prompt and skill improvements with blind A/B comparison.
git clone https://github.com/shinpr/rashomon.git -
instructor-large
Modelssentence similarity model by hkunlp · sentence-transformers
hkunlp/instructor-large -
meta-agents-research-environments
Datasets & benchmarksMeta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
git clone https://github.com/facebookresearch/meta-agents-research-environments.git -
cave-agent
Agents & frameworksStateful runtime management for LLM agents—inject, manipulate, and retrieve Python objects across turns.
git clone https://github.com/acodercat/cave-agent.git -
MimirQ
AI repositories中文优先的企业 RAG 知识库:可控解析、治理、切块、混合检索、重排、引用、图谱、评测与 Dify 接入。
git clone https://github.com/skygazer42/MimirQ.git -
Awesome-Datasets-Hub
Datasets & benchmarksA curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.
git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.