Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
uptrain
PromptsUpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
git clone https://github.com/uptrain-ai/uptrain.git -
Simreal-MLBench
Datasets & benchmarks[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.
git clone https://github.com/Simreal-AI/Simreal-MLBench.git -
ResearchClawBench
Datasets & benchmarks🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery
git clone https://github.com/InternScience/ResearchClawBench.git -
OctoCodingBench
Datasets & benchmarksHugging Face dataset by MiniMaxAI
MiniMaxAI/OctoCodingBench -
agentenv-framework
MCP serversCreating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.
git clone https://github.com/scaleapi/agentenv-framework.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
agent-skills-eval
SkillsA test runner for agentskills.io-style AI agent skills
git clone https://github.com/darkrishabh/agent-skills-eval.git -
ResolveLab
Agents & frameworksFault-injected tool environments with side-effect grading for evaluating and post-training LLM agents
git clone https://github.com/Ta0p1/ResolveLab.git -
futureagi-sdk
PromptsProduction-grade AI evaluation, prompt management & observability SDK. Automated evaluations with sub-100ms guardrails. No human-in-the-loop required. Python + TypeScript.
git clone https://github.com/future-agi/futureagi-sdk.git -
DataMate
AI repositoriesDataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.
git clone https://github.com/ModelEngine-Group/DataMate.git -
rag-experiment-accelerator
AI repositoriesThe RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.
git clone https://github.com/microsoft/rag-experiment-accelerator.git -
WebCanvas
Agents & frameworksAll-in-one Web Agent framework for post-training. Start building with a few clicks!
git clone https://github.com/iMeanAI/WebCanvas.git -
RouterArena
Agents & frameworksRouterArena: An open framework for evaluating LLM routers with standardized datasets, metrics, an automated framework, and a live leaderboard.
git clone https://github.com/RouteWorks/RouterArena.git -
Omni-SafetyBench
Datasets & benchmarks[ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".
git clone https://github.com/THU-BPM/Omni-SafetyBench.git -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
AI-Engineer-Headquarters
AI repositoriesA collection of scientific methods, processes, algorithms, and systems to build stories & models.
git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git -
language-ai-engineering-lab
PromptsLanguage AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production.
git clone https://github.com/gil-son/language-ai-engineering-lab.git -
fable-method
SkillsThe Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.
git clone https://github.com/Sahir619/fable-method.git -
Awesome-LLM-Long-Context-Modeling
Datasets & benchmarks📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git -
SkillCorpus
SkillsOpen-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.
git clone https://github.com/EverMind-AI/SkillCorpus.git -
Awesome-LLM-in-Social-Science
Agents & frameworksAwesome papers involving LLMs in Social Science.
git clone https://github.com/ValueByte-AI/Awesome-LLM-in-Social-Science.git -
qiaomu-meta-skill
Skills把工作流变成可研究、可评测、可发布的乔木 Agent Skill | Turn workflows into researched, tested, release-ready agent skills.
git clone https://github.com/joeseesun/qiaomu-meta-skill.git -
xVerify
Datasets & benchmarksxVerify: Efficient Answer Verifier for Reasoning Model Evaluations
git clone https://github.com/IAAR-Shanghai/xVerify.git -
ms-cookbook
AI repositories魔搭紫皮书|ModelScope Cookbook:面向开发者的开源模型应用实战指南,覆盖模型选型、推理、微调、评测、RAG、Agent 与 AIGC,从跑通第一个模型到构建实际应用。
git clone https://github.com/modelscope/ms-cookbook.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.