Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
langfuse
Prompts🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
git clone https://github.com/langfuse/langfuse.git -
mlflow
PromptsThe open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
git clone https://github.com/mlflow/mlflow.git -
prompty
AI repositoriesPrompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
git clone https://github.com/microsoft/prompty.git -
EvalForge
AI repositoriesEvaluator-neutral AI evaluation evidence, baseline regression gates, and JSON, JUnit, and SARIF reports for CI.
git clone https://github.com/jsdhwfmax/EvalForge.git -
deepresearch-agent
Agents & frameworks一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。
git clone https://github.com/qiqihezh/deepresearch-agent.git -
palico-ai
Agents & frameworksBuild, Improve Performance, and Productionize your AI Application
git clone https://github.com/palico-ai/palico-ai.git -
coding.sh
MCP serversOpen-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.
git clone https://github.com/jackulau/coding.sh.git -
LLM Eval Harness
MCP serversRun a golden set and gate LLM model swaps on accuracy, with latency and cost per request.
ghcr.io/razilzul/llm-eval-harness:1.0.0 -
generative-ai
Agents & frameworksComprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
git clone https://github.com/genieincodebottle/generative-ai.git -
AI-Infra-Guard
AI repositoriesA full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
git clone https://github.com/Tencent/AI-Infra-Guard.git -
DecisionPivot
Datasets & benchmarksDecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
git clone https://github.com/StartLuxLabs/DecisionPivot.git -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
opencode-skill-creator
SkillsCreate, test, and optimize OpenCode skills. Supports OpenCode V2 and V1 ≥ 1.18.29. Adapted from Anthropic's skill-creator.
git clone https://github.com/antongulin/opencode-skill-creator.git -
EnterpriseRAG-Bench
AI repositoriesDataset and benchmark for RAG on company internal documents.
git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git -
nasde-toolkit
SkillsCLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys.
git clone https://github.com/NoesisVision/nasde-toolkit.git -
future-agi
Agents & frameworksOpen-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
git clone https://github.com/future-agi/future-agi.git -
chinese-llm-benchmark
Agents & frameworks非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git -
overtime
Agents & frameworksGive a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.
git clone https://github.com/ukanwat/overtime.git -
Tracely-ai
AI repositoriesTrace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
git clone https://github.com/Jwuthri/Tracely-ai.git -
humanities_data_benchmark
Datasets & benchmarksLLM Benchmark Suite for Humanities Data
git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git -
tuui
MCP serversA desktop MCP client designed as a tool unitary utility integration, accelerating AI adoption through the Model Context Protocol (MCP) and enabling cross-vendor LLM API orchestration.
git clone https://github.com/AI-QL/tuui.git -
labelhub-platform
MCP servers面向 LLM 训练数据生产的可信工作台:AI 提供证据、人类裁决,append-only 账本、可复现导出、评测关卡与 MCP Server。
git clone https://github.com/Iceberg86-gb/labelhub-platform.git -
cap-evolve
MCP serversOptimize any AI agent’s skills, tools/MCP, and prompts against your own evals.
git clone https://github.com/skillberry-ai/cap-evolve.git -
evaluation-guidebook
Docs & tutorialsSharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
git clone https://github.com/huggingface/evaluation-guidebook.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.