Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
promptfoo
PromptsTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
git clone https://github.com/promptfoo/promptfoo.git -
phoenix
PromptsAI Observability & Evaluation
git clone https://github.com/Arize-ai/phoenix.git -
opik
PromptsDebug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
git clone https://github.com/comet-ml/opik.git -
gepa
PromptsOptimize prompts, code, and more with AI-powered Reflective Optimization
git clone https://github.com/gepa-ai/gepa.git -
helicone
Prompts🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
git clone https://github.com/Helicone/helicone.git -
judgeval
PromptsThe Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
git clone https://github.com/JudgmentLabs/judgeval.git -
langfuse
Prompts🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
git clone https://github.com/langfuse/langfuse.git -
mlflow
PromptsThe open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
git clone https://github.com/mlflow/mlflow.git -
ai-reliability-copilot
PromptsTurn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.
git clone https://github.com/YanpengQi7/ai-reliability-copilot.git -
reforge-ai-evals
PromptsOpen-source AI evals toolkit: skills that guide AI coding agents to write eval rubrics, eval datasets (User Input Grid), trace codes, code evaluators and LLM-as-a-judge prompts. From the Reforge AI Evals course by Calibre.
git clone https://github.com/Calibre-Labs/reforge-ai-evals.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.