Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
opik
PromptsDebug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
git clone https://github.com/comet-ml/opik.git -
promptfoo
PromptsTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
git clone https://github.com/promptfoo/promptfoo.git -
mlflow
PromptsThe open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
git clone https://github.com/mlflow/mlflow.git -
phoenix
PromptsAI Observability & Evaluation
git clone https://github.com/Arize-ai/phoenix.git -
helicone
Prompts🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
git clone https://github.com/Helicone/helicone.git -
Kiln
PromptsBuild, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
git clone https://github.com/Kiln-AI/Kiln.git -
langfuse
Prompts🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
git clone https://github.com/langfuse/langfuse.git -
ChainForge
PromptsAn open-source visual programming environment for battle-testing prompts to LLMs.
git clone https://github.com/ianarawjo/ChainForge.git -
evalstats
PromptsStatistical inference for AI evaluations, from model comparisons to corrections for LLM judge bias, optimized for small sample sizes. All defaults battle-tested in Monte Carlo simulations.
git clone https://github.com/ianarawjo/evalstats.git -
ragbits
PromptsBuilding blocks for rapid development of GenAI applications
git clone https://github.com/deepsense-ai/ragbits.git -
promptpex
PromptsTest Generation for Prompts
git clone https://github.com/microsoft/promptpex.git -
langtrace
PromptsLangtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
git clone https://github.com/Scale3-Labs/langtrace.git -
gepa
PromptsOptimize prompts, code, and more with AI-powered Reflective Optimization
git clone https://github.com/gepa-ai/gepa.git -
langwatch
PromptsOpen-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
git clone https://github.com/langwatch/langwatch.git -
awesome-gpt-6-astra
PromptsEvidence-backed use cases, prompts, integrations, evaluations, and safety notes for OpenAI GPT-6 Astra.
git clone https://github.com/Anil-matcha/awesome-gpt-6-astra.git -
uptrain
PromptsUpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
git clone https://github.com/uptrain-ai/uptrain.git -
futureagi-sdk
PromptsProduction-grade AI evaluation, prompt management & observability SDK. Automated evaluations with sub-100ms guardrails. No human-in-the-loop required. Python + TypeScript.
git clone https://github.com/future-agi/futureagi-sdk.git -
language-ai-engineering-lab
PromptsLanguage AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production.
git clone https://github.com/gil-son/language-ai-engineering-lab.git -
reforge-ai-evals
PromptsOpen-source AI evals toolkit: skills that guide AI coding agents to write eval rubrics, eval datasets (User Input Grid), trace codes, code evaluators and LLM-as-a-judge prompts. From the Reforge AI Evals course by Calibre.
git clone https://github.com/Calibre-Labs/reforge-ai-evals.git -
judgeval
PromptsThe Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
git clone https://github.com/JudgmentLabs/judgeval.git -
codex-keysmith
PromptsVersioned Codex instruction deployment with preview, ownership manifests, hook isolation, scenario evaluation, and recovery.
git clone https://github.com/Jia-Ethan/codex-keysmith.git -
promptbench
PromptsA unified evaluation framework for large language models
git clone https://github.com/microsoftarchive/promptbench.git -
Safety-Prompts
PromptsChinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts,用于评估和提升大模型的安全性。
git clone https://github.com/thu-coai/Safety-Prompts.git -
--- name: project-evaluation-for-production-decision description: A skill for evaluating projects to determine if they are ready for production, considering technical, formal, and practical aspects. --- # Project…
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.