InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. ragas

    AI repositories

    Supercharge Your LLM Application Evaluations 🚀

    Indexed GitHub, vibrantlabsai 16.3k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/vibrantlabsai/ragas.git
  2. opik

    Prompts

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Indexed GitHub, comet-ml 23.4k ★Apache-2.0Updated today

    git clone https://github.com/comet-ml/opik.git
  3. promptfoo

    Prompts

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    Indexed GitHub, promptfoo 27.3k ★MITUpdated today

    git clone https://github.com/promptfoo/promptfoo.git
  4. Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

    Indexed GitHub, NVIDIA 1,963 ★Apache-2.0Updated today

    git clone https://github.com/NVIDIA/SkillEvaluator.git
  5. giskard-oss

    AI repositories

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    Indexed GitHub, Giskard-AI 5,954 ★Apache-2.0Updated today

    git clone https://github.com/Giskard-AI/giskard-oss.git
  6. evaluation-guidebook

    Docs & tutorials

    Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

    Stale GitHub, huggingface 2,155 ★Custom licence (see repository)Updated 11 months ago

    git clone https://github.com/huggingface/evaluation-guidebook.git
  7. mlflow

    Prompts

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    Indexed GitHub, mlflow 28.8k ★Apache-2.0Updated today

    git clone https://github.com/mlflow/mlflow.git
  8. future-agi

    Agents & frameworks

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    Indexed GitHub, future-agi 2,267 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/future-agi/future-agi.git
  9. phoenix

    Prompts

    AI Observability & Evaluation

    Indexed GitHub, Arize-ai 12.3k ★Custom licence (see repository)Updated today

    git clone https://github.com/Arize-ai/phoenix.git
  10. trulens

    Agents & frameworks

    Evaluation and Tracking for LLM Experiments and AI Agents

    Stale GitHub, truera 3,601 ★MITUpdated 34 days ago

    git clone https://github.com/truera/trulens.git
  11. skill-up

    Skills

    Evaluate Agent Skills and environments. Evolve Skills with evidence.

    Indexed GitHub, alibaba 1,476 ★Apache-2.0Updated today

    git clone https://github.com/alibaba/skill-up.git
  12. helicone

    Prompts

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    Indexed GitHub, Helicone 6,281 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Helicone/helicone.git
  13. evalscope

    AI repositories

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    Indexed GitHub, modelscope 3,697 ★Apache-2.0Updated today

    git clone https://github.com/modelscope/evalscope.git
  14. Kiln

    Prompts

    Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

    Indexed GitHub, Kiln-AI 5,283 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/Kiln-AI/Kiln.git
  15. langfuse

    Prompts

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    Indexed GitHub, langfuse 37.5k ★Custom licence (see repository)Updated today

    git clone https://github.com/langfuse/langfuse.git
  16. tiny-universe

    AI repositories

    《大模型白盒子构建指南》:一个全手搓的Tiny-Universe

    Stale GitHub, datawhalechina 5,106 ★Updated 9 months ago

    git clone https://github.com/datawhalechina/tiny-universe.git
  17. text classification model by vectara · transformers

    Stale Hugging Face Hub, vectara 367 ♥apache-2.0Updated 41 days ago

    vectara/hallucination_evaluation_model
  18. lmnr

    Tools & apps

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    Indexed GitHub, lmnr-ai 3,492 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/lmnr-ai/lmnr.git
  19. ClawBench

    Datasets & benchmarks

    Open-source benchmark for browser AI agents on daily tasks.

    Stale GitHub, TIGER-AI-Lab 1,229 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/TIGER-AI-Lab/ClawBench.git
  20. opencompass

    Datasets & benchmarks

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    Indexed GitHub, open-compass 7,596 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/open-compass/opencompass.git
  21. yao-meta-skill

    Tools & apps

    YAO = Yielding AI Outcomes. A rigorous engineering, evaluation, governance, and portability system for reusable agent skills.

    Stale GitHub, yaojingang 2,732 ★MITUpdated 7 days ago

    git clone https://github.com/yaojingang/yao-meta-skill.git
  22. Gym

    AI repositories

    Evaluate and improve models and agents using environments

    Indexed GitHub, NVIDIA-NeMo 1,334 ★Apache-2.0Updated today

    git clone https://github.com/NVIDIA-NeMo/Gym.git
  23. intelligent-audit-system

    Agents & frameworks

    AuditPilot: auditable enterprise AI agents for evidence-grounded workflows, governed tools, evaluation harnesses, human review, and remediation delivery.

    Stale GitHub, Ricky-7-Yan 1,180 ★MITUpdated 11 days ago

    git clone https://github.com/Ricky-7-Yan/intelligent-audit-system.git
  24. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/oumi-ai/oumi.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.