InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. phoenix

    Prompts

    AI Observability & Evaluation

    Indexed GitHub, Arize-ai 12.3k ★Custom licence (see repository)Updated today

    git clone https://github.com/Arize-ai/phoenix.git
  2. promptfoo

    Prompts

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    Indexed GitHub, promptfoo 27.3k ★MITUpdated today

    git clone https://github.com/promptfoo/promptfoo.git
  3. trulens

    Agents & frameworks

    Evaluation and Tracking for LLM Experiments and AI Agents

    Stale GitHub, truera 3,601 ★MITUpdated 34 days ago

    git clone https://github.com/truera/trulens.git
  4. LLM-eval-survey

    Datasets & benchmarks

    The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

    Stale GitHub, MLGroupJLU 1,615 ★Updated 7 days ago

    git clone https://github.com/MLGroupJLU/LLM-eval-survey.git
  5. giskard-oss

    AI repositories

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    Indexed GitHub, Giskard-AI 5,954 ★Apache-2.0Updated today

    git clone https://github.com/Giskard-AI/giskard-oss.git
  6. Awesome-LLM-Eval

    Datasets & benchmarks

    Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.

    Stale GitHub, onejune2018 659 ★MITUpdated 11 months ago

    git clone https://github.com/onejune2018/Awesome-LLM-Eval.git
  7. A test runner for agentskills.io-style AI agent skills

    Stale GitHub, darkrishabh 802 ★MITUpdated 33 days ago

    git clone https://github.com/darkrishabh/agent-skills-eval.git
  8. agentic_security

    Agents & frameworks

    Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

    Indexed GitHub, msoedov 2,046 ★Apache-2.0Updated yesterday

    git clone https://github.com/msoedov/agentic_security.git
  9. opencompass

    Datasets & benchmarks

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    Indexed GitHub, open-compass 7,596 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/open-compass/opencompass.git
  10. opik

    Prompts

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Indexed GitHub, comet-ml 23.4k ★Apache-2.0Updated today

    git clone https://github.com/comet-ml/opik.git
  11. overmind

    AI repositories

    The platform for continuously improving AI agents.

    Stale GitHub, overmind-core 753 ★AGPL-3.0Updated 7 days ago

    git clone https://github.com/overmind-core/overmind.git
  12. LLM-Engineers-Handbook

    AI repositories

    The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

    Stale GitHub, PacktPublishing 5,377 ★MITUpdated 6 months ago

    git clone https://github.com/PacktPublishing/LLM-Engineers-Handbook.git
  13. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  14. WebCanvas

    Agents & frameworks

    All-in-one Web Agent framework for post-training. Start building with a few clicks!

    Stale GitHub, iMeanAI 281 ★MITUpdated 16 months ago

    git clone https://github.com/iMeanAI/WebCanvas.git
  15. gepa

    Prompts

    Optimize prompts, code, and more with AI-powered Reflective Optimization

    Stale GitHub, gepa-ai 7,394 ★MITUpdated 7 days ago

    git clone https://github.com/gepa-ai/gepa.git
  16. GTA

    Agents & frameworks

    [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2

    Indexed GitHub, open-compass 153 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/open-compass/GTA.git
  17. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/oumi-ai/oumi.git
  18. Awesome-LLM-in-Social-Science

    Agents & frameworks

    Awesome papers involving LLMs in Social Science.

    Indexed GitHub, ValueByte-AI 653 ★MITUpdated 2 days ago

    git clone https://github.com/ValueByte-AI/Awesome-LLM-in-Social-Science.git
  19. lmnr

    Tools & apps

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    Indexed GitHub, lmnr-ai 3,492 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/lmnr-ai/lmnr.git
  20. helicone

    Prompts

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    Indexed GitHub, Helicone 6,281 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Helicone/helicone.git
  21. CivAgent

    Agents & frameworks

    CivAgent is an LLM-based Human-like Agent acting as a Digital Player within the Strategy Game Unciv.

    Stale GitHub, fuxiAIlab 167 ★MPL-2.0Updated 20 months ago

    git clone https://github.com/fuxiAIlab/CivAgent.git
  22. AI-Engineer-Headquarters

    AI repositories

    A collection of scientific methods, processes, algorithms, and systems to build stories & models.

    Stale GitHub, hemansnation 3,776 ★Updated 19 days ago

    git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git
  23. cave-agent

    Agents & frameworks

    Stateful runtime management for LLM agents—inject, manipulate, and retrieve Python objects across turns.

    Stale GitHub, acodercat 204 ★MITUpdated 2 months ago

    git clone https://github.com/acodercat/cave-agent.git
  24. judgeval

    Prompts

    The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

    Indexed GitHub, JudgmentLabs 1,074 ★Apache-2.0Updated today

    git clone https://github.com/JudgmentLabs/judgeval.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.