InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. open_llm_leaderboard

    Tools & apps

    Hugging Face space by open-llm-leaderboard

    Indexed Hugging Face Hub, open-llm-leaderboard 14.2k ♥Updated 5 months ago 1 note

    open-llm-leaderboard/open_llm_leaderboard
  2. promptfoo

    Prompts

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    Indexed GitHub, promptfoo 27.3k ★MITUpdated today

    git clone https://github.com/promptfoo/promptfoo.git
  3. phoenix

    Prompts

    AI Observability & Evaluation

    Indexed GitHub, Arize-ai 12.3k ★Custom licence (see repository)Updated today

    git clone https://github.com/Arize-ai/phoenix.git
  4. ragas

    AI repositories

    Supercharge Your LLM Application Evaluations 🚀

    Indexed GitHub, vibrantlabsai 16.3k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/vibrantlabsai/ragas.git
  5. opik

    Prompts

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Indexed GitHub, comet-ml 23.4k ★Apache-2.0Updated today

    git clone https://github.com/comet-ml/opik.git
  6. UGI-Leaderboard

    Tools & apps

    Hugging Face space by DontPlanToEnd

    Indexed Hugging Face Hub, DontPlanToEnd 2,216 ♥Updated 30 days ago 1 note

    DontPlanToEnd/UGI-Leaderboard
  7. giskard-oss

    AI repositories

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    Indexed GitHub, Giskard-AI 5,954 ★Apache-2.0Updated today

    git clone https://github.com/Giskard-AI/giskard-oss.git
  8. evalscope

    AI repositories

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    Indexed GitHub, modelscope 3,697 ★Apache-2.0Updated today

    git clone https://github.com/modelscope/evalscope.git
  9. trulens

    Agents & frameworks

    Evaluation and Tracking for LLM Experiments and AI Agents

    Stale GitHub, truera 3,601 ★MITUpdated 34 days ago

    git clone https://github.com/truera/trulens.git
  10. Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

    Indexed GitHub, NVIDIA 1,963 ★Apache-2.0Updated today

    git clone https://github.com/NVIDIA/SkillEvaluator.git
  11. future-agi

    Agents & frameworks

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    Indexed GitHub, future-agi 2,267 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/future-agi/future-agi.git
  12. Kiln

    Prompts

    Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

    Indexed GitHub, Kiln-AI 5,283 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/Kiln-AI/Kiln.git
  13. Qwen3.8-27B

    Models

    image text to text model by Qwen · transformers

    Indexed Hugging Face Hub, Qwen 20.2k ♥apache-2.0Updated 9 days ago

    Qwen/Qwen3.8-27B
  14. langfuse

    Prompts

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    Indexed GitHub, langfuse 37.5k ★Custom licence (see repository)Updated today

    git clone https://github.com/langfuse/langfuse.git
  15. evaluation-guidebook

    Docs & tutorials

    Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

    Stale GitHub, huggingface 2,155 ★Custom licence (see repository)Updated 11 months ago

    git clone https://github.com/huggingface/evaluation-guidebook.git
  16. mlflow

    Prompts

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    Indexed GitHub, mlflow 28.8k ★Apache-2.0Updated today

    git clone https://github.com/mlflow/mlflow.git
  17. opencompass

    Datasets & benchmarks

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    Indexed GitHub, open-compass 7,596 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/open-compass/opencompass.git
  18. LLM-eval-survey

    Datasets & benchmarks

    The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

    Stale GitHub, MLGroupJLU 1,615 ★Updated 7 days ago

    git clone https://github.com/MLGroupJLU/LLM-eval-survey.git
  19. sam3

    Models

    mask generation model by facebook · transformers

    Indexed Hugging Face Hub, facebook 6,732 ♥otherUpdated 12 months ago 1 note

    facebook/sam3
  20. DeepSeek-R1

    Models

    text generation model by deepseek-ai · transformers

    Indexed Hugging Face Hub, deepseek-ai 14.5k ♥mitUpdated 19 months ago

    deepseek-ai/DeepSeek-R1
  21. lmnr

    Tools & apps

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    Indexed GitHub, lmnr-ai 3,492 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/lmnr-ai/lmnr.git
  22. A test runner for agentskills.io-style AI agent skills

    Stale GitHub, darkrishabh 802 ★MITUpdated 33 days ago

    git clone https://github.com/darkrishabh/agent-skills-eval.git
  23. helicone

    Prompts

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    Indexed GitHub, Helicone 6,281 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Helicone/helicone.git
  24. skill-up

    Skills

    Evaluate Agent Skills and environments. Evolve Skills with evidence.

    Indexed GitHub, alibaba 1,476 ★Apache-2.0Updated today

    git clone https://github.com/alibaba/skill-up.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.