InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. ragas

    AI repositories

    Supercharge Your LLM Application Evaluations 🚀

    Indexed GitHub, vibrantlabsai 16.3k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/vibrantlabsai/ragas.git
  2. giskard-oss

    AI repositories

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    Indexed GitHub, Giskard-AI 5,954 ★Apache-2.0Updated today

    git clone https://github.com/Giskard-AI/giskard-oss.git
  3. evalscope

    AI repositories

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    Indexed GitHub, modelscope 3,697 ★Apache-2.0Updated today

    git clone https://github.com/modelscope/evalscope.git
  4. tiny-universe

    AI repositories

    《大模型白盒子构建指南》:一个全手搓的Tiny-Universe

    Stale GitHub, datawhalechina 5,106 ★Updated 9 months ago

    git clone https://github.com/datawhalechina/tiny-universe.git
  5. Gym

    AI repositories

    Evaluate and improve models and agents using environments

    Indexed GitHub, NVIDIA-NeMo 1,334 ★Apache-2.0Updated today

    git clone https://github.com/NVIDIA-NeMo/Gym.git
  6. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/oumi-ai/oumi.git
  7. EnterpriseRAG-Bench

    AI repositories

    Dataset and benchmark for RAG on company internal documents.

    Stale GitHub, onyx-dot-app 592 ★MITUpdated 2 months ago

    git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git
  8. EvalForge

    AI repositories

    Evaluator-neutral AI evaluation evidence, baseline regression gates, and JSON, JUnit, and SARIF reports for CI.

    Indexed GitHub, jsdhwfmax 194 ★MITUpdated today

    git clone https://github.com/jsdhwfmax/EvalForge.git
  9. VLMEvalKit

    AI repositories

    Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

    Indexed GitHub, open-compass 4,487 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/open-compass/VLMEvalKit.git
  10. PyRIT

    AI repositories

    Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.

    Indexed GitHub, microsoft 4,869 ★MITUpdated today

    git clone https://github.com/microsoft/PyRIT.git
  11. AI-Infra-Guard

    AI repositories

    A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

    Stale GitHub, Tencent 7,471 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Tencent/AI-Infra-Guard.git
  12. WeKnora

    AI repositories

    Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

    Indexed GitHub, Tencent 37.5k ★Custom licence (see repository)Updated today

    git clone https://github.com/Tencent/WeKnora.git
  13. prompty

    AI repositories

    Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.

    Stale GitHub, microsoft 1,288 ★MITUpdated 7 days ago

    git clone https://github.com/microsoft/prompty.git
  14. Tracely-ai

    AI repositories

    Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.

    Stale GitHub, Jwuthri 1,311 ★MITUpdated 8 days ago

    git clone https://github.com/Jwuthri/Tracely-ai.git
  15. XRAG

    AI repositories

    XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced Retrieval-Augmented Generation

    Stale GitHub, DocAILab 611 ★Apache-2.0Updated 15 days ago

    git clone https://github.com/DocAILab/XRAG.git
  16. overmind

    AI repositories

    The platform for continuously improving AI agents.

    Stale GitHub, overmind-core 753 ★AGPL-3.0Updated 7 days ago

    git clone https://github.com/overmind-core/overmind.git
  17. LLM-Engineers-Handbook

    AI repositories

    The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

    Stale GitHub, PacktPublishing 5,377 ★MITUpdated 6 months ago

    git clone https://github.com/PacktPublishing/LLM-Engineers-Handbook.git
  18. dingo

    AI repositories

    Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

    Stale GitHub, MigoXLab 764 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/MigoXLab/dingo.git
  19. gorilla

    AI repositories

    Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

    Stale GitHub, ShishirPatil 13.1k ★Apache-2.0Updated 7 months ago

    git clone https://github.com/ShishirPatil/gorilla.git
  20. railtracks

    AI repositories

    An agentic framework that helps developers build resilient agentic systems

    Indexed GitHub, RailtownAI 235 ★MITUpdated 3 days ago

    git clone https://github.com/RailtownAI/railtracks.git
  21. DataMate

    AI repositories

    DataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.

    Stale GitHub, ModelEngine-Group 374 ★MITUpdated 5 days ago

    git clone https://github.com/ModelEngine-Group/DataMate.git
  22. rag-experiment-accelerator

    AI repositories

    The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

    Indexed GitHub, microsoft 320 ★Custom licence (see repository)Updated today

    git clone https://github.com/microsoft/rag-experiment-accelerator.git
  23. AI-Engineer-Headquarters

    AI repositories

    A collection of scientific methods, processes, algorithms, and systems to build stories & models.

    Stale GitHub, hemansnation 3,776 ★Updated 19 days ago

    git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git
  24. ms-cookbook

    AI repositories

    魔搭紫皮书|ModelScope Cookbook:面向开发者的开源模型应用实战指南,覆盖模型选型、推理、微调、评测、RAG、Agent 与 AIGC,从跑通第一个模型到构建实际应用。

    Stale GitHub, modelscope 518 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/modelscope/ms-cookbook.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.