Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
ragas
AI repositoriesSupercharge Your LLM Application Evaluations 🚀
git clone https://github.com/vibrantlabsai/ragas.git -
giskard-oss
AI repositories🐢 Open-Source Evaluation & Testing library for LLM Agents
git clone https://github.com/Giskard-AI/giskard-oss.git -
evalscope
AI repositoriesA streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
git clone https://github.com/modelscope/evalscope.git -
tiny-universe
AI repositories《大模型白盒子构建指南》:一个全手搓的Tiny-Universe
git clone https://github.com/datawhalechina/tiny-universe.git -
Gym
AI repositoriesEvaluate and improve models and agents using environments
git clone https://github.com/NVIDIA-NeMo/Gym.git -
oumi
AI repositoriesEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
git clone https://github.com/oumi-ai/oumi.git -
EnterpriseRAG-Bench
AI repositoriesDataset and benchmark for RAG on company internal documents.
git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git -
EvalForge
AI repositoriesEvaluator-neutral AI evaluation evidence, baseline regression gates, and JSON, JUnit, and SARIF reports for CI.
git clone https://github.com/jsdhwfmax/EvalForge.git -
VLMEvalKit
AI repositoriesOpen-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
git clone https://github.com/open-compass/VLMEvalKit.git -
PyRIT
AI repositoriesOpen-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
git clone https://github.com/microsoft/PyRIT.git -
AI-Infra-Guard
AI repositoriesA full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
git clone https://github.com/Tencent/AI-Infra-Guard.git -
WeKnora
AI repositoriesOpen-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
git clone https://github.com/Tencent/WeKnora.git -
prompty
AI repositoriesPrompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
git clone https://github.com/microsoft/prompty.git -
Tracely-ai
AI repositoriesTrace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
git clone https://github.com/Jwuthri/Tracely-ai.git -
XRAG
AI repositoriesXRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced Retrieval-Augmented Generation
git clone https://github.com/DocAILab/XRAG.git -
overmind
AI repositoriesThe platform for continuously improving AI agents.
git clone https://github.com/overmind-core/overmind.git -
LLM-Engineers-Handbook
AI repositoriesThe LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices
git clone https://github.com/PacktPublishing/LLM-Engineers-Handbook.git -
dingo
AI repositoriesDingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
git clone https://github.com/MigoXLab/dingo.git -
gorilla
AI repositoriesGorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
git clone https://github.com/ShishirPatil/gorilla.git -
railtracks
AI repositoriesAn agentic framework that helps developers build resilient agentic systems
git clone https://github.com/RailtownAI/railtracks.git -
DataMate
AI repositoriesDataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.
git clone https://github.com/ModelEngine-Group/DataMate.git -
rag-experiment-accelerator
AI repositoriesThe RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.
git clone https://github.com/microsoft/rag-experiment-accelerator.git -
AI-Engineer-Headquarters
AI repositoriesA collection of scientific methods, processes, algorithms, and systems to build stories & models.
git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git -
ms-cookbook
AI repositories魔搭紫皮书|ModelScope Cookbook:面向开发者的开源模型应用实战指南,覆盖模型选型、推理、微调、评测、RAG、Agent 与 AIGC,从跑通第一个模型到构建实际应用。
git clone https://github.com/modelscope/ms-cookbook.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.