Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
ragas
AI repositoriesSupercharge Your LLM Application Evaluations 🚀
git clone https://github.com/vibrantlabsai/ragas.git -
EverOS
AI repositoriesOne portable memory layer for every AI agent: local-first, Markdown-native, user-owned, and self-evolving across apps, tools, and workflows.
git clone https://github.com/EverMind-AI/EverOS.git -
giskard-oss
AI repositories🐢 Open-Source Evaluation & Testing library for LLM Agents
git clone https://github.com/Giskard-AI/giskard-oss.git -
evidently
AI repositoriesEvidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
git clone https://github.com/evidentlyai/evidently.git -
evalscope
AI repositoriesA streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
git clone https://github.com/modelscope/evalscope.git -
EvoAgentX
AI repositories🚀 EvoAgentX: Building a Self-Evolving Ecosystem of AI Agents
git clone https://github.com/ANative-Lab/EvoAgentX.git -
mastra
AI repositoriesMastra is the modern TypeScript framework for AI-powered applications and agents.
git clone https://github.com/mastra-ai/mastra.git -
tiny-universe
AI repositories《大模型白盒子构建指南》:一个全手搓的Tiny-Universe
git clone https://github.com/datawhalechina/tiny-universe.git -
oumi
AI repositoriesEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
git clone https://github.com/oumi-ai/oumi.git -
WeKnora
AI repositoriesOpen-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
git clone https://github.com/Tencent/WeKnora.git -
maka
AI repositoriesApache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.
git clone https://github.com/apache/maka.git -
PyRIT
AI repositoriesOpen-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
git clone https://github.com/microsoft/PyRIT.git -
MemOS
AI repositoriesSelf-evolving memory OS for LLM & AI Agents: ultra-persistent memory, hybrid-retrieval, and cross-task skill reuse, with 35.24% token savings and DeepSeek Harness support.
git clone https://github.com/MemTensor/MemOS.git -
agentops
AI repositoriesPython SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
git clone https://github.com/AgentOps-AI/agentops.git -
AutoGPT
AI repositoriesAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
git clone https://github.com/Significant-Gravitas/AutoGPT.git -
magpie
AI repositoriesEvery agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
git clone https://github.com/yetone/magpie.git -
failproofai
AI repositoriesObservability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.
git clone https://github.com/FailproofAI/failproofai.git -
VLMEvalKit
AI repositoriesOpen-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
git clone https://github.com/open-compass/VLMEvalKit.git -
simba
AI repositoriesOpenSource Production ready Customer service with built in Evals and monitoring
git clone https://github.com/GitHamza0206/simba.git -
LLM4AD_Next
AI repositoriesA next-generation automatic algorithm design platform, making automated algorithm design more accessible and easier to use
git clone https://github.com/Optima-CityU/LLM4AD_Next.git -
easy-dataset
AI repositoriesA powerful tool for creating datasets for LLM fine-tuning 、RAG and Eval
git clone https://github.com/ConardLi/easy-dataset.git -
AI-Infra-Guard
AI repositoriesA full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
git clone https://github.com/Tencent/AI-Infra-Guard.git -
ai-system-design-guide
AI repositoriesAI system design guide for engineers building production AI systems and evals.
git clone https://github.com/ombharatiya/ai-system-design-guide.git -
Gym
AI repositoriesEvaluate and improve models and agents using environments
git clone https://github.com/NVIDIA-NeMo/Gym.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.