Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
vidore-benchmark
AI repositoriesVision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
git clone https://github.com/illuin-tech/vidore-benchmark.git -
benchmark-qed
AI repositoriesAutomated benchmarking of Retrieval-Augmented Generation (RAG) systems
git clone https://github.com/microsoft/benchmark-qed.git -
mempalace
AI repositoriesThe best-benchmarked open-source AI memory system. And it's free.
git clone https://github.com/MemPalace/mempalace.git -
folkbench
AI repositories帮你的 AI 编程工具找到不降智、不断连、高性价比的好线路 · Live LLM relay rankings, model integrity probes & Tauri/Rust switcher for Claude Code, Codex & Gemini CLI
git clone https://github.com/Folkbench/folkbench.git -
beir
AI repositoriesA Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
git clone https://github.com/beir-cellar/beir.git -
SDGym
AI repositoriesBenchmarking synthetic data generation methods.
git clone https://github.com/sdv-dev/SDGym.git -
docext
AI repositoriesAn on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)
git clone https://github.com/NanoNets/docext.git -
Gym
AI repositoriesEvaluate and improve models and agents using environments
git clone https://github.com/NVIDIA-NeMo/Gym.git -
toon
AI repositories🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.
git clone https://github.com/toon-format/toon.git -
DreamLayer-Eval
AI repositoriesDreamLayer Eval: open-source benchmarking for image and video diffusion models. Automate prompts, seeds, and metrics for reproducible results. A research project from DreamLayer AI (dreamlayer.io).
git clone https://github.com/TheDesignFounder/DreamLayer-Eval.git -
EnterpriseRAG-Bench
AI repositoriesDataset and benchmark for RAG on company internal documents.
git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git -
evalscope
AI repositoriesA streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
git clone https://github.com/modelscope/evalscope.git -
agentic-ai
AI repositoriesAgentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.
git clone https://github.com/mahmoudrabie/agentic-ai.git -
XRAG
AI repositoriesXRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced Retrieval-Augmented Generation
git clone https://github.com/DocAILab/XRAG.git -
VLM2Vec
AI repositoriesThis repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
git clone https://github.com/TIGER-AI-Lab/VLM2Vec.git -
somali-language-standard
AI repositoriesThe open, machine-readable standard for the Somali language — orthography, grammar, terminology, translation, and AI/benchmark resources, as a versioned, citable RFC-style catalog.
git clone https://github.com/somastllc/somali-language-standard.git -
Introduction-to-Quantitative-Finance
AI repositoriesAI+金融(量化):1.多因子股票量化框架开源教程 2.学界和业界的经典资料收录 3.AI + 金融的相关工作,包括LLM, Agent, benchmark(evaluation), etc.
git clone https://github.com/Barca0412/Introduction-to-Quantitative-Finance.git -
code-review-graph
AI repositoriesLocal-first code intelligence graph for MCP and CLI. Builds a persistent map of your codebase so AI coding tools read only what matters, with benchmarked context reductions on reviews and large-repo workflows.
git clone https://github.com/tirth8205/code-review-graph.git -
Awesome-GraphRAG
AI repositoriesAwesome-GraphRAG: A curated list of resources (surveys, papers, benchmarks, and opensource projects) on graph-based retrieval-augmented generation.
git clone https://github.com/DEEP-PolyU/Awesome-GraphRAG.git -
VLMEvalKit
AI repositoriesOpen-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
git clone https://github.com/open-compass/VLMEvalKit.git -
agentops
AI repositoriesPython SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
git clone https://github.com/AgentOps-AI/agentops.git -
awesome-decision-models
AI repositoriesA curated list of decision models (System One / typed decision models): hosted APIs, open-weight models, runtimes, SDKs, applications, benchmarks, and papers.
git clone https://github.com/AnotiaWang/awesome-decision-models.git -
fastRAG
AI repositoriesEfficient Retrieval Augmentation and Generation Framework
git clone https://github.com/IntelLabs/fastRAG.git -
hermes-official-plugins
AI repositories☤ The official plugins for Hermes Agent, built and maintained by Nous Research: Home Assistant, Spotify, Claude Subscription DirectSDK and more, one install away from the Hermes plugin catalog.
git clone https://github.com/NousResearch/hermes-official-plugins.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.