Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
agent-memory-benchmark
Agents & frameworksAgent Memory Benchmark
git clone https://github.com/vectorize-io/agent-memory-benchmark.git -
chinese-llm-benchmark
Agents & frameworks非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git -
benchmarks
Agents & frameworksReproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.
git clone https://github.com/coval-ai/benchmarks.git -
ai-agents-the-definitive-guide
Agents & frameworksRepo for AI Agents The Definitive Guide
git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git -
Bittune
Agents & frameworksLocal-first engineering agent that deploys, benchmarks and tunes vLLM/SGLang inference services on your own GPUs, with auditable run evidence.
git clone https://github.com/bitcloud-ai/Bittune.git -
GateMem
Agents & frameworksGateMem: a benchmark and evaluation toolkit for memory governance in multi-principal shared-memory LLM agents.
git clone https://github.com/rzhub/GateMem.git -
AgentBench
Agents & frameworksA Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
git clone https://github.com/THUDM/AgentBench.git -
BenchMARL
Agents & frameworksBenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL). BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded in its two core tenets: reproducibility and standardization.
git clone https://github.com/facebookresearch/BenchMARL.git -
WebCanvas
Agents & frameworksAll-in-one Web Agent framework for post-training. Start building with a few clicks!
git clone https://github.com/iMeanAI/WebCanvas.git -
mas-lab
Agents & frameworksSpecification-driven multi-agent systems with OpenTelemetry observability, agent traces, benchmarking, MCP, and A2A.
git clone https://github.com/ioc-project/mas-lab.git -
OpenRCA
Agents & frameworks[ICLR'25] OpenRCA: Can Large Language Models Locate the Root Cause of Software Failures?
git clone https://github.com/microsoft/OpenRCA.git -
Awesome-CUA
Agents & frameworksOpen, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
git clone https://github.com/reacher-z/Awesome-CUA.git -
embodied-first-exam
Agents & frameworksThe first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
git clone https://github.com/mll-lab-nu/embodied-first-exam.git -
AlgoTune
Agents & frameworksAlgoTune is a NeurIPS 2025 benchmark made up of 154 math, physics, and computer science problems. The goal is write code that solves each problem, and is faster than existing implementations.
git clone https://github.com/oripress/AlgoTune.git -
MiroFlow
Agents & frameworks🏆 Top-1 on 5+ benchmarks | Web UI | Supports MiroThinker, Claude, Kimi, OpenAI
git clone https://github.com/MiroMindAI/MiroFlow.git -
Agents_Failure_Attribution
Agents & frameworksBenchmark for automated failure attributions in agentic systems (🏆 ICML 2025 Spotlight)
git clone https://github.com/ag2ai/Agents_Failure_Attribution.git -
RouterArena
Agents & frameworksRouterArena: An open framework for evaluating LLM routers with standardized datasets, metrics, an automated framework, and a live leaderboard.
git clone https://github.com/RouteWorks/RouterArena.git -
overtime
Agents & frameworksGive a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.
git clone https://github.com/ukanwat/overtime.git -
agy-staff
Agents & frameworksHire Google's Antigravity CLI (agy) as a fast Gemini staffer for Claude Code, OpenAI Codex, and DeepSeek Harness.
git clone https://github.com/keli-wen/agy-staff.git -
all-agentic-architectures
Agents & frameworks35 production-grade agentic AI architectures (Reflexion, LATS, GraphRAG, MemGPT, Voyager, BrowserAgent, ...) — a Python library and runnable textbook with multi-provider LLM support and a 17-task benchmark leaderboard.
git clone https://github.com/FareedKhan-dev/all-agentic-architectures.git -
artemis
Agents & frameworksARTEMIS turns natural-language instructions into reliable Android automation. It automates end-to-end workflows, captures logs, and integrates seamlessly with AI coding assistants such as Antigravity, Codex, and Claude Code. It also achieves 99%+ success rate on AndroidWorld Benchmark.
git clone https://github.com/google/artemis.git -
GTA
Agents & frameworks[NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2
git clone https://github.com/open-compass/GTA.git -
Research-Assistant-Agent
Agents & frameworksMulti-agent research execution system: paper reproduction, benchmark evaluation, AutoResearch and claim-evidence graphs
git clone https://github.com/zzp1221/Research-Assistant-Agent.git -
windmill
Agents & frameworksCode-first platform for workflows, apps and AI agents, on the fastest self-hostable orchestrator (benchmarked against Airflow, Prefect and Temporal). Business teams build with AI, engineers control what ships. Open-source alternative to Lovable, Retool and n8n.
git clone https://github.com/windmill-labs/windmill.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.