InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. BenchMARL

    Agents & frameworks

    BenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL). BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded in its two core tenets: reproducibility and standardization.

    Stale GitHub, facebookresearch 672 ★MITUpdated 9 months ago

    git clone https://github.com/facebookresearch/BenchMARL.git
  2. agent-memory-benchmark

    Agents & frameworks

    Agent Memory Benchmark

    Stale GitHub, vectorize-io 159 ★Updated 16 days ago

    git clone https://github.com/vectorize-io/agent-memory-benchmark.git
  3. chinese-llm-benchmark

    Agents & frameworks

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    Indexed GitHub, jeinlee1991 6,518 ★Updated yesterday

    git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
  4. ai-agents-the-definitive-guide

    Agents & frameworks

    Repo for AI Agents The Definitive Guide

    Stale GitHub, Nicolepcx 2,747 ★Updated 3 months ago

    git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git
  5. benchmarks

    Agents & frameworks

    Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

    Stale GitHub, coval-ai 104 ★Custom licence (see repository)Updated 24 days ago

    git clone https://github.com/coval-ai/benchmarks.git
  6. Bittune

    Agents & frameworks

    Local-first engineering agent that deploys, benchmarks and tunes vLLM/SGLang inference services on your own GPUs, with auditable run evidence.

    Indexed GitHub, bitcloud-ai 304 ★MITUpdated 3 days ago

    git clone https://github.com/bitcloud-ai/Bittune.git
  7. AgentBench

    Agents & frameworks

    A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

    Stale GitHub, THUDM 3,772 ★Apache-2.0Updated 9 months ago

    git clone https://github.com/THUDM/AgentBench.git
  8. GateMem

    Agents & frameworks

    GateMem: a benchmark and evaluation toolkit for memory governance in multi-principal shared-memory LLM agents.

    Stale GitHub, rzhub 139 ★MITUpdated 35 days ago

    git clone https://github.com/rzhub/GateMem.git
  9. WebCanvas

    Agents & frameworks

    All-in-one Web Agent framework for post-training. Start building with a few clicks!

    Stale GitHub, iMeanAI 281 ★MITUpdated 16 months ago

    git clone https://github.com/iMeanAI/WebCanvas.git
  10. OpenRCA

    Agents & frameworks

    [ICLR'25] OpenRCA: Can Large Language Models Locate the Root Cause of Software Failures?

    Stale GitHub, microsoft 451 ★MITUpdated 12 days ago

    git clone https://github.com/microsoft/OpenRCA.git
  11. mas-lab

    Agents & frameworks

    Specification-driven multi-agent systems with OpenTelemetry observability, agent traces, benchmarking, MCP, and A2A.

    Indexed GitHub, ioc-project 65 ★Apache-2.0Updated yesterday

    git clone https://github.com/ioc-project/mas-lab.git
  12. Awesome-CUA

    Agents & frameworks

    Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

    Stale GitHub, reacher-z 659 ★Custom licence (see repository)Updated 6 days ago

    git clone https://github.com/reacher-z/Awesome-CUA.git
  13. embodied-first-exam

    Agents & frameworks

    The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

    Stale GitHub, mll-lab-nu 236 ★MITUpdated 10 days ago

    git clone https://github.com/mll-lab-nu/embodied-first-exam.git
  14. AlgoTune

    Agents & frameworks

    AlgoTune is a NeurIPS 2025 benchmark made up of 154 math, physics, and computer science problems. The goal is write code that solves each problem, and is faster than existing implementations.

    Stale GitHub, oripress 118 ★MITUpdated 4 months ago

    git clone https://github.com/oripress/AlgoTune.git
  15. MiroFlow

    Agents & frameworks

    🏆 Top-1 on 5+ benchmarks | Web UI | Supports MiroThinker, Claude, Kimi, OpenAI

    Stale GitHub, MiroMindAI 3,120 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/MiroMindAI/MiroFlow.git
  16. Agents_Failure_Attribution

    Agents & frameworks

    Benchmark for automated failure attributions in agentic systems (🏆 ICML 2025 Spotlight)

    Stale GitHub, ag2ai 397 ★MITUpdated 17 days ago

    git clone https://github.com/ag2ai/Agents_Failure_Attribution.git
  17. RouterArena

    Agents & frameworks

    RouterArena: An open framework for evaluating LLM routers with standardized datasets, metrics, an automated framework, and a live leaderboard.

    Stale GitHub, RouteWorks 157 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/RouteWorks/RouterArena.git
  18. artemis

    Agents & frameworks

    ARTEMIS turns natural-language instructions into reliable Android automation. It automates end-to-end workflows, captures logs, and integrates seamlessly with AI coding assistants such as Antigravity, Codex, and Claude Code. It also achieves 99%+ success rate on AndroidWorld Benchmark.

    Stale GitHub, google 11.7k ★Apache-2.0Updated 46 days ago

    git clone https://github.com/google/artemis.git
  19. all-agentic-architectures

    Agents & frameworks

    35 production-grade agentic AI architectures (Reflexion, LATS, GraphRAG, MemGPT, Voyager, BrowserAgent, ...) — a Python library and runnable textbook with multi-provider LLM support and a 17-task benchmark leaderboard.

    Indexed GitHub, FareedKhan-dev 4,756 ★MITUpdated today

    git clone https://github.com/FareedKhan-dev/all-agentic-architectures.git
  20. agy-staff

    Agents & frameworks

    Hire Google's Antigravity CLI (agy) as a fast Gemini staffer for Claude Code, OpenAI Codex, and DeepSeek Harness.

    Indexed GitHub, keli-wen 860 ★MITUpdated 4 days ago

    git clone https://github.com/keli-wen/agy-staff.git
  21. overtime

    Agents & frameworks

    Give a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.

    Stale GitHub, ukanwat 399 ★MITUpdated 30 days ago

    git clone https://github.com/ukanwat/overtime.git
  22. GTA

    Agents & frameworks

    [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2

    Indexed GitHub, open-compass 153 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/open-compass/GTA.git
  23. aideml

    Agents & frameworks

    AIDE: an LLM agent for machine learning engineering - the research Weco grew out of. Referenced in OpenAI MLE-bench.

    Stale GitHub, WecoAI 1,575 ★MITUpdated 33 days ago

    git clone https://github.com/WecoAI/aideml.git
  24. windmill

    Agents & frameworks

    Code-first platform for workflows, apps and AI agents, on the fastest self-hostable orchestrator (benchmarked against Airflow, Prefect and Temporal). Business teams build with AI, engineers control what ships. Open-source alternative to Lovable, Retool and n8n.

    Indexed GitHub, windmill-labs 18.6k ★Custom licence (see repository)Updated today

    git clone https://github.com/windmill-labs/windmill.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.