InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. SciCode

    Datasets & benchmarks

    A benchmark that challenges language models to code solutions for scientific problems

    Stale GitHub, scicode-bench 246 ★Apache-2.0Updated 15 days ago

    git clone https://github.com/scicode-bench/SciCode.git
  2. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  3. Hugging Face dataset by ProGamerGov

    Stale Hugging Face Hub, ProGamerGov 158 ♥mitUpdated 2 years ago

    ProGamerGov/synthetic-dataset-1m-dalle3-high-quality-captions
  4. inferock-bench

    Datasets & benchmarks

    Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.

    Stale GitHub, inferock 107 ★Custom licence (see repository)Updated 3 months ago

    git clone https://github.com/inferock/inferock-bench.git
  5. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  6. autoguardrails

    Datasets & benchmarks

    Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

    Stale GitHub, SantanderAI 129 ★Apache-2.0Updated 47 days ago

    git clone https://github.com/SantanderAI/autoguardrails.git
  7. TwinFix

    Datasets & benchmarks

    TwinFix — AI agent that diagnoses, reproduces and fixes bugs in a disposable twin environment. 在一次性孪生环境中自动诊断、复现并修复故障的 AI Agent。

    Indexed GitHub, spacesky-cell 117 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/spacesky-cell/TwinFix.git
  8. Simreal-MLBench

    Datasets & benchmarks

    [Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

    Stale GitHub, Simreal-AI 301 ★Apache-2.0Updated 54 days ago

    git clone https://github.com/Simreal-AI/Simreal-MLBench.git
  9. agent-orchestra

    Datasets & benchmarks

    Agents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.

    Stale GitHub, 3338902669-ops 93 ★Apache-2.0Updated 25 days ago

    git clone https://github.com/3338902669-ops/agent-orchestra.git
  10. awesome-ai-gateway

    Datasets & benchmarks

    ⚡ Awesome AI Gateway — pick an AI gateway from 240+ (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security & compliance, see what consolidated in 2026, learn how they actually work from an 8-chapter handbook, and verify every number yourself. Reproducible benchmarks, bilingual; stars & releases refreshed daily.

    Indexed GitHub, cuihuan 158 ★CC0-1.0Updated 2 days ago

    git clone https://github.com/cuihuan/awesome-ai-gateway.git
  11. apple-silicon-llm-bench

    Datasets & benchmarks

    Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

    Indexed GitHub, john-rocky 73 ★MITUpdated yesterday

    git clone https://github.com/john-rocky/apple-silicon-llm-bench.git
  12. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 5 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  13. teor

    Datasets & benchmarks

    Benchmark aberto de LLMs para o processo legislativo brasileiro

    Stale GitHub, como-votam 30 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/como-votam/teor.git
  14. agent-duel

    Datasets & benchmarks

    Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

    Stale GitHub, bottomless 35 ★Custom licence (see repository)Updated 6 days ago

    git clone https://github.com/bottomless/agent-duel.git
  15. bigcodebench

    Datasets & benchmarks

    [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

    Stale Archived GitHub, bigcode-project 516 ★Apache-2.0Updated 10 months ago 1 note

    git clone https://github.com/bigcode-project/bigcodebench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.