InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  2. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  3. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  4. active_genie

    Datasets & benchmarks

    The Lodash for GenAI: Real Value + Consistent + Model-Agnostic

    Stale GitHub, Roriz 168 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/Roriz/active_genie.git
  5. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  6. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  7. ollama-webui-intel

    Datasets & benchmarks

    Ollama with intel (i)GPU acceleration in docker and benchmark

    Indexed GitHub, NikolasEnt 53 ★Updated 2 days ago

    git clone https://github.com/NikolasEnt/ollama-webui-intel.git
  8. Awesome-AI4AI

    Datasets & benchmarks

    Updated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.

    Stale GitHub, KaiWU5 218 ★MITUpdated 8 days ago

    git clone https://github.com/KaiWU5/Awesome-AI4AI.git
  9. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 3 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  10. Awesome-LLM-Eval

    Datasets & benchmarks

    Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.

    Stale GitHub, onejune2018 659 ★MITUpdated 11 months ago

    git clone https://github.com/onejune2018/Awesome-LLM-Eval.git
  11. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  12. turnkeyml

    Datasets & benchmarks

    No-code CLI designed for accelerating ONNX workflows

    Stale GitHub, onnx 247 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/onnx/turnkeyml.git
  13. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  14. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  15. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  16. UHGEval

    Datasets & benchmarks

    [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

    Stale GitHub, IAAR-Shanghai 183 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/IAAR-Shanghai/UHGEval.git
  17. teor

    Datasets & benchmarks

    Benchmark aberto de LLMs para o processo legislativo brasileiro

    Stale GitHub, como-votam 30 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/como-votam/teor.git
  18. ai-model-world

    Datasets & benchmarks

    AI 大模型世界,把 556 个大模型拟人化成像素小人的可视化站点。进来就能看到此刻谁最聪明、谁最会写代码、谁最便宜、谁刚发布,往下是国内与国外分区的厂商广场、完整的发布时间线和多维排行榜。搜索认模型名、厂商和能力,输入「多模态」会直接列出全部多模态模型。数据取自 Epoch AI、models.dev、LiveBench 与 Hugging Face,每小时自动同步,所有文案由真实数据生成,不调用任何 LLM。Next.js 静态导出,零后端。

    Indexed GitHub, liyupi 274 ★MITUpdated today

    git clone https://github.com/liyupi/ai-model-world.git
  19. elimination_game

    Datasets & benchmarks

    A multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other

    Stale GitHub, lechmazur 298 ★Updated 10 months ago

    git clone https://github.com/lechmazur/elimination_game.git
  20. apple-silicon-llm-bench

    Datasets & benchmarks

    Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

    Indexed GitHub, john-rocky 73 ★MITUpdated today

    git clone https://github.com/john-rocky/apple-silicon-llm-bench.git
  21. NIMStats

    Datasets & benchmarks

    📊 Automated hourly benchmarks for 20+ NVIDIA NIM models — interactive dashboard, zero infra, self-hostable. Open-source & community-driven.

    Indexed GitHub, MauroDruwel 57 ★Updated today

    git clone https://github.com/MauroDruwel/NIMStats.git
  22. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  23. VisualNews-Repository

    Datasets & benchmarks

    [EMNLP'21] Visual News: Benchmark and Challenges in News Image Captioning

    Stale GitHub, FuxiaoLiu 106 ★Updated 2 years ago

    git clone https://github.com/FuxiaoLiu/VisualNews-Repository.git
  24. claude-protein-binder-design

    Datasets & benchmarks

    Hugging Face dataset by Anthropic

    Indexed Hugging Face Hub, Anthropic 244 ♥cc-by-4.0Updated 3 months ago

    Anthropic/claude-protein-binder-design

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.