InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. Ditto

    Datasets & benchmarks

    A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment".

    Stale GitHub, OFA-Sys 214 ★MITUpdated 2 years ago

    git clone https://github.com/OFA-Sys/Ditto.git
  2. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  3. NextTransformer

    Datasets & benchmarks

    :sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

    Indexed GitHub, Interactive-Active-AI 435 ★MITUpdated today

    git clone https://github.com/Interactive-Active-AI/NextTransformer.git
  4. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  5. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  6. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  7. active_genie

    Datasets & benchmarks

    The Lodash for GenAI: Real Value + Consistent + Model-Agnostic

    Stale GitHub, Roriz 168 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/Roriz/active_genie.git
  8. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  9. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  10. ollama-webui-intel

    Datasets & benchmarks

    Ollama with intel (i)GPU acceleration in docker and benchmark

    Indexed GitHub, NikolasEnt 53 ★Updated 2 days ago

    git clone https://github.com/NikolasEnt/ollama-webui-intel.git
  11. evalscope

    AI repositories

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    Indexed GitHub, modelscope 3,697 ★Apache-2.0Updated today

    git clone https://github.com/modelscope/evalscope.git
  12. Awesome-AI4AI

    Datasets & benchmarks

    Updated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.

    Stale GitHub, KaiWU5 218 ★MITUpdated 8 days ago

    git clone https://github.com/KaiWU5/Awesome-AI4AI.git
  13. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 3 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  14. Awesome-LLM-Eval

    Datasets & benchmarks

    Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.

    Stale GitHub, onejune2018 659 ★MITUpdated 11 months ago

    git clone https://github.com/onejune2018/Awesome-LLM-Eval.git
  15. WebCanvas

    Agents & frameworks

    All-in-one Web Agent framework for post-training. Start building with a few clicks!

    Stale GitHub, iMeanAI 281 ★MITUpdated 16 months ago

    git clone https://github.com/iMeanAI/WebCanvas.git
  16. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  17. turnkeyml

    Datasets & benchmarks

    No-code CLI designed for accelerating ONNX workflows

    Stale GitHub, onnx 247 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/onnx/turnkeyml.git
  18. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  19. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  20. OpenRCA

    Agents & frameworks

    [ICLR'25] OpenRCA: Can Large Language Models Locate the Root Cause of Software Failures?

    Stale GitHub, microsoft 451 ★MITUpdated 12 days ago

    git clone https://github.com/microsoft/OpenRCA.git
  21. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  22. UHGEval

    Datasets & benchmarks

    [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

    Stale GitHub, IAAR-Shanghai 183 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/IAAR-Shanghai/UHGEval.git
  23. mas-lab

    Agents & frameworks

    Specification-driven multi-agent systems with OpenTelemetry observability, agent traces, benchmarking, MCP, and A2A.

    Indexed GitHub, ioc-project 65 ★Apache-2.0Updated yesterday

    git clone https://github.com/ioc-project/mas-lab.git
  24. teor

    Datasets & benchmarks

    Benchmark aberto de LLMs para o processo legislativo brasileiro

    Stale GitHub, como-votam 30 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/como-votam/teor.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.