InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. Awesome-Datasets-Hub

    Datasets & benchmarks

    A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.

    Stale GitHub, ahammadmejbah 149 ★Updated 4 months ago

    git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git
  2. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  3. OSWorld-V2

    Datasets & benchmarks

    OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

    Indexed GitHub, xlang-ai 421 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/xlang-ai/OSWorld-V2.git
  4. LongCite

    Datasets & benchmarks

    LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

    Stale GitHub, THUDM 523 ★Apache-2.0Updated 22 months ago

    git clone https://github.com/THUDM/LongCite.git
  5. opencompass

    Datasets & benchmarks

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    Indexed GitHub, open-compass 7,596 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/open-compass/opencompass.git
  6. FastEval

    Datasets & benchmarks

    Fast & more realistic evaluation of chat language models. Includes leaderboard.

    Stale GitHub, FastEval 187 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/FastEval/FastEval.git
  7. TrustLLM

    Datasets & benchmarks

    [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

    Stale GitHub, HowieHwong 632 ★MITUpdated 43 days ago

    git clone https://github.com/HowieHwong/TrustLLM.git
  8. Ditto

    Datasets & benchmarks

    A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment".

    Stale GitHub, OFA-Sys 214 ★MITUpdated 2 years ago

    git clone https://github.com/OFA-Sys/Ditto.git
  9. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  10. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  11. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  12. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  13. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  14. NextTransformer

    Datasets & benchmarks

    :sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

    Indexed GitHub, Interactive-Active-AI 435 ★MITUpdated today

    git clone https://github.com/Interactive-Active-AI/NextTransformer.git
  15. active_genie

    Datasets & benchmarks

    The Lodash for GenAI: Real Value + Consistent + Model-Agnostic

    Stale GitHub, Roriz 168 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/Roriz/active_genie.git
  16. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  17. ollama-webui-intel

    Datasets & benchmarks

    Ollama with intel (i)GPU acceleration in docker and benchmark

    Indexed GitHub, NikolasEnt 53 ★Updated 2 days ago

    git clone https://github.com/NikolasEnt/ollama-webui-intel.git
  18. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  19. Awesome-AI4AI

    Datasets & benchmarks

    Updated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.

    Stale GitHub, KaiWU5 218 ★MITUpdated 8 days ago

    git clone https://github.com/KaiWU5/Awesome-AI4AI.git
  20. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  21. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  22. turnkeyml

    Datasets & benchmarks

    No-code CLI designed for accelerating ONNX workflows

    Stale GitHub, onnx 247 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/onnx/turnkeyml.git
  23. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  24. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.