InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. LLM-eval-survey

    Datasets & benchmarks

    The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

    Stale GitHub, MLGroupJLU 1,615 ★Updated 7 days ago

    git clone https://github.com/MLGroupJLU/LLM-eval-survey.git
  2. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  3. ArtifactsBenchmark

    Datasets & benchmarks

    Stale GitHub, Tencent-Hunyuan 280 ★Custom licence (see repository)Updated 10 months ago

    git clone https://github.com/Tencent-Hunyuan/ArtifactsBenchmark.git
  4. Static-to-Dynamic-LLMEval

    Datasets & benchmarks

    The official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"

    Stale GitHub, SeekingDream 493 ★Updated 49 days ago

    git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git
  5. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  6. ChemLLMBench

    Datasets & benchmarks

    Official Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)

    Stale GitHub, ChemFoundationModels 176 ★Updated 2 years ago

    git clone https://github.com/ChemFoundationModels/ChemLLMBench.git
  7. opencompass

    Datasets & benchmarks

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    Indexed GitHub, open-compass 7,596 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/open-compass/opencompass.git
  8. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  9. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 3 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  10. LawBench

    Datasets & benchmarks

    Benchmarking Legal Knowledge of Large Language Models

    Stale GitHub, open-compass 455 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/open-compass/LawBench.git
  11. aurora

    Datasets & benchmarks

    [ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

    Stale GitHub, wenhaochai 149 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/wenhaochai/aurora.git
  12. xFinder

    Datasets & benchmarks

    [ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

    Stale GitHub, IAAR-Shanghai 178 ★Custom licence (see repository)Updated 12 months ago

    git clone https://github.com/IAAR-Shanghai/xFinder.git
  13. JBB-Behaviors

    Datasets & benchmarks

    Hugging Face dataset by JailbreakBench

    Stale Hugging Face Hub, JailbreakBench 139 ♥mitUpdated 2 years ago

    JailbreakBench/JBB-Behaviors
  14. LightCompress

    Datasets & benchmarks

    [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

    Stale GitHub, ModelTC 751 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/ModelTC/LightCompress.git
  15. AmericanStories

    Datasets & benchmarks

    Hugging Face dataset by dell-research-harvard

    Stale Hugging Face Hub, dell-research-harvard 181 ♥cc-by-4.0Updated 20 months ago

    dell-research-harvard/AmericanStories
  16. UHGEval

    Datasets & benchmarks

    [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

    Stale GitHub, IAAR-Shanghai 183 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/IAAR-Shanghai/UHGEval.git
  17. Awesome-LLM-Eval

    Datasets & benchmarks

    Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.

    Stale GitHub, onejune2018 659 ★MITUpdated 11 months ago

    git clone https://github.com/onejune2018/Awesome-LLM-Eval.git
  18. HallusionBench

    Datasets & benchmarks

    [CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

    Stale GitHub, tianyi-lab 344 ★BSD-3-ClauseUpdated 13 months ago

    git clone https://github.com/tianyi-lab/HallusionBench.git
  19. ReForm-Eval

    Datasets & benchmarks

    An benchmark for evaluating the capabilities of large vision-language models (LVLMs)

    Stale GitHub, FudanDISC 145 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/FudanDISC/ReForm-Eval.git
  20. TrustLLM

    Datasets & benchmarks

    [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

    Stale GitHub, HowieHwong 632 ★MITUpdated 43 days ago

    git clone https://github.com/HowieHwong/TrustLLM.git
  21. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  22. meta-agents-research-environments

    Datasets & benchmarks

    Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.

    Stale GitHub, facebookresearch 567 ★MITUpdated 47 days ago

    git clone https://github.com/facebookresearch/meta-agents-research-environments.git
  23. Awesome-Scientific-LLM-Benchmarks

    Datasets & benchmarks

    A curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.

    Stale GitHub, subinium 42 ★MITUpdated 14 days ago

    git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git
  24. ai-model-world

    Datasets & benchmarks

    AI 大模型世界,把 556 个大模型拟人化成像素小人的可视化站点。进来就能看到此刻谁最聪明、谁最会写代码、谁最便宜、谁刚发布,往下是国内与国外分区的厂商广场、完整的发布时间线和多维排行榜。搜索认模型名、厂商和能力,输入「多模态」会直接列出全部多模态模型。数据取自 Epoch AI、models.dev、LiveBench 与 Hugging Face,每小时自动同步,所有文案由真实数据生成,不调用任何 LLM。Next.js 静态导出,零后端。

    Indexed GitHub, liyupi 274 ★MITUpdated today

    git clone https://github.com/liyupi/ai-model-world.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.