Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
debate
Datasets & benchmarksAdversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.
git clone https://github.com/lechmazur/debate.git -
Berkeley-Function-Calling-Leaderboard
Datasets & benchmarksHugging Face dataset by gorilla-llm
gorilla-llm/Berkeley-Function-Calling-Leaderboard -
arena-ai-leaderboards
Datasets & benchmarks📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON with historical tracking.
git clone https://github.com/oolong-tea-2026/arena-ai-leaderboards.git -
Awesome-LLM-Eval
Datasets & benchmarksAwesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.
git clone https://github.com/onejune2018/Awesome-LLM-Eval.git -
lora-speedrun
Datasets & benchmarksSpeedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.
git clone https://github.com/Saivineeth147/lora-speedrun.git -
ai-model-world
Datasets & benchmarksAI 大模型世界,把 556 个大模型拟人化成像素小人的可视化站点。进来就能看到此刻谁最聪明、谁最会写代码、谁最便宜、谁刚发布,往下是国内与国外分区的厂商广场、完整的发布时间线和多维排行榜。搜索认模型名、厂商和能力,输入「多模态」会直接列出全部多模态模型。数据取自 Epoch AI、models.dev、LiveBench 与 Hugging Face,每小时自动同步,所有文案由真实数据生成,不调用任何 LLM。Next.js 静态导出,零后端。
git clone https://github.com/liyupi/ai-model-world.git -
ClawProBench
Datasets & benchmarksClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.
git clone https://github.com/suyoumo/ClawProBench.git -
FastEval
Datasets & benchmarksFast & more realistic evaluation of chat language models. Includes leaderboard.
git clone https://github.com/FastEval/FastEval.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
polymarket-paper-trader
Datasets & benchmarksPaper trading simulator for Polymarket
git clone https://github.com/agent-next/polymarket-paper-trader.git -
LMeterX
Datasets & benchmarksA general-purpose API load testing platform that supports LLM services、A2A/MCP services and business HTTP interfaces, enabling one-click performance testing, result comparison, and AI-powered intelligent analysis and summarization. 一站式通用 API 压测平台,支持大模型推理业/务 HTTP 接口、A2A/MCP服务,一键完成性能测试、结果对比与 AI 智能分析总结
git clone https://github.com/MigoXLab/LMeterX.git -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
explicit-edit-benchmark
Datasets & benchmarksBenchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.
git clone https://github.com/alexshpunt/explicit-edit-benchmark.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
aurora
Datasets & benchmarks[ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
git clone https://github.com/wenhaochai/aurora.git -
mac-ml-speed-test
Datasets & benchmarksA few quick scripts focused on testing TensorFlow/PyTorch/Llama 2 on macOS.
git clone https://github.com/mrdbourke/mac-ml-speed-test.git -
deepmark
Datasets & benchmarksDeepmark AI enables a unique testing environment for language models (LLM) assessment on task-specific metrics and on your own data so your GenAI-powered solution has predictable and reliable performance.
git clone https://github.com/IngestAI/deepmark.git -
internship-warehouse
Datasets & benchmarksHugging Face dataset by FlyRank
FlyRank/internship-warehouse -
opencompass
Datasets & benchmarksOpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
git clone https://github.com/open-compass/opencompass.git -
llm-colosseum
Datasets & benchmarksBenchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
git clone https://github.com/OpenGenerativeAI/llm-colosseum.git -
LLM-eval-survey
Datasets & benchmarksThe official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
git clone https://github.com/MLGroupJLU/LLM-eval-survey.git -
Awesome-LLM-Long-Context-Modeling
Datasets & benchmarks📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git -
prompts.chat
Datasets & benchmarksHugging Face dataset by fka
fka/prompts.chat -
dolma
Datasets & benchmarksHugging Face dataset by allenai
allenai/dolma
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.