Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
Multi-User-LLM-Agent
Datasets & benchmarksOfficial code for the paper: "Multi-User Large Language Model Agents"
git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git -
UltraX-Preview
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraX-Preview -
PyHealth
Datasets & benchmarksA deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
git clone https://github.com/sunlabuiuc/PyHealth.git -
NoLiMa
Datasets & benchmarksOfficial repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"
git clone https://github.com/adobe-research/NoLiMa.git -
elimination_game
Datasets & benchmarksA multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other
git clone https://github.com/lechmazur/elimination_game.git -
DeepPlanning
Datasets & benchmarksHugging Face dataset by Qwen
Qwen/DeepPlanning -
ollama-webui-intel
Datasets & benchmarksOllama with intel (i)GPU acceleration in docker and benchmark
git clone https://github.com/NikolasEnt/ollama-webui-intel.git -
swt-bench
Datasets & benchmarks[NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation
git clone https://github.com/logic-star-ai/swt-bench.git -
WritingBench
Datasets & benchmarksWritingBench: A Comprehensive Benchmark for Generative Writing
git clone https://github.com/X-PLUG/WritingBench.git -
ping_pong_bench
Datasets & benchmarksA benchmark for role-playing language models
git clone https://github.com/IlyaGusev/ping_pong_bench.git -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
Awesome-Scientific-LLM-Benchmarks
Datasets & benchmarksA curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.
git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git -
MMLU-CF
Datasets & benchmarksA Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]
git clone https://github.com/microsoft/MMLU-CF.git -
apple-silicon-llm-bench
Datasets & benchmarksReproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.
git clone https://github.com/john-rocky/apple-silicon-llm-bench.git -
UltraData-SFT-2605
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraData-SFT-2605 -
Finance-Instruct-500k
Datasets & benchmarksHugging Face dataset by Josephgflowers
Josephgflowers/Finance-Instruct-500k -
guidelines
Datasets & benchmarksHugging Face dataset by epfl-llm
epfl-llm/guidelines -
Ultra-FineWeb-L3
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/Ultra-FineWeb-L3 -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
ollama-benchmark
Datasets & benchmarksOllama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
git clone https://github.com/LarHope/ollama-benchmark.git -
OSWorld-V2
Datasets & benchmarksOSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
git clone https://github.com/xlang-ai/OSWorld-V2.git -
nagi-bench
Datasets & benchmarksOne-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.
git clone https://github.com/nagi-studio/nagi-bench.git -
llm-value-rankings
Datasets & benchmarksDaily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜
git clone https://github.com/yyh-001/llm-value-rankings.git -
UltraData-RL-2609
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraData-RL-2609
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.