InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

99 results with any of your words Clear search and filters
  1. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  2. LongPage

    Datasets & benchmarks

    Hugging Face dataset by Pageshift-Entertainment

    Stale Hugging Face Hub, Pageshift-Entertainment 161 ♥cc-by-4.0Updated 10 months ago

    Pageshift-Entertainment/LongPage
  3. PyHealth

    Datasets & benchmarks

    A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.

    Indexed GitHub, sunlabuiuc 1,699 ★MITUpdated today

    git clone https://github.com/sunlabuiuc/PyHealth.git
  4. geobench

    Datasets & benchmarks

    GeoGuessr benchmark for language models

    Indexed GitHub, ccmdi 65 ★MITUpdated 3 days ago

    git clone https://github.com/ccmdi/geobench.git
  5. OSCAR-2201

    Datasets & benchmarks

    Hugging Face dataset by oscar-corpus

    Stale Hugging Face Hub, oscar-corpus 134 ♥cc0-1.0Updated 15 months ago 1 note

    oscar-corpus/OSCAR-2201
  6. debate

    Datasets & benchmarks

    Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.

    Stale GitHub, lechmazur 38 ★Updated 21 days ago

    git clone https://github.com/lechmazur/debate.git
  7. ai-hardware-fit

    Datasets & benchmarks

    Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

    Indexed GitHub, jaeseok614 44 ★MITUpdated today

    git clone https://github.com/jaeseok614/ai-hardware-fit.git
  8. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  9. Ditto

    Datasets & benchmarks

    A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment".

    Stale GitHub, OFA-Sys 214 ★MITUpdated 2 years ago

    git clone https://github.com/OFA-Sys/Ditto.git
  10. humanities_data_benchmark

    Datasets & benchmarks

    LLM Benchmark Suite for Humanities Data

    Stale GitHub, RISE-UNIBAS 32 ★GPL-3.0Updated 12 days ago

    git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git
  11. Awesome-AI-Scientist

    Datasets & benchmarks

    🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.

    Indexed GitHub, Omni-Scientist 105 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git
  12. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  13. Awesome-Scientific-LLM-Benchmarks

    Datasets & benchmarks

    A curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.

    Stale GitHub, subinium 42 ★MITUpdated 14 days ago

    git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git
  14. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  15. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  16. Multi-User-LLM-Agent

    Datasets & benchmarks

    Official code for the paper: "Multi-User Large Language Model Agents"

    Stale GitHub, Kordi-AI 39 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git
  17. NIMStats

    Datasets & benchmarks

    📊 Automated hourly benchmarks for 20+ NVIDIA NIM models — interactive dashboard, zero infra, self-hostable. Open-source & community-driven.

    Indexed GitHub, MauroDruwel 57 ★Updated today

    git clone https://github.com/MauroDruwel/NIMStats.git
  18. Awesome-Datasets-Hub

    Datasets & benchmarks

    A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.

    Stale GitHub, ahammadmejbah 149 ★Updated 4 months ago

    git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git
  19. apple-silicon-llm-bench

    Datasets & benchmarks

    Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

    Indexed GitHub, john-rocky 73 ★MITUpdated today

    git clone https://github.com/john-rocky/apple-silicon-llm-bench.git
  20. llm-price-compass

    Datasets & benchmarks

    This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient LLM GPU selections and cost-effective AI models. LLM provider price comparison, gpu benchmarks to price per token calculation, gpu benchmark table

    Stale GitHub, arc53 221 ★MITUpdated 23 months ago

    git clone https://github.com/arc53/llm-price-compass.git
  21. deepmark

    Datasets & benchmarks

    Deepmark AI enables a unique testing environment for language models (LLM) assessment on task-specific metrics and on your own data so your GenAI-powered solution has predictable and reliable performance.

    Stale GitHub, IngestAI 104 ★AGPL-3.0Updated 2 years ago

    git clone https://github.com/IngestAI/deepmark.git
  22. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  23. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  24. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.