InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

1,332 results Clear search and filters
  1. OneMillion-Bench

    Datasets & benchmarks

    [NeurIPS 2026] $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Stale GitHub, humanlaya 55 ★Apache-2.0Updated 28 days ago

    git clone https://github.com/humanlaya/OneMillion-Bench.git
  2. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 4 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  3. aa-agentperf-local

    Datasets & benchmarks

    Benchmark local LLM serving by replaying real agent trajectories

    Stale GitHub, ArtificialAnalysis 105 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git
  4. PhysGym

    Datasets & benchmarks

    A benchmark suite for evaluating LLM-based interactive scientific reasoning.

    Indexed GitHub, principia-ai 103 ★MITUpdated 4 days ago

    git clone https://github.com/principia-ai/PhysGym.git
  5. Run up to 25 Apify Actors in order in one call, passing each step's dataset to the next.

    Stale Official MCP Registry, io.github.Nero-Engine Updated 2 months ago 1 note

  6. Extract text, tables and metadata from every PDF linked in a dataset, CSV or Google Sheet.

    Stale Official MCP Registry, io.github.Nero-Engine Updated 2 months ago 1 note

  7. ollama-benchmark

    Datasets & benchmarks

    Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.

    Stale GitHub, LarHope 174 ★MITUpdated 7 months ago

    git clone https://github.com/LarHope/ollama-benchmark.git
  8. GDPevo

    Datasets & benchmarks

    A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

    Stale GitHub, Prism-Shadow 119 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/Prism-Shadow/GDPevo.git
  9. BusterX

    Datasets & benchmarks

    [NeurIPS '26] BusterX & [EACL '27 Findings] BusterX++

    Indexed GitHub, l8cv 52 ★BSD-3-ClauseUpdated 4 days ago

    git clone https://github.com/l8cv/BusterX.git
  10. AgentLab

    Datasets & benchmarks

    AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.

    Stale GitHub, ServiceNow 653 ★Custom licence (see repository)Updated 42 days ago

    git clone https://github.com/ServiceNow/AgentLab.git
  11. gamebot

    Datasets & benchmarks

    [ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games

    Stale GitHub, Visual-AI 36 ★Updated 11 days ago

    git clone https://github.com/Visual-AI/gamebot.git
  12. OCR every image and scanned PDF linked in an Apify dataset, CSV, Excel or Google Sheet, keeping...

    Stale Official MCP Registry, io.github.Nero-Engine Updated 48 days ago 1 note

  13. doc2harness

    Datasets & benchmarks

    Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

    Stale GitHub, YuyaoGe 116 ★Custom licence (see repository)Updated 21 days ago

    git clone https://github.com/YuyaoGe/doc2harness.git
  14. One-Eval

    Datasets & benchmarks

    Automated system for LLM evaluation via agents. Doc as below:

    Stale GitHub, OpenDCAI 165 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/OpenDCAI/One-Eval.git
  15. llmspeed

    Datasets & benchmarks

    Feel how fast a local LLM runs before you buy the hardware

    Stale GitHub, nicorobinreal 152 ★MITUpdated 28 days ago

    git clone https://github.com/nicorobinreal/llmspeed.git
  16. aurora

    Datasets & benchmarks

    [ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

    Stale GitHub, wenhaochai 149 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/wenhaochai/aurora.git
  17. civ6-mcp

    Datasets & benchmarks

    An MCP server that lets LLM agents play Civilization VI.

    Stale GitHub, lmwilki 218 ★MITUpdated 35 days ago

    git clone https://github.com/lmwilki/civ6-mcp.git
  18. swe-sweep

    Datasets & benchmarks

    How many bugs can LMs find & fix in large codebases?

    Indexed GitHub, facebookresearch 240 ★Custom licence (see repository)Updated 5 days ago

    git clone https://github.com/facebookresearch/swe-sweep.git
  19. Send one HTTP request, or one per row of a dataset, CSV or Google Sheet, and get the response back

    Stale Official MCP Registry, io.github.Nero-Engine Updated 46 days ago 1 note

  20. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 6 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  21. polymarket-paper-trader

    Datasets & benchmarks

    Paper trading simulator for Polymarket

    Stale GitHub, agent-next 495 ★MITUpdated 16 days ago

    git clone https://github.com/agent-next/polymarket-paper-trader.git
  22. Static-to-Dynamic-LLMEval

    Datasets & benchmarks

    The official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"

    Stale GitHub, SeekingDream 493 ★Updated 50 days ago

    git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git
  23. ollama-webui-intel

    Datasets & benchmarks

    Ollama with intel (i)GPU acceleration in docker and benchmark

    Indexed GitHub, NikolasEnt 53 ★Updated 3 days ago

    git clone https://github.com/NikolasEnt/ollama-webui-intel.git
  24. lora-speedrun

    Datasets & benchmarks

    Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

    Stale GitHub, Saivineeth147 149 ★MITUpdated 2 months ago

    git clone https://github.com/Saivineeth147/lora-speedrun.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.