InnerVault

Library

85,766 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

1,332 results Clear search and filters
  1. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 13 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  2. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  3. Act as a Lead Data Analyst. You are equipped with a Data Engineering background, enabling you to understand both data collection and analysis processes. When a data problem or dataset is presented, your…

    Indexed prompts.chat (CC0 prompt collection), ozzy2438 CC0-1.0

  4. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  5. ChemLLMBench

    Datasets & benchmarks

    Official Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)

    Stale GitHub, ChemFoundationModels 176 ★Updated 2 years ago

    git clone https://github.com/ChemFoundationModels/ChemLLMBench.git
  6. teor

    Datasets & benchmarks

    Benchmark aberto de LLMs para o processo legislativo brasileiro

    Stale GitHub, como-votam 30 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/como-votam/teor.git
  7. OSWorld-V2

    Datasets & benchmarks

    OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

    Indexed GitHub, xlang-ai 421 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/xlang-ai/OSWorld-V2.git
  8. Data Scientist

    Prompts

    I want you to act as a data scientist. Imagine you're working on a challenging project for a cutting-edge tech company. You've been tasked with extracting valuable insights from a large dataset related to user behavior…

    Indexed prompts.chat (CC0 prompt collection), shvuuuu CC0-1.0

  9. LightCompress

    Datasets & benchmarks

    [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

    Stale GitHub, ModelTC 751 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/ModelTC/LightCompress.git
  10. Ditto

    Datasets & benchmarks

    A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment".

    Stale GitHub, OFA-Sys 214 ★MITUpdated 2 years ago

    git clone https://github.com/OFA-Sys/Ditto.git
  11. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  12. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  13. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  14. Query our database of federal and state laws and incentives for alternative fuels and vehicles, air quality, fuel efficiency, and other transportation-related topics. This dataset powers the Federal and State Laws and Incentives on the Alternative Fuels Data Center.

    Indexed APIs.guru OpenAPI Directory, nrel.gov Updated 3 years ago

  15. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  16. Multi-User-LLM-Agent

    Datasets & benchmarks

    Official code for the paper: "Multi-User Large Language Model Agents"

    Stale GitHub, Kordi-AI 39 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git
  17. Act as a Robotics and AI Research Assistant. You are an expert in utilizing the StanfordVL/BEHAVIOR-1K dataset for advancing research in robotics and artificial intelligence. Your task is to guide researchers in…

    Indexed prompts.chat (CC0 prompt collection), liangyue636@gmail.com CC0-1.0

  18. sacp

    Datasets & benchmarks

    State-Aware Collaboration Protocol: auditable work receipts for AI agents.

    Stale GitHub, aDragon0707 107 ★MITUpdated 33 days ago

    git clone https://github.com/aDragon0707/sacp.git
  19. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 5 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  20. PyHealth

    Datasets & benchmarks

    A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.

    Indexed GitHub, sunlabuiuc 1,699 ★MITUpdated yesterday

    git clone https://github.com/sunlabuiuc/PyHealth.git
  21. Glyd

    Datasets & benchmarks

    Lossless AI compression: 33% less GPU memory, bit for bit.

    Stale GitHub, surya-koritala 321 ★Custom licence (see repository)Updated 15 days ago

    git clone https://github.com/surya-koritala/Glyd.git
  22. NextTransformer

    Datasets & benchmarks

    :sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

    Indexed GitHub, Interactive-Active-AI 435 ★MITUpdated yesterday

    git clone https://github.com/Interactive-Active-AI/NextTransformer.git
  23. ds4-on-spark

    Datasets & benchmarks

    Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark, and full continuous batch support

    Stale GitHub, Entrpi 409 ★MITUpdated 2 months ago

    git clone https://github.com/Entrpi/ds4-on-spark.git
  24. ResearchClawBench

    Datasets & benchmarks

    🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

    Stale GitHub, InternScience 275 ★MITUpdated 20 days ago

    git clone https://github.com/InternScience/ResearchClawBench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.