InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. uptrain

    Prompts

    UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

    Stale GitHub, uptrain-ai 2,366 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/uptrain-ai/uptrain.git
  2. Simreal-MLBench

    Datasets & benchmarks

    [Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

    Stale GitHub, Simreal-AI 301 ★Apache-2.0Updated 53 days ago

    git clone https://github.com/Simreal-AI/Simreal-MLBench.git
  3. ResearchClawBench

    Datasets & benchmarks

    🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

    Stale GitHub, InternScience 275 ★MITUpdated 19 days ago

    git clone https://github.com/InternScience/ResearchClawBench.git
  4. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  5. agentenv-framework

    MCP servers

    Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

    Indexed GitHub, scaleapi 441 ★Apache-2.0Updated today

    git clone https://github.com/scaleapi/agentenv-framework.git
  6. ParseBench

    Datasets & benchmarks

    ParseBench - A Document Parsing Benchmark for AI Agents

    Indexed GitHub, run-llama 640 ★Apache-2.0Updated today

    git clone https://github.com/run-llama/ParseBench.git
  7. A test runner for agentskills.io-style AI agent skills

    Stale GitHub, darkrishabh 802 ★MITUpdated 33 days ago

    git clone https://github.com/darkrishabh/agent-skills-eval.git
  8. ResolveLab

    Agents & frameworks

    Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

    Stale GitHub, Ta0p1 106 ★MITUpdated 11 days ago

    git clone https://github.com/Ta0p1/ResolveLab.git
  9. futureagi-sdk

    Prompts

    Production-grade AI evaluation, prompt management & observability SDK. Automated evaluations with sub-100ms guardrails. No human-in-the-loop required. Python + TypeScript.

    Stale GitHub, future-agi 52 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/future-agi/futureagi-sdk.git
  10. DataMate

    AI repositories

    DataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.

    Stale GitHub, ModelEngine-Group 374 ★MITUpdated 5 days ago

    git clone https://github.com/ModelEngine-Group/DataMate.git
  11. rag-experiment-accelerator

    AI repositories

    The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

    Indexed GitHub, microsoft 320 ★Custom licence (see repository)Updated today

    git clone https://github.com/microsoft/rag-experiment-accelerator.git
  12. WebCanvas

    Agents & frameworks

    All-in-one Web Agent framework for post-training. Start building with a few clicks!

    Stale GitHub, iMeanAI 281 ★MITUpdated 16 months ago

    git clone https://github.com/iMeanAI/WebCanvas.git
  13. RouterArena

    Agents & frameworks

    RouterArena: An open framework for evaluating LLM routers with standardized datasets, metrics, an automated framework, and a live leaderboard.

    Stale GitHub, RouteWorks 157 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/RouteWorks/RouterArena.git
  14. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  15. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  16. AI-Engineer-Headquarters

    AI repositories

    A collection of scientific methods, processes, algorithms, and systems to build stories & models.

    Stale GitHub, hemansnation 3,776 ★Updated 19 days ago

    git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git
  17. Language AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production.

    Stale GitHub, gil-son 129 ★GPL-3.0Updated 15 days ago

    git clone https://github.com/gil-son/language-ai-engineering-lab.git
  18. fable-method

    Skills

    The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.

    Stale GitHub, Sahir619 2,300 ★MITUpdated 44 days ago

    git clone https://github.com/Sahir619/fable-method.git
  19. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 3 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  20. SkillCorpus

    Skills

    Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.

    Stale GitHub, EverMind-AI 700 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/EverMind-AI/SkillCorpus.git
  21. Awesome-LLM-in-Social-Science

    Agents & frameworks

    Awesome papers involving LLMs in Social Science.

    Indexed GitHub, ValueByte-AI 653 ★MITUpdated 2 days ago

    git clone https://github.com/ValueByte-AI/Awesome-LLM-in-Social-Science.git
  22. 把工作流变成可研究、可评测、可发布的乔木 Agent Skill | Turn workflows into researched, tested, release-ready agent skills.

    Stale GitHub, joeseesun 389 ★MITUpdated 12 days ago

    git clone https://github.com/joeseesun/qiaomu-meta-skill.git
  23. xVerify

    Datasets & benchmarks

    xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

    Stale GitHub, IAAR-Shanghai 150 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/IAAR-Shanghai/xVerify.git
  24. ms-cookbook

    AI repositories

    魔搭紫皮书|ModelScope Cookbook:面向开发者的开源模型应用实战指南,覆盖模型选型、推理、微调、评测、RAG、Agent 与 AIGC,从跑通第一个模型到构建实际应用。

    Stale GitHub, modelscope 518 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/modelscope/ms-cookbook.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.