InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. langfuse

    Prompts

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    Indexed GitHub, langfuse 37.5k ★Custom licence (see repository)Updated today

    git clone https://github.com/langfuse/langfuse.git
  2. mlflow

    Prompts

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    Indexed GitHub, mlflow 28.8k ★Apache-2.0Updated today

    git clone https://github.com/mlflow/mlflow.git
  3. prompty

    AI repositories

    Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.

    Stale GitHub, microsoft 1,288 ★MITUpdated 7 days ago

    git clone https://github.com/microsoft/prompty.git
  4. EvalForge

    AI repositories

    Evaluator-neutral AI evaluation evidence, baseline regression gates, and JSON, JUnit, and SARIF reports for CI.

    Indexed GitHub, jsdhwfmax 194 ★MITUpdated today

    git clone https://github.com/jsdhwfmax/EvalForge.git
  5. deepresearch-agent

    Agents & frameworks

    一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。

    Stale GitHub, qiqihezh 165 ★Updated 6 months ago

    git clone https://github.com/qiqihezh/deepresearch-agent.git
  6. palico-ai

    Agents & frameworks

    Build, Improve Performance, and Productionize your AI Application

    Stale GitHub, palico-ai 343 ★MITUpdated 24 months ago

    git clone https://github.com/palico-ai/palico-ai.git
  7. coding.sh

    MCP servers

    Open-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.

    Indexed GitHub, jackulau 30 ★MITUpdated today

    git clone https://github.com/jackulau/coding.sh.git
  8. LLM Eval Harness

    MCP servers

    Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.

    Stale Official MCP Registry, io.github.razilzul Updated 29 days ago 2 notes

    ghcr.io/razilzul/llm-eval-harness:1.0.0
  9. generative-ai

    Agents & frameworks

    Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

    Stale GitHub, genieincodebottle 2,669 ★MITUpdated 7 days ago

    git clone https://github.com/genieincodebottle/generative-ai.git
  10. AI-Infra-Guard

    AI repositories

    A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

    Stale GitHub, Tencent 7,471 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Tencent/AI-Infra-Guard.git
  11. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  12. ClawBench

    Datasets & benchmarks

    Open-source benchmark for browser AI agents on daily tasks.

    Stale GitHub, TIGER-AI-Lab 1,229 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/TIGER-AI-Lab/ClawBench.git
  13. Create, test, and optimize OpenCode skills. Supports OpenCode V2 and V1 ≥ 1.18.29. Adapted from Anthropic's skill-creator.

    Stale GitHub, antongulin 174 ★Apache-2.0Updated 29 days ago

    git clone https://github.com/antongulin/opencode-skill-creator.git
  14. EnterpriseRAG-Bench

    AI repositories

    Dataset and benchmark for RAG on company internal documents.

    Stale GitHub, onyx-dot-app 592 ★MITUpdated 2 months ago

    git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git
  15. nasde-toolkit

    Skills

    CLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys.

    Stale GitHub, NoesisVision 14 ★MITUpdated 32 days ago

    git clone https://github.com/NoesisVision/nasde-toolkit.git
  16. future-agi

    Agents & frameworks

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    Indexed GitHub, future-agi 2,267 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/future-agi/future-agi.git
  17. chinese-llm-benchmark

    Agents & frameworks

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    Indexed GitHub, jeinlee1991 6,518 ★Updated yesterday

    git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
  18. overtime

    Agents & frameworks

    Give a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.

    Stale GitHub, ukanwat 399 ★MITUpdated 30 days ago

    git clone https://github.com/ukanwat/overtime.git
  19. Tracely-ai

    AI repositories

    Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.

    Stale GitHub, Jwuthri 1,311 ★MITUpdated 8 days ago

    git clone https://github.com/Jwuthri/Tracely-ai.git
  20. humanities_data_benchmark

    Datasets & benchmarks

    LLM Benchmark Suite for Humanities Data

    Stale GitHub, RISE-UNIBAS 32 ★GPL-3.0Updated 12 days ago

    git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git
  21. tuui

    MCP servers

    A desktop MCP client designed as a tool unitary utility integration, accelerating AI adoption through the Model Context Protocol (MCP) and enabling cross-vendor LLM API orchestration.

    Stale GitHub, AI-QL 1,157 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/AI-QL/tuui.git
  22. labelhub-platform

    MCP servers

    面向 LLM 训练数据生产的可信工作台:AI 提供证据、人类裁决,append-only 账本、可复现导出、评测关卡与 MCP Server。

    Stale GitHub, Iceberg86-gb 48 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/Iceberg86-gb/labelhub-platform.git
  23. cap-evolve

    MCP servers

    Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

    Indexed GitHub, skillberry-ai 70 ★Apache-2.0Updated today

    git clone https://github.com/skillberry-ai/cap-evolve.git
  24. evaluation-guidebook

    Docs & tutorials

    Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

    Stale GitHub, huggingface 2,155 ★Custom licence (see repository)Updated 11 months ago

    git clone https://github.com/huggingface/evaluation-guidebook.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.