InnerVault

TheAgentCompany

Stale GitHub, TheAgentCompany

An agent benchmark with tasks in a simulated software company.

Open the source (opens github.com) https://github.com/theagentcompany/theagentcompany

What it is

Datasets & benchmarks from TheAgentCompany.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/TheAgentCompany/TheAgentCompany.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. tau2-bench

    Datasets & benchmarks

    τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

    Indexed GitHub, sierra-research 2,442 ★MITUpdated 6 days ago

    git clone https://github.com/sierra-research/tau2-bench.git
  2. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated today

    git clone https://github.com/grafana/o11y-bench.git
  3. GDPevo

    Datasets & benchmarks

    A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

    Stale GitHub, Prism-Shadow 119 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/Prism-Shadow/GDPevo.git
  4. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  5. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  6. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git