OpenART
OpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environments. It stress-tests agent runtimes against multi-step state poisoning, privilege escalation, and tool-use vulnerabilities across 10,000+ benchmark scenarios.
Open the source (opens github.com) https://github.com/ai45lab/openart
What it is
Skills from AI45Lab.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/AI45Lab/OpenART.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
skill-up
SkillsEvaluate Agent Skills and environments. Evolve Skills with evidence.
git clone https://github.com/alibaba/skill-up.git -
SkillEvaluator
SkillsMulti-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
git clone https://github.com/NVIDIA/SkillEvaluator.git -
kasetto
Skills📼 Declarative AI agent environment manager, written in Rust
git clone https://github.com/pivoshenko/kasetto.git -
coder_eval
SkillsPlaywright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
git clone https://github.com/UiPath/coder_eval.git -
fable-method
SkillsThe Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.
git clone https://github.com/Sahir619/fable-method.git -
agent-eval-loop
SkillsBuild evals from traces, code, and coding sessions, then measure and improve AI agent behavior, including your coding agents
git clone https://github.com/relai-ai/agent-eval-loop.git