What the agent did
Tool calls, normalized commands, work-directory artifacts, skills, transcript, and final response.
01 / VERIFY THE WORK
Catch broken agent workflows before they ship.
Dynobox is the open-source test runner for SKILL.md and agent workflows. Run them across supported harnesses and check observed tool calls, commands, and work-directory artifacts with explicit assertions.
Find skipped steps, bad outputs, and behavior regressions, with captured evidence for failed assertions.
View docs →
$ npx dynobox init~/skills/dyno-from-skill
❯ npx dynobox run dyno/dyno-from-skill.dyno.mjs
■ dynobox 0.10.1
discovered 1 dyno · 1 scenario · harnesses: 2
[dyno-from-skill] generates test dyno
create dyno from skill
claude-code/sonnet/da...✓ 13 assertions
✓skill.referenced(dyno-from-skill)
✓artifact.exists(example.dyno.mjs)
✓verify.command(dynobox validate)
…
codex/gpt-5.4-mini/da...✓ 13 assertions
✓ 2 jobs passed · 26 assertions · 3m25s
02 / EVIDENCE OVER OPINION
Behavior is the contract.
Agents are non-deterministic. Your acceptance criteria shouldn't be. Define what must happen, then evaluate captured evidence on each run.
What the test requires
Explicit matchers authored in TypeScript or YAML and compiled before the harness runs.
Why it passed or failed
Evaluator-recorded evidence attached to assertion results for inspection and comparison.
03 / DEFINE THE CHECKS
Turn instructions
into checks.
A dyno translates skill requirements into deterministic assertions. Each result connects a requirement to evidence captured during the run.
Read the assertion reference →| CHECK | WHAT IT ASSERTS | OBSERVED EVIDENCE | STATUS |
|---|---|---|---|
skill.referenced | Observe a dyno authoring skill reference | dyno-from-skill / SKILL.md | PASS |
http.called | Observe the authoring reference request | config-authoring.md · status 200 | PASS |
artifact.exists | Find the generated dyno in the work directory | dyno / wordcount.dyno.mjs | PASS |
artifact.contains | Find the Dynobox SDK import | defineDyno · @dynobox/sdk | PASS |
command.notCalled | Observe no command that runs the dyno | no dynobox run observed | PASS |
verify.command | Check that validation exits successfully | dynobox validate · exit 0 | PASS |
04 / BUILD A RECORD
See when behavior changes.
Compare compatible jobs across saved runs and inspect assertion outcome and evidence changes.
Open the dashboard ↗COMPARE RUNS
1 REGRESSIONcreate dyno from skill
verify.commandGenerated dyno passes validationBASELINEPASSED | COMPARISONFAILED |
|---|---|
1 dynobox validate wordcount.dyno.mjs | 1 dynobox validate wordcount.dyno.mjs |
2− exit code 0 · valid dyno | |
| 2+ exit code 1 · invalid assertion |
05 / COMMON QUESTIONS
Before you
run.
How Dynobox works, what it measures, and where it fits in your workflow.
Read the docs →01How is this different from other agent-eval tools?
Most eval tools grade output quality by asking one model to score another model's answer. Dynobox looks at behavior instead. It records what your agent does inside the harness, including tool calls, shell commands, files, eligible HTTP requests, and its final message, then checks that evidence against assertions you wrote. Your assertions are the only criteria, so a pass means the same thing every run.
02Agent runs are non-deterministic. How can the tests be?
The assertion is the fixed part, not the run. You write down what has to be true, like "committed but never pushed" or "read the file without editing it," and Dynobox checks each run's evidence against it. When you want to know how stable the behavior is,--iterations repeats a scenario and reports how many runs passed.
03Do I need an account?
No. Dynobox runs on your machine and uses the agent harnesses you already have installed. The one exception is--save-run, which publishes a compact run summary to the dashboard and needs you to log in.
04Should I run dynos locally or in CI?
Both, at different points. Author and debug locally where you can iterate quickly, then run targeted suites in CI when skills, prompts, or agent workflows change. Every scenario invokes a real agent harness and jobs run serially, so suite duration is roughly the combined runtime of every scenario, harness, and iteration. See theGitHub Actions pattern in the docs.
05Which agents and models does it support?
Claude Code, Codex CLI, and OpenCode, each through its own executable on your PATH. Model and permission mode are configurable per harness, so you can run the same scenario across all three and see where they behave differently.
READY TO VERIFY