verify.commandGenerated dyno passes validationBASELINEPASSED | COMPARISONFAILED |
|---|---|
1 dynobox validate wordcount.dyno.mjs | 1 dynobox validate wordcount.dyno.mjs |
2− exit code 0 · valid dyno | |
| 2+ exit code 1 · invalid assertion |
Run real workflows. Check the evidence. Catch bugs before they ship.
import {
artifact, command, defineDyno, sequence, skill,
} from '@dynobox/sdk';
export default defineDyno({
name: '[commit-skill] no writes',
target: 'commit-skill',
scenarios: [{
name: 'no push',
harnesses: ['claude-code', 'codex', 'opencode'],
prompt: 'Use the commit skill to commit the README.md '
+ 'change. Do not push or amend.',
setup: [
'git init',
'git config user.email tester@dynobox.com',
'git config user.name "Dynobox Test"',
'git add . && git commit -m "chore: initial commit"',
'printf "\nCommit skill smoke change.\n" >> README.md',
],
assertions: [
sequence.inOrder([
command.called('git', {args: ['status']}),
command.called('git', {args: ['diff']}),
command.called('git', {args: ['commit']}),
]),
skill.referenced('commit'),
artifact.exists('.agents/skills/commit/SKILL.md'),
command.notCalled('git', {args: ['push']}),
command.notCalled('git', {args: ['commit', '--amend']}),
],
}],
});name: "[commit-skill] no writes"
target: commit-skill
scenarios:
- name: no push
harnesses: [claude-code, codex, opencode]
prompt: >-
Use the commit skill to commit the README.md change.
Do not push or amend.
setup:
- git init
- git config user.email tester@dynobox.com
- git config user.name "Dynobox Test"
- 'git add . && git commit -m "chore: initial commit"'
- 'printf "\nCommit skill smoke change.\n" >> README.md'
assertions:
- type: sequence.inOrder
steps:
- type: command.called
executable: git
command: {args: [status]}
- type: command.called
executable: git
command: {args: [diff]}
- type: command.called
executable: git
command: {args: [commit]}
- type: skill.referenced
skill: commit
- type: artifact.exists
path: .agents/skills/commit/SKILL.md
- type: command.notCalled
executable: git
command: {args: [push]}
- type: command.notCalled
executable: git
command: {args: [commit, --amend]}Supported adapters: Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI. Harnesses must already be installed and authenticated.
| TOOLS | tool.called · tool.notCalled |
|---|---|
| COMMANDS | command.called · command.notCalled |
| FILES | artifact.exists · artifact.notExistsartifact.contains · artifact.unchanged |
| SKILLS | skill.referenced |
| NETWORK | http.called · http.notCalled |
| RESPONSE | transcript.contains · finalMessage.contains |
| LOGIC | sequence.inOrder · anyOf |
| VERIFY | verify.command |
Assertions are evaluated against evidence recorded during the run. A model does not grade the result.
defineDyno({
// ...
cliMocks: {
vitest: {
responses: [
{exitCode: 1, stderr: 'one test failed'},
{exitCode: 0, stdout: 'all tests passed'},
],
onExhausted: 'repeat-last',
},
},
endpoints: {
package: http.endpoint({
method: 'GET',
url: 'https://registry.npmjs.org/prettier',
}),
},
// ...
});CLI mocks replace bare executable calls on PATH with static, sequential, or handler responses.
HTTP capture observes child-process traffic that honors the per-job proxy and generated CA. It does not cover every harness-native web tool.
A single pass can be luck. Repeat each scenario and harness pair to measure consistency; Dynobox reports the pass rate and keeps evidence from every failure.
verify.commandGenerated dyno passes validationBASELINEPASSED | COMPARISONFAILED |
|---|---|
1 dynobox validate wordcount.dyno.mjs | 1 dynobox validate wordcount.dyno.mjs |
2− exit code 0 · valid dyno | |
| 2+ exit code 1 · invalid assertion |