Running in CI

DART exits non-zero when tests fail and writes reports CI can render:

dart -c suite.yaml \
  --report junit:results.xml,json:results.json \
  --log run.log

.github/workflows/integration.yml

- name: Integration tests
  working-directory: tests            # local step paths resolve from here
  run: dart -c integration.yaml --report junit:results.xml
- name: Publish results
  uses: dorny/test-reporter@v1
  if: always()
  with:
    path: tests/results.xml
    reporter: java-junit
    fail-on-empty: false              # a run that aborts early writes no report

Note: report files are written on every exit from the test phase onward, so a failed test, a --stop-on-error abort, an --until exit, or a teardown failure all still produce the artifact. A run that aborts before the test phase writes no report at all — a platform or node setup failure, a fact-gathering error, a step or test construction error, and a failing setup step all return early, as do --setup-only and an --until target inside setup. Pipeline steps that consume the report must tolerate its absence, which is what fail-on-empty: false above does.

Useful flags for pipelines:

Flag	Why
--check	Validate the suite without touching infrastructure — a fast pre-commit lint, with the limits noted below
--report junit:PATH	Per-test results for GitHub/GitLab/Jenkins test panels
--vars key=value	Point one suite at staging or prod without editing YAML; comma-separated, and values may not themselves contain commas
--only tag=smoke	Run a subset; --skip tag=slow excludes. A filter that excludes every test is an error and exits non-zero
-s, --stop-on-error	Stop at the first failure
-d, --debug	Stream command output live while debugging a suite
-co, --color	auto (default), always, or never — auto colors when stdout is a terminal and honours NO_COLOR

One limit on --check is worth knowing before it is wired into a pre-commit hook: it validates only what step construction can check without a live node. file_template reads and parses its source when the step is built, so a missing or broken template is caught, whereas a missing file_push source is not — that surfaces mid-setup, after nodes exist. The stand-in node it substitutes for each declared node implements exactly that type’s real capabilities, so reboot and snapshot steps are accepted or rejected exactly as a real run would.

Parameterize suites so one file serves every environment:

suite: api smoke
vars:
  target: staging.internal      # dart -c suite.yaml --vars target=prod.internal
nodes:
  - name: local
    type: local
tests:
  - name: api answers
    node: local
    type: http_request
    tags: [smoke]
    options: { url: "https://{{var.target}}/health" }