<craft-app>-test.yaml¶
This reference describes the usage of and provides examples for every key in a
configuration file for a package’s tests, <craft-app>-test.yaml.
Top-level keys¶
Top-level keys define global exclusion patterns, prepare and restore scripts, and execution timeouts.
exclude¶
Type
list[str]
Description
The list of file and directory patterns relative to the project root to exclude from the test environment. Defaults to [‘.git’, ‘.tox’].
Examples
exclude:
- .git
- .tox
- docs/
prepare¶
Type
str
Description
The shell script executed once on the test host before running any test suite.
Examples
prepare: echo "Setting up global test prerequisites"
prepare-each¶
Type
str
Description
The shell script executed before each individual test task in every suite.
Examples
prepare-each: rm -rf /tmp/test-output
restore¶
Type
str
Description
The shell script executed once after all test suites complete, regardless of outcome.
Examples
restore: echo "Tearing down global test environment"
restore-each¶
Type
str
Description
The shell script executed after each individual test task in every suite completes.
Examples
restore-each: rm -rf /tmp/test-output
debug¶
Type
str
Description
The shell script executed if any step fails during the test run.
Examples
debug: journalctl -xe
debug-each¶
Type
str
Description
The shell script executed if an individual test task fails.
Examples
debug-each: dmesg | tail -n 50
kill-timeout¶
Type
str
Description
The maximum duration allowed for tests before ending the process. Expressed as a duration string, such as ‘30m’ or ‘1h’.
Examples
kill-timeout: 30m
Backend keys¶
Backend keys configure the runtime environments where tests execute. The craft
backend is used by default by Craft Application to configure system images and worker
instances for tests. Other backends can be manually specified.
backends¶
Type
dict[str, CraftSpreadBackend]
Description
The mapping of backend configurations used to execute tests.
Usually includes the craft backend, which Craft Application populates automatically with
target system images.
Examples
backends:
craft:
systems:
- ubuntu-24.04
backends.<backend-name>.type¶
Type
str
Description
The backend driver type, such as craft, lxd, or adhoc. For backends other than craft, Spread uses the backend name as the type when this key is omitted.
Examples
type: craft
backends.<backend-name>.systems¶
Type
list[str | dict[str, CraftSpreadSystem | None]]
Description
The list of systems (operating system distributions or architectures) available for running tests.
Can be specified as system names or mappings of system names to system configurations.
Examples
systems:
- ubuntu-24.04
- ubuntu-22.04:
workers: 2
backends.<backend-name>.allocate¶
Type
str
Description
The command or script to allocate a remote instance for an adhoc backend.
Examples
allocate: allocate-cloud-instance
backends.<backend-name>.discard¶
Type
str
Description
The command or script to release an allocated remote instance for an adhoc backend when finished.
Examples
discard: release-cloud-instance
backends.<backend-name>.prepare¶
Type
str
Description
The shell script executed on the backend system before running any tests.
This should only be used to run backend-specific preparation. It is not compatible
with the craft backend.
Examples
prepare: apt-get update
backends.<backend-name>.prepare-each¶
Type
str
Description
The shell script executed on the backend system before each test task.
Examples
prepare-each: systemctl restart test-service
backends.<backend-name>.restore¶
Type
str
Description
The shell script executed on the backend system after all tests finish.
This should only be used for backend-specific cleanup or restoration. It is not
compatible with the craft backend.
Examples
restore: apt-get clean
backends.<backend-name>.restore-each¶
Type
str
Description
The shell script executed on the backend system after each test task completes.
Examples
restore-each: systemctl stop test-service
backends.<backend-name>.debug¶
Type
str
Description
The shell script executed on the backend system if a backend prepare or restore step encounters an error.
Examples
debug: cat /var/log/syslog
backends.<backend-name>.debug-each¶
Type
str
Description
The shell script executed on the backend system if an individual test task fails.
Examples
debug-each: systemctl status test-service
System keys¶
When a backend system requires specific configuration options rather than default
settings, it is specified as a mapping under the backend systems list.
backends.<backend-name>.systems.<system-name>.workers¶
Type
int
Description
The number of concurrent worker instances to create for this system.
Examples
workers: 2
backends.<backend-name>.systems.<system-name>.image¶
Type
str
Description
The custom container or virtual machine image to use for this system.
In <craft-app>-test.yaml, the craft backend uses the backend’s image
identifier format to override the image selected by Craft Application for the system.
Examples
image: ubuntu:24.04
image: ubuntu-noble-daily-amd64
Suite keys¶
Suite keys configure individual test suites. The suite key must match the path of the directory containing the tests and end with a trailing forward slash (/).
suites¶
Type
dict[str, CraftSpreadSuite]
Description
The mapping of test suite directories relative to the project root to their suite configurations.
Each suite directory key must end with a trailing forward slash (/).
Examples
suites:
tests/spread/general/:
summary: General integration tests
suites.<suite-path>.summary¶
Type
str
Description
A brief description of what the test suite covers.
Examples
summary: General integration tests
suites.<suite-path>.systems¶
Type
list[str]
Description
The list of systems on which to run this suite. If omitted, the suite runs on all systems defined in the backend.
Examples
systems:
- ubuntu-24.04
suites.<suite-path>.environment¶
Type
dict[str, str]
Description
The environment variables defined for all tests in this suite, expressed as key-value pairs.
Examples
environment:
TEST_MODE: production
VERBOSE: '1'
suites.<suite-path>.prepare¶
Type
str
Description
The shell script executed once before executing any tests in this suite.
Examples
prepare: echo "Preparing suite"
suites.<suite-path>.prepare-each¶
Type
str
Description
The shell script executed before each individual test task in this suite.
Examples
prepare-each: test -f "${CRAFT_ARTIFACT}"
suites.<suite-path>.restore¶
Type
str
Description
The shell script executed once after all tests in this suite complete.
Examples
restore: echo "Restoring suite"
suites.<suite-path>.restore-each¶
Type
str
Description
The shell script executed after each individual test task in this suite completes.
Examples
restore-each: rm -rf /tmp/task-cache
suites.<suite-path>.debug¶
Type
str
Description
The shell script executed if a suite prepare or restore step encounters an error.
Examples
debug: cat /tmp/suite-error.log
suites.<suite-path>.debug-each¶
Type
str
Description
The shell script executed if an individual test task in this suite fails.
Examples
debug-each: echo "Task $SPREAD_TASK failed"
suites.<suite-path>.kill-timeout¶
Type
str
Description
The maximum duration allowed for an individual test task in this suite before ending the process.
Overrides the top-level kill-timeout key.
Examples
kill-timeout: 15m