Add a "dry run" option

[setrofim]

In some circumstances it may be useful to be able to see what is going to happen during a WA run without actually running anything; e.g. there may be a complicated agenda where the resulting sequence of execution may not be immediately obvious.

Proposal

Add a --dry-run command line option to the run command. This will result in log output for the run with no actions actually taken on the device.

Discussion

An immediately obvious way of implementing this would be to introduce an alternative runner which derives from the existing runner implementation and overrides some of the methods such as _reboot_device and _run_workload_iteration to only produce log output.

[garethstockwell]

In addition to reboot events and workload iterations, it would also be useful for the dry run output to show configuration changes, for example writes to sysfiles.

[setrofim]

Logging actual changes that would occur on a device may be tricky. It may be possible to implement this level of detail buy adding a "dry-run" Device interface, rather than a Runner; so the runner executes the workload as normal but all calls to execute(), set_sysfile_value(), push_file(), etc will just result in log output.

The problem is that some of these changes will occur depending on the information read from the device, so a live connection is not available, it would not be possible to log everything that would happend. Only way around it would be constrain the "dry run" option to only work if a live connection to the device is aviable. In addition, executing extensions may also result in changes on the host as well as on the device, and there is no straight forward way to shim that, as there is no unified interface for the host unlike the target device (extensions can use all the standard Python APIs to, e.g., kick off subprocess or create fils and directories etc).

On the other hand, if we're only talking about logging configuration that would be applied, rather than actual changes that would happen on the device, that would be straight forward to do with the DryRunner implmentation outlined in the OP.