From c4cd253a45a874f596ad736a87eab6b1b7f1fd1d Mon Sep 17 00:00:00 2001 From: Tim Pillinger <26465611+wxtim@users.noreply.github.com> Date: Wed, 29 May 2024 13:06:11 +0100 Subject: [PATCH] Add details of skip mode --- .../running-workflows/simulation-modes.rst | 104 ++------- src/user-guide/task-implementation/index.rst | 1 + .../task-implementation/run-modes.rst | 206 ++++++++++++++++++ 3 files changed, 222 insertions(+), 89 deletions(-) create mode 100644 src/user-guide/task-implementation/run-modes.rst diff --git a/src/user-guide/running-workflows/simulation-modes.rst b/src/user-guide/running-workflows/simulation-modes.rst index d7f08914ed..661290b88e 100644 --- a/src/user-guide/running-workflows/simulation-modes.rst +++ b/src/user-guide/running-workflows/simulation-modes.rst @@ -1,101 +1,27 @@ .. _SimulationMode: -Simulation Modes ----------------- +Run Modes +--------- -Cylc can run a workflow without running the real jobs, which may be -long-running and resource-hungry. +The "live" and "simulation" modes described in +:ref:`task-run-modes` can be applied to the entire workflow. -**Dummy mode** replaces real jobs with background ``sleep`` jobs on the -scheduler host. This avoids :term:`job runner` directives that request compute -resources for real workflow tasks, and it allows any workflow configuration to -be run locally in dummy mode. +This is useful when developing workflows. Tasks submitted +using :ref:`task-run-modes.dummy` and +:ref:`task-run-modes.simulation` pathways can be used while +developing workflows to understand how different outcomes +and run lengths will affect the workflow. -.. code-block:: console - - $ cylc play --mode=dummy # real dummy jobs - -.. note:: - - All configured task ``script`` items including ``init-script`` are ignored - in dummy mode. If your ``init-script`` is required to run even local dummy - jobs, the relevant environment setup should be done elsewhere. - - -**Simulation mode** does not run real jobs at all, and does not generate job -log files. +To apply a run mode to a workflow: .. code-block:: console - $ cylc play --mode=simulation # no real jobs - - -Simulated Run Length -^^^^^^^^^^^^^^^^^^^^ - -The default dummy or simulated job run length is 10 seconds. It can be -changed with :cylc:conf:`[runtime][][simulation]default run length`. - -If :cylc:conf:`[runtime][]execution time limit` and -:cylc:conf:`[runtime][][simulation]speedup factor` are both set, -run length is computed by dividing the time limit by the speedup factor. - - -Simulated Failure -^^^^^^^^^^^^^^^^^ - -Tasks always complete custom outputs, by default they succeed. - -.. warning:: - - In simulation mode a succeeded output may not imply that - submitted and/or started outputs are generated - so you will not - be able to test graph pathways such as ``task:started => do_something``. - -You can set some or all instances of a task to fail using -:cylc:conf:`[runtime][][simulation]fail cycle points`. -``fail cycle points`` takes either a list of cycle point strings or "all". - -Tasks set to fail will succeed on their second or following simulated -submission. If you want all submissions to fail, set -:cylc:conf:`[runtime][][simulation]fail try 1 only = False`. - -For example, to simulate a task you know to be flaky on the half -hour but not on the hour: - -.. code-block:: cylc - - [[get_observations]] - execution retry delays = PT30S - [[[simulation]]] - fail cycle points = 2022-01-01T00:30Z, 2022-01-01T01:30Z - -In another case you might not expect the retry to work, and want to test -whether your failure handling works correctly: - -.. code-block:: cylc - - [[get_data]] - execution retry delays = PT30S - [[[simulation]]] - fail try 1 only = false - fail cycle points = 2022-01-01T03:00Z + $ cylc play --mode=dummy + $ cylc play --mode=simulation Limitations ^^^^^^^^^^^ -Dummy tasks run locally, so dummy mode does not test communication with remote -job platforms. However, it is easy to write a live-mode test workflow with -simple ``sleep 10`` tasks that submit to a remote platform. - -Alternate path branching is difficult to simulate effectively. You can -configure certain tasks to fail via -:cylc:conf:`[runtime][][simulation]`, but all branches based -on mutually exclusive custom outputs will run because all custom outputs get -artificially completed in dummy mode and in simulation mode. - -.. note:: - - Run mode is recorded in the workflow run database. Cylc will not let you - *restart* a dummy mode workflow in live mode, or vice versa. Instead, - install a new instance of the workflow and run it from scratch in the new mode. +Workflow run mode is recorded in the workflow run database. Cylc will not let you +*restart* a dummy mode workflow in live mode, or vice versa. Instead, +install a new instance of the workflow and run it from scratch in the new mode. diff --git a/src/user-guide/task-implementation/index.rst b/src/user-guide/task-implementation/index.rst index 4e16136758..214518155d 100644 --- a/src/user-guide/task-implementation/index.rst +++ b/src/user-guide/task-implementation/index.rst @@ -16,3 +16,4 @@ job fails and the task is re-run). job-scripts job-submission ssh-job-management + run-modes diff --git a/src/user-guide/task-implementation/run-modes.rst b/src/user-guide/task-implementation/run-modes.rst new file mode 100644 index 0000000000..63771e7794 --- /dev/null +++ b/src/user-guide/task-implementation/run-modes.rst @@ -0,0 +1,206 @@ +.. _task-run-modes: + +Task Run Modes +============== + +.. versionadded:: 8.4.0 + + Before 8.4.0 run modes could only be set for the workflow, not + for individual tasks. + + Skip mode is new at 8.4.0. + +As well as the default task implementation ("live" mode) +other implementations can be selected: + +* By setting the scheduler run mode as an argument to the + ``cylc play --mode`` command ("simulation" and "dummy" modes). +* By setting :cylc:conf:`[runtime][]run mode` for + "skip" mode. + +.. note:: + + :cylc:conf:`[runtime][]run mode` will override the + run mode set on the scheduler for any task for which it it set. + +#. :ref:`task-run-modes.simulation`: Intended to be used + when developing a workflow, the scheduler simulates + running a task without creating a "real" job. +#. :ref:`task-run-modes.dummy`: Intended to be used + when developing a workflow, the scheduler replaces + the workflow scripts with dummy scripts and returns + to the default ("live") submission pathway. +#. :ref:`task-run-modes.skip`: Intended to allow more control of the workflow, + the scheduler marks task outputs as completed. + +.. _task-run-modes.dummy: + +Dummy Mode +---------- + +**Dummy mode** replaces real jobs with background jobs on the +scheduler host which use ``sleep`` to simulate the run length according to +the settings described for simulation mode. +This avoids :term:`job runner` directives that request compute +resources for real workflow tasks, and it allows any workflow configuration to +be run locally in dummy mode. + +Limitations +^^^^^^^^^^^ + +Dummy tasks run locally, so dummy mode does not test communication with remote +job platforms. However, it is easy to write a live-mode test workflow with +simple ``sleep 10`` tasks that submit to a remote platform. + + +.. _task-run-modes.simulation: + +Simulation Mode +--------------- + +**Simulation mode** does not run real jobs, and does not generate job +log files. + + +Simulated Run Length +^^^^^^^^^^^^^^^^^^^^ + +The default dummy or simulated job run length is 10 seconds. It can be +changed with :cylc:conf:`[runtime][][simulation]default run length`. + +If :cylc:conf:`[runtime][]execution time limit` and +:cylc:conf:`[runtime][][simulation]speedup factor` are both set, +run length is computed by dividing the +:cylc:conf:`[runtime][]execution time limit` by +:cylc:conf:`[runtime][][simulation]` speedup factor. + +Simulated Failure +^^^^^^^^^^^^^^^^^ + +Tasks always complete all custom outputs, by default they succeed. + + If you want to test individual pathways, use + :ref:`skip mode `. + +You can set some or all instances of a task to fail using +:cylc:conf:`[runtime][][simulation]fail cycle points`. +``fail cycle points`` takes either a list of cycle point strings or "all". + +Tasks set to fail will succeed on their second or following simulated +submission. If you want all submissions to fail, set +:cylc:conf:`[runtime][][simulation]fail try 1 only = False`. + +For example, to simulate a task you know to be flaky on the half +hour but not on the hour: + +.. code-block:: cylc + + [[get_observations]] + execution retry delays = PT30S + [[[simulation]]] + fail cycle points = 2022-01-01T00:30Z, 2022-01-01T01:30Z + +In another case you might not expect the retry to work, and want to test +whether your failure handling works correctly: + +.. code-block:: cylc + + [[get_data]] + execution retry delays = PT30S + [[[simulation]]] + fail try 1 only = false + fail cycle points = 2022-01-01T03:00Z + +Limitations +^^^^^^^^^^^ + +Alternate path branching is difficult to simulate effectively. You can +configure certain tasks to fail via +:cylc:conf:`[runtime][][simulation]`, but all branches based +on mutually exclusive custom outputs will run because all custom outputs get +artificially completed in dummy mode and in simulation mode. + + +.. _task-run-modes.skip: + +Skip Mode +--------- + +Skip mode is designed as an aid to workflow control: + +* Allows creation of dummy tasks as part of workflow design. +* Allows skipping of tasks in a running workflow. + +Skip mode allows the user to specify which task outputs +will be emitted using :cylc:conf:`[runtime][][skip]outputs`. + +By default task event handlers are disabled by skip mode, but they +can be enabled using +:cylc:conf:`[runtime][][skip]disable task event handlers`. + +Set task to succeeded +^^^^^^^^^^^^^^^^^^^^^ + +.. admonition:: Scenario + + We don't need a task to run, but want to set it to succeed. + +Broadcast :cylc:conf:`[runtime][]run mode` setting the +value to ``skip``. + +Create a Dummy Task +^^^^^^^^^^^^^^^^^^^ + +.. admonition:: Scenario + + We have a large family to large family trigger. + + .. code-block:: cylc + + [task parameters] + x = 1..N # Where N is big. + [scheduling] + [[graph]] + R1 = FAMILY:succeed-all => FAMILY2 + [runtime] + [[FAMILY, FAMILY2]] + [[task1]] + inherit = FAMILY + [[task2]] + inherit = FAMILY2 + + If we increase N, the number of dependencies being tracked + by the scheduler is N^2. + +Adding a skip mode task between the families: + +.. code-block:: cylc + + FAMILY:succeed-all => skiptask => FAMILY2 + +Will reduce the number of dependencies to 2x. + +Parameter Exclusion +^^^^^^^^^^^^^^^^^^^ + +.. admonition:: Scenario + + We want to skip a small number of tasks from a parameterized + group of tasks: + + .. code-block:: cylc + + [task parameters] + # House number 13 doesn't actually exist on this street... + house_number = 1..20 + +We can use skip mode to make sure that a parameter task always +succeeds without running anything: + +.. code-block:: cylc + + [runtime] + [[post parcel]] + script = send letter + [[post parcel]] + run mode = skip