From 7fc32561babdee8de79222c8bfe84579b5aa91b3 Mon Sep 17 00:00:00 2001 From: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com> Date: Thu, 1 Feb 2024 16:14:56 +0000 Subject: [PATCH 1/3] Update docs for `cylc lint` Deprecated `[cylc-lint]` in favour of `[tool.cylc.lint]` in `pyproject.toml` --- .../writing-workflows/configuration.rst | 48 ++++++++++++------- 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/src/user-guide/writing-workflows/configuration.rst b/src/user-guide/writing-workflows/configuration.rst index 832ce9ec079..90ba63c8f9f 100644 --- a/src/user-guide/writing-workflows/configuration.rst +++ b/src/user-guide/writing-workflows/configuration.rst @@ -65,7 +65,7 @@ documentation, configuration files, etc. When the workflow is :ref:`installed .. _SyntaxHighlighting: Syntax Highlighting For Workflow Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------------------------------------- Cylc provides syntax plugins for the following editors: @@ -107,7 +107,7 @@ WebStorm Install the `Cylc.tmbundle`_. Gross File Structure -^^^^^^^^^^^^^^^^^^^^ +-------------------- Cylc :cylc:conf:`flow.cylc` files consist of configuration items grouped under several top level section headings: @@ -134,7 +134,7 @@ several top level section headings: .. _Validation: Validation -^^^^^^^^^^ +---------- The ``cylc validate`` command evaluates the :cylc:conf:`flow.cylc` file against a specification that defines all legal entries, values and options. @@ -150,14 +150,20 @@ show an inlined copy of the workflow with correct line numbers. .. _cylc_lint_script: -``cylc lint`` -^^^^^^^^^^^^^ +Linting +------- + +The ``cylc lint`` command checks code style, deprecated syntax and other +issues in Cylc configuration files. .. seealso:: :ref:`How to configure Cylc lint at project level ` using a ``pyproject.toml``. +``cylc lint`` +^^^^^^^^^^^^^ + .. automodule:: cylc.flow.scripts.lint .. _lint.pyproject.toml: @@ -168,32 +174,38 @@ Configure ``cylc lint`` at project level You can configure ``cylc lint`` for each workflow using a ``pyproject.toml`` file. -To define settings for ``cylc lint`` use a ``[cylc-lint]`` section. -Within the ``[cylc-lint]`` section you may define the following: +To define settings for ``cylc lint`` use a ``[tool.cylc.lint]`` section. +Within the ``[tool.cylc.lint]`` section you may define the following: -rulesets - A list of rulesets to use. If you run cylc lint without setting rulesets - on the command line this value will override - the default (``['728', 'style']``). +``rulesets`` + A list of rulesets to use. - Allowed Values: '728', 'style' + Allowed values: ``'728'``, ``'style'``, ``'all'``. -ignore +``ignore`` Individual rules to ignore: A list of rule codes, such as ``S007``. -exclude +``exclude`` A list of files or glob patterns for files which will not be checked. -max-line-length - Set longest line length to permit in Cylc Configs for this project. - If unset, line length is not checked. +``max-line-length`` + Set longest line length to permit in Cylc configs for this project. + + Default: ``130``. + + +.. note:: + + .. versionchanged:: 8.3.0 + The ``[cylc-lint]`` section has been deprecated in favour of + ``[tool.cylc.lint]``. An example ``pyproject.toml`` might look like this: .. code-block:: toml - [cylc-lint] + [tool.cylc.lint] # Enforce a line limit of 99 chars max-line-length = 99 From 09bac61c6062138684379f166f4f8c05f4e2c22f Mon Sep 17 00:00:00 2001 From: Tim Pillinger <26465611+wxtim@users.noreply.github.com> Date: Fri, 9 Feb 2024 14:50:08 +0000 Subject: [PATCH 2/3] Update src/user-guide/writing-workflows/configuration.rst --- src/user-guide/writing-workflows/configuration.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/user-guide/writing-workflows/configuration.rst b/src/user-guide/writing-workflows/configuration.rst index 90ba63c8f9f..067b5450132 100644 --- a/src/user-guide/writing-workflows/configuration.rst +++ b/src/user-guide/writing-workflows/configuration.rst @@ -181,6 +181,8 @@ Within the ``[tool.cylc.lint]`` section you may define the following: A list of rulesets to use. Allowed values: ``'728'``, ``'style'``, ``'all'``. + + (You can override this on the command line.) ``ignore`` Individual rules to ignore: A list of rule codes, such as ``S007``. From 6b189e9dd38af0264756d3d476fe762c628503e6 Mon Sep 17 00:00:00 2001 From: Oliver Sanders Date: Thu, 15 Feb 2024 14:15:00 +0000 Subject: [PATCH 3/3] improve workflow events documentation (#685) --- .../writing-workflows/scheduler.rst | 118 +++++++++++++----- 1 file changed, 90 insertions(+), 28 deletions(-) diff --git a/src/user-guide/writing-workflows/scheduler.rst b/src/user-guide/writing-workflows/scheduler.rst index 5d941f5c799..497ab0ccb46 100644 --- a/src/user-guide/writing-workflows/scheduler.rst +++ b/src/user-guide/writing-workflows/scheduler.rst @@ -10,53 +10,115 @@ workflows. .. _user_guide.scheduler.workflow_event_handling: +.. _user_guide.scheduler.workflow_events: -Workflow Event Handling ------------------------ +Workflow Events +--------------- -Workflow event handlers allow configured commands to run when workflow events -occur. +There are two types of event in Cylc: -.. note:: +* workflow events e.g. ``startup`` and ``shutdown``, which pertain to the :term:`scheduler` +* task events e.g. ``submitted`` and ``failed``, which pertain to :term:`tasks `. - Cylc supports workflow events e.g. ``startup`` and ``shutdown`` - and task events e.g. ``submitted`` and ``failed``. +This section covers workflow events, for +task events see :ref:`user_guide.runtime.task_event_handling`. - See also :ref:`user_guide.runtime.task_event_handling`. +.. rubric:: Event Handlers -Workflow event handlers are configured by: +Workflow events have "handlers" (i.e. hooks) which allow configured commands to +run when workflow events occur. These can be configured by: * :cylc:conf:`flow.cylc[scheduler][events]` (per workflow) * :cylc:conf:`global.cylc[scheduler][events]` (user/site defaults) +.. rubric:: Abort On Event -Workflow Events -^^^^^^^^^^^^^^^ +As well as event handlers, you can tell the scheduler to abort (i.e., shut down +immediately with error status) on certain workflow events, using the +``abort on ...`` configurations. + +.. rubric:: Configuration + +Some workflow events have related configurations e.g. for setting the timeout. + +.. rubric:: List of workflow events: + +.. cylc-scope:: global.cylc[scheduler][events] + +.. describe:: startup + + :Event Handler: `startup handlers` -The list of events is: + The scheduler was started or restarted. + + E.G. using one of these commands ``cylc play``, ``cylc vip`` or ``cylc vr``. + +.. describe:: shutdown + + :Event Handler: `shutdown handlers` + + The scheduler was shut down. + + E.G. using the ``cylc stop`` command. + +.. describe:: abort + + :Event Handler: `abort handlers` -startup - The scheduler started running the workflow. -shutdown - The workflow finished and the scheduler will shut down. -abort The scheduler shut down early with error status, due to a fatal error condition or a configured timeout. -workflow timeout + +.. describe:: workflow timeout + + :Configuration: `workflow timeout` + :Event Handler: `workflow timeout handlers` + :Abort On Event: `abort on workflow timeout` + The workflow run timed out. -stall - The workflow stalled. -stall timeout + + The timer starts counting down at scheduler startup. It resets on workflow + restart. + + Note, the ``abort`` event is not raised by "Abort On Event" handlers. + +.. describe:: stall + + :Event Handler: `stall handlers` + + The workflow :term:`stalled ` (i.e. the scheduler cannot make any + further progress due to runtime events). + + E.G. a task failure is blocking the pathway through the graph. + +.. describe:: stall timeout + + :Configuration: `stall timeout` + :Event Handler: `stall timeout handlers` + :Abort On Event: `abort on stall timeout` + The workflow timed out after stalling. -inactivity timeout - The workflow timed out with no activity. -You can tell the scheduler to abort (i.e., shut down immediately with error -status) on certain workflow events, with the following settings: +.. describe:: inactivity timeout + + :Configuration: `inactivity timeout` + :Event Handler: `inactivity timeout handlers` + :Abort On Event: `abort on inactivity timeout` + + The workflow timed out with no activity (i.e. a period with no job + submissions or task messages). + + This can be useful for system administrators to help catch workflows which + have become stalled on external conditions or system issues. + +.. describe:: restart timeout + + :Configuration: `restart timeout` + + If a workflow that has run to completion is restarted, the scheduler will + have nothing to do so will shut down. This timeout gives the user a grace + period in which to trigger new tasks to continue the workflow run. -- abort on stall timeout -- abort on inactivity timeout -- abort on workflow timeout +.. cylc-scope:: Mail Events ^^^^^^^^^^^