Skip to content

Commit

Permalink
Update expire proposal.
Browse files Browse the repository at this point in the history
  • Loading branch information
hjoliver committed Apr 21, 2023
1 parent e06dc22 commit 94820ae
Showing 1 changed file with 21 additions and 11 deletions.
32 changes: 21 additions & 11 deletions docs/proposal-task-expire.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,29 +15,33 @@ not needed anymore.
NOTE that forced expiration is exactly the same, except that it is caused by
the user rather than the clock. In both cases we expire an incomplete task
(whether waiting or finished-but-incomplete) because it does not need to run
(or rerun) in order to be considered complete.
(or rerun) - it's completion is no longer required.

### Expired Tasks Are "Complete" and Should Not Cause a Stall
### Expired Tasks Should Not Cause a Stall

If a task expires, the workflow definition, or the user, has decreed that under
current conditions it *should* expire, i.e., we don't need to run it anymore.

That is an kind of success, and we should therefore mark expired tasks as
complete so the scheduler can forget them - not retain them as incomplete to
cause a stall.
Therefore expired tasks should not be retained as incomplete tasks (which is a
surprising/error condition) and allowed to stall the workflow.

### Tasks Downstream of an Expired Task Should Not Cause a Stall

If the workflow definition, or the user, has decreed that under current
circumstances a task should expire, i.e. that it should not run all all, then
by implication nothing downstream of it should run either.


```
a => foo => bar # a succeeds, foo expires so foo never runs
```
There is no reason to stall on account of `bar` here. Required outputs are always
conditional on the owner task actually running in the first place, and the
downstream graph should not spawn at all if the outputs it hangs off are not
generated.
So there is no reason to stall on account of `bar` here. Required outputs are
always conditional on the owner task actually running in the first place, and
the downstream graph should not spawn at all if the outputs it hangs off are
not generated.

From the perspective of `bar` this no different than if `foo` not running because
the branch it lives on wasn't taken at all:
From the perspective of `bar` this no different than if `foo` not running
because the branch it lives on wasn't taken at all:

```
a:x? => foo => bar # a does not generate :x, so foo never runs
Expand All @@ -64,6 +68,8 @@ expires, then :expire triggers must be used to achieve that.
We should enforce that :expire triggers be marked optional, because "required
expiration" doesn't really make sense.

(N.B. this is moot if we decide that `:expire` is not a task output - see
below).

### Optional :expire Does Not Mean :succeed Must Be Optional

Expand All @@ -75,13 +81,17 @@ means, *if the task runs, its success is required*.
The status of an expired task's required outputs is the same as that of the
required outputs of a task on a branch not taken at runtime.

```
foo:x? => bar
foo:y? => baz
```

Here, bar:succeed is required ONLY if bar runs, i.e. if branch x is taken. If
branch y is taken, the scheduler does not car that bar did not succeed.

```
foo | foo:expire? => bar
```

Here, foo:succeed is required ONLY if foo runs, i.e. if it doesn't expire. If
foo expires, the scheduler should not care that foo (and bar) did not succeed.
Expand Down

0 comments on commit 94820ae

Please sign in to comment.