Skip to content

Commit

Permalink
fix: prevent concurrent build to edit the same namespace
Browse files Browse the repository at this point in the history
This PR prevents builds to edit the same namespace at the same time.

Such a behavior could lead to a namespace to be accidentally wiped out.

Only `jx step helm apply` is locked.

A ConfigMap `jx-lock-{namespace}` is used as a lock. No other build can run while this configmap exists. Waiting builds can edit the data of the ConfigMap in order to be the next one to run. If a build sees that the locking or waiting build is "higher", it will fail. When the build finished, the ConfigMap is removed. A waiting build can also remove the ConfigMap if the lokcing pod has finished.

The algorithm is approximately:
```
    Label: CREATE
    try to create the ConfigMap
    if it succeeds:
        return
    Label: READ
    get the ConfigMap and the locking Pod
    if the locking Pod has finished
        remove the ConfigMap
        goto CREATE
    if the ConfigMap references a "higher" build
        fail
    if the ConfigMap references a "lower" build
        update the ConfigMap
    wait for the ConfigMap or the Pod to be updated
        if the ConfigMap is delete
            goto CREATE
        if the ConfigMap references a different build
            goto READ
        if the Pod has finished
            goto CREATE
```

fixes #6167

Signed-off-by: Aurélien Lambert <[email protected]>
  • Loading branch information
aure-olli committed Mar 26, 2020
1 parent ad2bff0 commit 754df94
Show file tree
Hide file tree
Showing 5 changed files with 1,213 additions and 6 deletions.
12 changes: 6 additions & 6 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ require (
github.com/ghodss/yaml v1.0.0
github.com/gliderlabs/ssh v0.1.1 // indirect
github.com/go-ole/go-ole v1.2.1 // indirect
github.com/go-openapi/jsonreference v0.19.2
github.com/go-openapi/spec v0.19.3
github.com/go-stack/stack v1.8.0 // indirect
github.com/go-openapi/jsonreference v0.19.3
github.com/go-openapi/loads v0.19.5 // indirect
github.com/go-openapi/spec v0.19.6
github.com/gobwas/glob v0.2.3 // indirect
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/protobuf v1.3.1
Expand All @@ -76,7 +76,7 @@ require (
github.com/knative/build v0.7.0 // indirect
github.com/knative/pkg v0.0.0-20190624141606-d82505e6c5b4
github.com/knative/serving v0.7.0
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/kubernetes-incubator/reference-docs v0.0.0-20190410022412-9642bd3f4de5 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lusis/go-slackbot v0.0.0-20180109053408-401027ccfef5 // indirect
github.com/lusis/slack-test v0.0.0-20180109053238-3c758769bfa6 // indirect
Expand All @@ -103,7 +103,7 @@ require (
github.com/sethvargo/go-password v0.1.2
github.com/shirou/gopsutil v0.0.0-20180901134234-eb1f1ab16f2e
github.com/shirou/w32 v0.0.0-20160930032740-bb4de0191aa4 // indirect
github.com/sirupsen/logrus v1.4.1
github.com/sirupsen/logrus v1.4.2
github.com/spf13/cobra v0.0.5
github.com/spf13/pflag v1.0.3
github.com/spf13/viper v1.3.2
Expand Down Expand Up @@ -134,7 +134,7 @@ require (
gopkg.in/src-d/go-git-fixtures.v3 v3.3.0 // indirect
gopkg.in/src-d/go-git.v4 v4.5.0
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.2.4
gopkg.in/yaml.v2 v2.2.8
k8s.io/api v0.0.0-20190718183219-b59d8169aab5
k8s.io/apiextensions-apiserver v0.0.0-20190718185103-d1ef975d28ce
k8s.io/apimachinery v0.0.0-20190703205208-4cfb76a8bf76
Expand Down
Loading

0 comments on commit 754df94

Please sign in to comment.