Skip to content

Commit

Permalink
data-plane-controller: controller for managing data-plane life cycles
Browse files Browse the repository at this point in the history
The controller uses the recent `automations` crate to monitor structured
changes to a bound data_planes and drive the data-plane to convergence.

It's modeled as a CI/CD pipeline, where an indicated branch of our
dry-dock repo (containing Pulumi and Ansible infrastructure) is deployed
for the given data-plane.

The controller performs the full lifecycle required for rolling updates:

- `pulumi up` to create new resources or respond to replacements
- Awaiting DNS propagation
- Running the Ansible playbook to provision instances
- Running `pulumi up` to reflect readiness of started instances
- Awaiting DNS propagation (again)

It also periodically refreshes a stack from remote providers to detect
changed or deleted resources, such as EC2 instance replacements, and
responds accordingly to heal the infrastructure.
  • Loading branch information
jgraettinger committed Oct 29, 2024
1 parent 507c78e commit 5783c8c
Show file tree
Hide file tree
Showing 9 changed files with 1,578 additions and 0 deletions.
38 changes: 38 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ humantime = "2.1"
humantime-serde = "1.1"
itertools = "0.10"
indexmap = { version = "1.8", features = ["serde"] }
ipnetwork = { version = "0.19", features = ["serde"] }
iri-string = "0.6.0"
jemallocator = { version = "0.3", features = ["profiling"] }
jemalloc-ctl = "0.3"
Expand Down Expand Up @@ -139,6 +140,7 @@ strum_macros = "0.24"
superslice = "1.0"
sqlx = { version = "0.6", features = [
"chrono",
"ipnetwork",
"json",
"macros",
"postgres",
Expand Down
34 changes: 34 additions & 0 deletions crates/data-plane-controller/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[package]
name = "data-plane-controller"
version.workspace = true
rust-version.workspace = true
edition.workspace = true
authors.workspace = true
homepage.workspace = true
repository.workspace = true
license.workspace = true

[dependencies]
async-process = { path = "../async-process" }
automations = { path = "../automations" }
models = { path = "../models" }
ops = { path = "../ops" }

anyhow = { workspace = true }
chrono = { version = "0.4", features = ["serde"] }
clap = { workspace = true }
futures = { workspace = true }
humantime = { workspace = true }
humantime-serde = { workspace = true }
ipnetwork = { workspace = true }
itertools = { workspace = true }
rustls = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
serde_yaml = { workspace = true }
sqlx = { workspace = true }
tempfile = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
url = { workspace = true }
Loading

0 comments on commit 5783c8c

Please sign in to comment.