Constraint changes causes replacements resulting in units being destroyed

[hloeung]

Description

Constraint changes or differences (see #344) causes units to be replaced and destroyed. This is scary for an environment hosting databases.

I can see use cases where we want to increase or scale up DB units. We do so by changing the constraints, adding new units, then cycling out the old.

Sadly, the current Juju Terraform provider sees constraint changes and causes replacements which means destroying units.

Urgency

Casually reporting

Terraform Juju Provider version

0.15.0

Terraform version

v1.7.2-dev

Juju version

3.5.3

Terraform Configuration(s)

locals {
  # Work around Juju `unexpected new value: .constraints: was cty.StringVal("... root-disk=20G"), but now cty.StringVal(" ... root-disk=20480M").`
  # https://github.com/juju/terraform-provider-juju/issues/344
  postgresql_disk_gigabytes       = var.disk_gigabytes * 1024
  postgresql_charm_constraints    = "arch=amd64 cores=${var.cores} mem=${var.mem_megabytes}M root-disk=${local.postgresql_disk_gigabytes}M"
  postgresql_root_disk_constraint = var.postgresql_use_local_storage ? " root-disk-source=local" : ""
}

resource "juju_application" "postgresql" {
  name  = "postgresql"
  model = var.juju_model_name
  trust = true

  charm {
    name     = "postgresql"
    channel  = var.postgresql_charm_channel
    revision = var.postgresql_charm_revision
    base     = var.postgresql_charm_base
  }

  expose {}

  units       = var.postgresql_charm_units
  constraints = "${local.postgresql_charm_constraints}${local.postgresql_root_disk_constraint}"
}

Reproduce / Test

terraform init & terraform apply

Update `postgresql_disk_gigabytes` to some other value

terraform apply

Debug/Panic Output

# module.postgresql-database.juju_application.postgresql must be replaced
-/+ resource "juju_application" "postgresql" {
      ~ constraints = "arch=amd64 cores=2 mem=4096M root-disk=20480M" -> "arch=amd64 cores=2 mem=4096M root-disk=30720M" # forces replacement
      ~ id          = "stg-is-dbaas-testing:postgresql" -> (known after apply)
        name        = "postgresql"
      ~ placement   = "5,6,7" -> (known after apply)
      + principal   = (known after apply)
      ~ storage     = [
          - {
              - count = 1 -> null
              - label = "pgdata-3" -> null
              - pool  = "rootfs" -> null
              - size  = "20G" -> null
            },
          - {
              - count = 1 -> null
              - label = "pgdata-4" -> null
              - pool  = "rootfs" -> null
              - size  = "20G" -> null
            },
          - {
              - count = 1 -> null
              - label = "pgdata-5" -> null
              - pool  = "rootfs" -> null
              - size  = "20G" -> null
            },
        ] -> (known after apply)
        # (3 unchanged attributes hidden)

      ~ charm {
            name     = "postgresql"
          ~ series   = "jammy" -> (known after apply)
            # (3 unchanged attributes hidden)
        }

        # (1 unchanged block hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

### Notes & References

_No response_

[hmlanigan]

The juju cli does not allow for constraints of existing units to be updated either. If you change an application's constraints, they apply to the next machines for the units deployed only, see juju set-constraints.

Juju lacks the ability to resize existing machines disk today.

As terraform is a declarative language so the expectation would be that every unit has machines with the same constraints. It's not as flexible as juju.

Your next problem would be that the provider will not let you choose which units are to be deleted. The oldest units are removed first.

I'd suggest that putting the DB on the root disk is a bad idea anyways. You'd want it on a volume which can be detached and reattached to specific units (juju attach-storage). This is not currently supported by the provider.

If this is a real world scenario, a design of how to accomplish it should be made and agreed to.

[hloeung]

Yes this is a real world scenario - databases grow and eventually needs to be resized increasing storage. Even with Juju managed storage (juju attach-storage), there isn't a way currently to grow the existing storage.

There's also the vCPU and RAM - as applications become used more, I can see a scenario where we need to scale up the no. of vCPUs and RAM.

We're fine with the Terraform provider not manage which units are deleted and which to cycle out. We just want a way to be able to update the constraint in the Terraform plan, then manually cycle out units and have Terraform provision new units with updated constraints in it's place. Or even for us operators/admins to manually resize the existing units to match the updated constraints.