Unexpected diff in rendered template file

[antonosmond]

Hi

I have a strange problem with a template_file which causes a diff every time I run a plan/apply.
I've tracked the cause of the problem down to the depends_on declared in my template_file resource. If the depends_on is there, it causes a diff, despite the fact that no values from the dependant resource are used within the template/template variables. I've even checked the rendered content in the remote state vs the rendered content locally and they're identical in every way, right down to line endings and white space. If I remove the depends_on, it works as expected and there is no diff. I can work around the issue but thought I'd point it out as it was unexpected.

Terraform Version

0.10.2

Affected Resource(s)

• data.template_file

Terraform Configuration Files

data "aws_s3_bucket_object" "archive" {
  bucket = "${var.bucket}"
  key    = "${var.product}/${var.service}/${var.service}-${var.version}.zip"
}

data "template_file" "user_data" {
  depends_on = ["data.aws_s3_bucket_object.archive"]
  template   = "${file("${path.module}/templates/userdata.ps1")}"

  vars {
    bucket  = "${var.bucket}"
    key     = "${var.product}/${var.service}/${var.service}-${var.version}.zip"
    service = "${var.service}"
    version = "${var.version}"
  }
}

Expected Behavior

No diff during plan/apply

Actual Behavior

Causes a diff when running plan/apply

Steps to Reproduce

1. terraform plan

Additional Info

The reason for my seemingly unused depends_on is that my userdata for my launch configs has script to pull an object from an S3 bucket. If the object doesn't exist in S3 at the point I run apply, I have to wait 10-20 minutes for the new ASG to bring all it's instances up and then timeout waiting on the ELB health check because the instances never became healthy because the userdata failed because the S3 object didn't exist. By adding the depends_on, it simply ensures that the S3 object exists before it attempts to bring everything up and if it doesn't exist, will fail immediately and prevent wasting time and money in AWS. I can move the depends on to my launch configs but the reason I had it on the userdata was the userdata is shared by multiple launch configs.

If you would like any additional info just give me a shout.

Thanks!

[ricoli]

Indeed, I've seen this same behaviour more than a year later, using version 1.0.0 of this provider

[apparentlymart]

Hi all,

The cause of this is actually in Terraform Core itself rather than in the provider. The issue is that depends_on for data resources currently forces Terraform to always defer the data source read to apply time, since depends_on doesn't give enough information for Terraform to know whether the resource is depending on an apply-time side-effect of the dependency. This limitation is captured in hashicorp/terraform#17034.

The workaround for now is to use implicit rather than explicit dependencies, which tells Terraform a specific attribute of the other object that is needed before the data resource can be processed and thus allows Terraform to recognize that it is safe to read it during the plan step. In this case this can be achieved by reading the bucket name through the other data resource, thus creating that dependency:

data "aws_s3_bucket_object" "archive" {
  bucket = "${var.bucket}"
  key    = "${var.product}/${var.service}/${var.service}-${var.version}.zip"
}

data "template_file" "user_data" {
  template   = "${file("${path.module}/templates/userdata.ps1")}"

  vars = {
    bucket  = "${data.aws_s3_bucket_object.archive.bucket}"
    key     = "${var.product}/${var.service}/${var.service}-${var.version}.zip"
    service = "${var.service}"
    version = "${var.version}"
  }
}

To eventually resolve hashicorp/terraform#17034 in Terraform Core we intend to adjust the lifecycle of data sources slightly so that the steps can happen in the right order to allow depends_on to work in a more useful way. That work has been on hold while we've been working on the 0.12 release but we will dig into some further design/prototyping for it afterwards.