fix(terraform): hcl object expressions to return references

[Emyrk]

Description

References included in hcl objects should be returned from 'Attribute.AllReferences()'. Traverse the object fields recursively to locate references.

Expressions such as:

data "foo" "bar" {
  tags = {
    "cacheTTL"   = var.cache
  }
}

Should return variable.cache in the AllReferences()

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[nikpivkin]

You need to use createDotReferenceFromTraversal, not referencesFromExpression.

[Emyrk]

I modified a unit test.

This is an object that sets foo to a conditional expression. That conditional expression has 2 references in it.

{foo = 1 == 1 ? var.bar : data.aws_ami.ubuntu.most_recent}

Maybe I am misunderstanding, but if I do

if ref, err := createDotReferenceFromTraversal(a.module, item.KeyExpr.Variables()...); err == nil {
  refs = append(refs, ref)
}
// And do the same for the ValueExpression

I get variable.bar.data.aws_ami.ubuntu.most_recent. Since Variables() call just appends all traversal types:

https://github.com/hashicorp/hcl/blob/main/hclsyntax/variables.go#L18-L20

It is not intelligent to know the conditional expression has 2 different references, and just returns them as a single list. Rather than 2.

[Emyrk]

@nikpivkin let me know I am misusing createDotReferenceFromTraversal

[nikpivkin]

Hi @Emyrk! Thanks for contributing!

Left a couple of comments. Also can you fix the linting issues?

[Emyrk]

Using your example (local.foo): local.foo, I also had to add these @nikpivkin

[Emyrk]

@nikpivkin unrelated, but is there a reason the underlying *hcl.Attribute is unexported on Attribute? I ask because I am looking to use this library in my use case, however there are times where the Value() will be unknown. I want to look into the HCL to deduce further what is causing it to be unknown, primarily for debugging/logging purposes.

I was wondering if a getter like func (a *Attribute) HCLAttribute() *hcl.Attribute would be totally out of the question.

[simar7]

@nikpivkin unrelated, but is there a reason the underlying *hcl.Attribute is unexported on Attribute? I ask because I am looking to use this library in my use case, however there are times where the Value() will be unknown. I want to look into the HCL to deduce further what is causing it to be unknown, primarily for debugging/logging purposes.

What's your use case for this? Value() is designed to return the attribute value in the case it is known and not null.

[Emyrk]

@simar7

What's your use case for this? Value() is designed to return the attribute value in the case it is known and not null.

I am doing work on https://github.com/coder/coder. Terraform is the configuration language we use for the underlying developer workspaces.

To give even more functionality/flexibility in the experience, we have a feature called "Build Parameters". See example image.

In order to speed up some of the inner loop and provide more immediate feedback on these parameters, I am seeking to use static anaylsis of the terraform (using your package, which is awesome by the way ❤️). Now this is not perfect, if the parameters include a reference to an external data source (docker_image.ubuntu.repo_digest), then the value will be unknown. This is correct, and I was using AllReferences() to fetch the dependencies.

I intend to use the existing terraform state to populate these unknown references in some second "pass/step". I do not see this package injesting a tfstate, so was going to build something to mutate the eval context.

---

My usage of the specific functionality is AllReferences(). This works well for an entire block (this PR included). However, in one of my use cases I have a block that looks like:

data "coder_workspace_tags" "custom_workspace_tags" {
  tags = {
    // AllReferences() = [docker_image.ubuntu.repo_digest, docker_image.centos]
    "foo" = docker_image.ubuntu.repo_digest
    "bar" = docker_image.centos.repo_digest
    "qux" = "quux"
  }
}

I am using attr.Each(func(key cty.Value, val cty.Value) to iterate over the fields foo and bar. Unfortunately, when calling .Value() to cast into a cty.Value, the references are lost into the cty.unknownType.

So I can tell the reference docker_image.<ubuntu | centos>.repo_digest is used by the block tags, but I cannot tell exactly where. Ideally I could deduce foo -> docker_image.ubuntu.repo_digest and bar -> docker_image.centos.repo_digest.

To do this, I would either need to have the underlying hcl.Expression myself, or expand upon AllReferences to pass in some traversal to do first?

I see additional value in exposing the hcl.Expression, as it is the raw information that backs a given cty.Value. I understand the point to encapsulate the expressions, it is just harder to hook into without the raw hcl.Expression.

[simar7]

@simar7

What's your use case for this? Value() is designed to return the attribute value in the case it is known and not null.

I am doing work on https://github.com/coder/coder. Terraform is the configuration language we use for the underlying developer workspaces.

To give even more functionality/flexibility in the experience, we have a feature called "Build Parameters". See example image.

In order to speed up some of the inner loop and provide more immediate feedback on these parameters, I am seeking to use static anaylsis of the terraform (using your package, which is awesome by the way ❤️). Now this is not perfect, if the parameters include a reference to an external data source (docker_image.ubuntu.repo_digest), then the value will be unknown. This is correct, and I was using AllReferences() to fetch the dependencies.

I intend to use the existing terraform state to populate these unknown references in some second "pass/step". I do not see this package injesting a tfstate, so was going to build something to mutate the eval context.

My usage of the specific functionality is AllReferences(). This works well for an entire block (this PR included). However, in one of my use cases I have a block that looks like:

data "coder_workspace_tags" "custom_workspace_tags" {
  tags = {
    // AllReferences() = [docker_image.ubuntu.repo_digest, docker_image.centos]
    "foo" = docker_image.ubuntu.repo_digest
    "bar" = docker_image.centos.repo_digest
    "qux" = "quux"
  }
}

I am using attr.Each(func(key cty.Value, val cty.Value) to iterate over the fields foo and bar. Unfortunately, when calling .Value() to cast into a cty.Value, the references are lost into the cty.unknownType.

So I can tell the reference docker_image.<ubuntu | centos>.repo_digest is used by the block tags, but I cannot tell exactly where. Ideally I could deduce foo -> docker_image.ubuntu.repo_digest and bar -> docker_image.centos.repo_digest.

To do this, I would either need to have the underlying hcl.Expression myself, or expand upon AllReferences to pass in some traversal to do first?

I see additional value in exposing the hcl.Expression, as it is the raw information that backs a given cty.Value. I understand the point to encapsulate the expressions, it is just harder to hook into without the raw hcl.Expression.

Thanks for the explanation. I don't see anything inherently wrong with exposing it. WDYT @nikpivkin?