[ISSUE] Issue with databricks_sql_table resource

[alexisleveratto]

Configuration

resource "databricks_sql_table" "system_views" {
  for_each = local.views_wks

  catalog_name   = local.catalog_name
  schema_name  = each.value.schema
  name                = each.value.table
  owner               = each.value.owner

  table_type = "VIEW"
  view_definition = format(<<-EOF
    SELECT *
    FROM system.%s.%s
    WHERE workspace_id IN (%s)
  EOF
    ,
    each.value.schema, each.value.table,
    join(", ", each.value.workspaces_id)
  )

  properties = {
    "delta.mergeSchema" = "true"
  }
  depends_on = [databricks_schema.system_schemas]
}

Expected Behavior

The property delta.mergeSchema should remain true when set explicitly, ensuring that schema changes are handled seamlessly in Databricks views without Terraform attempting to reset the property.

Actual Behavior

When delta.mergeSchema is set to true to handle schema changes, Terraform defaults this property to false. During the next terraform apply, Terraform attempts to set delta.mergeSchema back to true, causing the operation to fail.

Steps to Reproduce

1. Configure a Databricks view with delta.mergeSchema set to true.
2. Apply the configuration with Terraform.
3. Observe that Terraform reverts the property to false.
4. On subsequent terraform apply, Terraform attempts to reset the property to true, resulting in a failure.

Terraform and provider versions

databricks = {
source = "databricks/databricks"
version = "1.53.0"
}

Is it a regression?

This behavior has not been observed to work in previous versions of the provider.

Important Factoids

This behavior causes disruptions in environments where schema evolution needs to be managed dynamically.