[Feature] Add `databricks_credential` resource #4219

nkvuong · 2024-11-13T08:31:34Z

Changes

Add databricks_credential resource that represents service credential

Resolves #4214

Tests

make test run locally
relevant change in docs/ folder
covered with integration tests in internal/acceptance
relevant acceptance tests are passing
using Go SDK

alexott · 2024-11-13T11:25:28Z

docs/resources/credential.md

+
+A credential represents an authentication and authorization mechanism for accessing services on your cloud tenant. Each credential is subject to Unity Catalog access-control policies that control which users and groups can access the credential.
+
+To create credentials, you must be a Databricks account admin or have the `CREATE SERVICE CREDENTIAL` privilege. The user who creates the credential can delegate ownership to another user or group to manage permissions on it


Will create service credential work with service principals on azure?

Should we also mention CREATE_STORAGE_CREDENTIAL ?

the SDK only accepts SERVICE for purpose apparently

our go-sdk struct does not match the OpenAPI spec for some reason...maybe we need to wait for it to be regenerated

@farrucosanjurjo-db wdyt?

our go-sdk struct does not match the OpenAPI spec for some reason...maybe we need to wait for it to be regenerated

I think that's the issue here. We recently updated the proto files (and hence the OpenAPI specs) to include the management of storage credentials with the /credentials endpoints

Will create service credential work with service principals on azure?

No. We only support Azure Managed Identities

docs/resources/grant.md

docs/resources/credential.md

alexott · 2024-11-13T11:30:32Z

Should we add something regarding the IAM role requirements? https://docs.databricks.com/en/connect/unity-catalog/cloud-services/service-credentials.html#step-1-create-an-iam-role

alexott · 2024-11-14T08:21:02Z

catalog/resource_credential.go

+				return err
+			}
+			// Bind the current workspace if the external location is isolated, otherwise the read will fail
+			return bindings.AddCurrentWorkspaceBindings(ctx, d, w, updateCredRequest.NameArg, catalog.UpdateBindingsSecurableTypeServiceCredential)


How it will handle change from ISOLATED to OPEN?

bindings.AddCurrentWorkspaceBindings function handles that logic

alexott · 2024-11-14T08:22:08Z

docs/resources/credential.md

+
+A credential represents an authentication and authorization mechanism for accessing services on your cloud tenant. Each credential is subject to Unity Catalog access-control policies that control which users and groups can access the credential.
+
+To create credentials, you must be a Databricks account admin or have the `CREATE SERVICE CREDENTIAL` privilege. The user who creates the credential can delegate ownership to another user or group to manage permissions on it


@farrucosanjurjo-db wdyt?

docs/resources/credential.md

docs/resources/grant.md

madtrick · 2024-11-15T12:27:54Z

docs/resources/credential.md

+- `name` - Name of Credentials, which must be unique within the [databricks_metastore](metastore.md). Change forces creation of a new resource.
+- `purpose` - Indicates the purpose of the credential. Can be `SERVICE`.
+- `owner` - (Optional) Username/groupname/sp application_id of the credential owner.
+- `read_only` - (Optional) Indicates whether the credential is only usable for read operations.


Note this only applies to storage credentials

madtrick · 2024-11-15T12:29:04Z

docs/resources/credential.md

+
+- `name` - Name of Credentials, which must be unique within the [databricks_metastore](metastore.md). Change forces creation of a new resource.
+- `purpose` - Indicates the purpose of the credential. Can be `SERVICE`.
+- `owner` - (Optional) Username/groupname/sp application_id of the credential owner.


Afaik, the owner can't be set. It's derived from the identity of the user creating the credential

unless this also applies to updates

this applies to update - and in case of create, we execute an update as well

madtrick · 2024-11-15T12:29:15Z

docs/resources/credential.md

+The following arguments are required:
+
+- `name` - Name of Credentials, which must be unique within the [databricks_metastore](metastore.md). Change forces creation of a new resource.
+- `purpose` - Indicates the purpose of the credential. Can be `SERVICE`.


Suggested change

- `purpose` - Indicates the purpose of the credential. Can be `SERVICE`.

- `purpose` - Indicates the purpose of the credential. Can be `SERVICE` or `STORAGE`.

madtrick · 2024-11-15T12:31:50Z

docs/resources/credential.md

+- `skip_validation` - (Optional) Suppress validation errors if any & force save the credential.
+- `force_destroy` - (Optional) Delete credential regardless of its dependencies.
+- `force_update` - (Optional) Update credential regardless of its dependents.
+- `isolation_mode` - (Optional) Whether the credential is accessible from all workspaces or a specific set of workspaces. Can be `ISOLATION_MODE_ISOLATED` or `ISOLATION_MODE_OPEN`. Setting the credential to `ISOLATION_MODE_ISOLATED` will automatically allow access from the current workspace.


Suggested change

- `isolation_mode` - (Optional) Whether the credential is accessible from all workspaces or a specific set of workspaces. Can be `ISOLATION_MODE_ISOLATED` or `ISOLATION_MODE_OPEN`. Setting the credential to `ISOLATION_MODE_ISOLATED` will automatically allow access from the current workspace.

- `isolation_mode` - (Optional) Whether the credential is accessible from all workspaces or a specific set of workspaces. Can be `ISOLATION_MODE_ISOLATED` or `ISOLATION_MODE_OPEN`. Setting the credential to `ISOLATION_MODE_ISOLATED` will automatically restrict access to only from the current workspace.

When isolation_mode is changed, do we implicitly make a call to bind the credential to the current workspace?

yes, otherwise TF cannot manage the credential - I don't want to say "automatically restrict access to only from the current workspace", as customers will add additional bindings as necessary via the databricks_workspace_binding resource

madtrick · 2024-11-15T12:41:38Z

docs/resources/credential.md

@@ -0,0 +1,96 @@
+---


For storage credentials we also have some docs under data-sources/. Will you add those too?

https://github.com/databricks/terraform-provider-databricks/blob/2b381b07a2148ee4871e83ee321479da7a79cec2/docs/data-sources/storage_credential.md

https://github.com/databricks/terraform-provider-databricks/blob/2b381b07a2148ee4871e83ee321479da7a79cec2/docs/data-sources/storage_credentials.md

those are data sources, we need to implement them in a separate PR

github-actions · 2024-11-16T09:28:41Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

PR number: 4219
Commit SHA: 615b20e1bcdacc9fde45f7e27db0b4e4554d032f

Checks will be approved automatically on success.

eng-dev-ecosystem-bot · 2024-11-16T09:29:14Z

Test Details: go/deco-tests/11869025907

nkvuong requested review from a team as code owners November 13, 2024 08:31

nkvuong requested review from hectorcast-db and removed request for a team November 13, 2024 08:31

nkvuong temporarily deployed to test-trigger-is November 13, 2024 08:31 — with GitHub Actions Inactive

nkvuong temporarily deployed to test-trigger-is November 13, 2024 08:40 — with GitHub Actions Inactive

alexott requested a review from farrucosanjurjo-db November 13, 2024 08:42

nkvuong temporarily deployed to test-trigger-is November 13, 2024 08:49 — with GitHub Actions Inactive

alexott reviewed Nov 13, 2024

View reviewed changes

nkvuong temporarily deployed to test-trigger-is November 13, 2024 15:52 — with GitHub Actions Inactive

nkvuong temporarily deployed to test-trigger-is November 13, 2024 15:53 — with GitHub Actions Inactive

nkvuong temporarily deployed to test-trigger-is November 14, 2024 07:04 — with GitHub Actions Inactive

alexott reviewed Nov 14, 2024

View reviewed changes

nkvuong temporarily deployed to test-trigger-is November 14, 2024 09:46 — with GitHub Actions Inactive

nkvuong temporarily deployed to test-trigger-is November 14, 2024 09:47 — with GitHub Actions Inactive

nkvuong force-pushed the feature/credential branch from 4519765 to b8b2e2a Compare November 15, 2024 11:59

nkvuong temporarily deployed to test-trigger-is November 15, 2024 11:59 — with GitHub Actions Inactive

nkvuong requested a review from alexott November 15, 2024 11:59

madtrick reviewed Nov 15, 2024

View reviewed changes

nkvuong force-pushed the feature/credential branch from b8b2e2a to 56ac60c Compare November 16, 2024 02:51

nkvuong temporarily deployed to test-trigger-is November 16, 2024 02:51 — with GitHub Actions Inactive

nkvuong added 7 commits November 16, 2024 17:28

WIP

de8d27b

fix test

4958aff

update grant docs

a6a41f3

feedback

dd7fc85

fix test again

eb8ff28

more fix

fad4bae

update doc

615b20e

nkvuong force-pushed the feature/credential branch from 56ac60c to 615b20e Compare November 16, 2024 09:28

nkvuong temporarily deployed to test-trigger-is November 16, 2024 09:28 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add `databricks_credential` resource #4219

[Feature] Add `databricks_credential` resource #4219

nkvuong commented Nov 13, 2024 •

edited

Loading

alexott Nov 13, 2024

alexott Nov 13, 2024

nkvuong Nov 13, 2024

nkvuong Nov 13, 2024 •

edited

Loading

alexott Nov 14, 2024

madtrick Nov 15, 2024

madtrick Nov 15, 2024

alexott commented Nov 13, 2024

alexott Nov 14, 2024

nkvuong Nov 14, 2024

alexott Nov 14, 2024

madtrick Nov 15, 2024

madtrick Nov 15, 2024

madtrick Nov 15, 2024

nkvuong Nov 15, 2024

madtrick Nov 15, 2024

madtrick Nov 15, 2024

nkvuong Nov 15, 2024

madtrick Nov 15, 2024

nkvuong Nov 15, 2024

github-actions bot commented Nov 16, 2024

eng-dev-ecosystem-bot commented Nov 16, 2024


		A credential represents an authentication and authorization mechanism for accessing services on your cloud tenant. Each credential is subject to Unity Catalog access-control policies that control which users and groups can access the credential.

		To create credentials, you must be a Databricks account admin or have the `CREATE SERVICE CREDENTIAL` privilege. The user who creates the credential can delegate ownership to another user or group to manage permissions on it

	- `purpose` - Indicates the purpose of the credential. Can be `SERVICE`.
	- `purpose` - Indicates the purpose of the credential. Can be `SERVICE` or `STORAGE`.

	- `isolation_mode` - (Optional) Whether the credential is accessible from all workspaces or a specific set of workspaces. Can be `ISOLATION_MODE_ISOLATED` or `ISOLATION_MODE_OPEN`. Setting the credential to `ISOLATION_MODE_ISOLATED` will automatically allow access from the current workspace.
	- `isolation_mode` - (Optional) Whether the credential is accessible from all workspaces or a specific set of workspaces. Can be `ISOLATION_MODE_ISOLATED` or `ISOLATION_MODE_OPEN`. Setting the credential to `ISOLATION_MODE_ISOLATED` will automatically restrict access to only from the current workspace.

[Feature] Add databricks_credential resource #4219

Are you sure you want to change the base?

[Feature] Add databricks_credential resource #4219

Conversation

nkvuong commented Nov 13, 2024 • edited Loading

Changes

Tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nkvuong Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexott commented Nov 13, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 16, 2024

eng-dev-ecosystem-bot commented Nov 16, 2024

[Feature] Add `databricks_credential` resource #4219

[Feature] Add `databricks_credential` resource #4219

nkvuong commented Nov 13, 2024 •

edited

Loading

nkvuong Nov 13, 2024 •

edited

Loading