Cloud Build Application CICD examples (#1192)

* Containerize an application on new commit * Deploy the latest application image to Cloud Run upon creation Co-authored-by: Andrew Gold <[email protected]>
GoogleCloudPlatform · Jan 9, 2024 · be94e6f · be94e6f
1 parent 0c6fbe9
commit be94e6f
Show file tree

Hide file tree

Showing 12 changed files with 443 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -112,8 +112,11 @@ them to fit your particular use case.
 *   [Carbon Footprint Reporting](examples/carbon-foortprint-dashboard) - Example of 
     using the prebuilt Data studio & Looker template for analysing GCP Carbon Footprint Estimates.
 *   [Cloud Audit Log Samples](examples/audit-log-examples/) - A sample
-*   collection of Audit Logs for Users and Customers to better the structure,
+    collection of Audit Logs for Users and Customers to better the structure,
     contents, and values contained in various log events.
+*   [Cloud Build Application CICD Examples](examples/cloudbuild-application-cicd) - 
+    Cloud Build CI/CD Examples for Applications like containerization &
+    deployment to Cloud Run.
 *   [Cloud Build with Proxy Running in Background](examples/cloudbuild-with-tcp-proxy) -
     Examples of cloudbuild with docker-compose running tcp proxy in the
     background for all build steps.

diff --git a/examples/cloudbuild-application-cicd/README.md b/examples/cloudbuild-application-cicd/README.md
@@ -0,0 +1,42 @@
+This is a collection of Cloud Build Examples for common CICD Tasks to be used
+for applications where their repository is connected to Google Cloud Build.
+
+These `.yaml` examples can be copied to the root directory of applications where
+their traditional Dockerfiles are or you can choose to add them to a
+`cloudbuild/`, `.cloudbuild/`, or `.ci/` directory.
+
+Each example has a terraform Cloud Build Trigger Example to be used for any IAC
+Setup.
+
+## Containerize:
+
+This Example allows you to containerize an application and push to a specific
+Artifact Registry Docker repository. Cloud Build will containerize the
+application based on the `Dockerfile` defined within the repository used.
+
+## Deploy to Cloud Run
+
+This example allows you to deploy your application to Cloud Run when a new
+container image is pushed to Artifact Registry.
+
+This trigger is dependant on the Containerization Cloud Build triggers of the
+application. After a Containerization trigger completes, it should push the new
+container image as `latest` to Artifact Registry within the project. This
+makes uses of the `gcr`
+[pubsub Topic](https://cloud.google.com/artifact-registry/docs/configure-notifications#overview)
+Setup or create the gcr topic either manually or with the below terraform code:
+
+```
+# Create Pupsub Topic that captures Artifact Registry Events.
+# https://cloud.google.com/artifact-registry/docs/configure-notifications#overview
+resource "google_pubsub_topic" "cicd_ar_image_pub" {
+  project = var.project_cicd
+  name    = "gcr"
+}
+```
+
+Since the Containerization build is done within cloud build, then this Cloud
+Build Trigger will be able to pick up that message using the filter provided.
+
+Once the pub/sub message is received for the application's image
+containerization, then this trigger will kick off a Cloud Run Deployment.
diff --git a/examples/cloudbuild-application-cicd/containerize/README.md b/examples/cloudbuild-application-cicd/containerize/README.md
@@ -0,0 +1,34 @@
+# Universal Application Containerizer with Push to Artifact Registry
+
+This Example allows you to containerize an application and push to a specific
+Artifact Registry Docker Repository. Cloud build will containerize the
+application based on the `Dockerfile` defined within the repository used.
+
+Two image tags will be created.
+
+1. The `latest` tag.
+2. The SHORT_SHA tag. This is the shortform of the commit ID. This will be the long term tag on a specific image version.
+
+## Getting started
+
+Copy the `containerize.yaml` file into your application's git repository.
+Modify the substitution default values as needed.
+
+## Using the Terraform Example:
+
+This Terraform is an example Cloud Build Trigger that allows you to containerize
+the latests version of your application and then push the new image to Artifact
+Registry.
+
+Instead of using this as-is, you will want to copy a version of this into your
+existing terraform configurations for your CICD pipelines and applications.
+
+## Terraform Variables
+
+| Name                | Description                                                            | Type     |
+| ------------------- | ---------------------------------------------------------------------- | -------- |
+| cicd_project_id     | The name of the CICD GCP Project to deply the Cloud Build Triggers to. | `string` |
+| region              | The root region to deploy the Cloud Build Triggers to.                 | `string` |
+| cicd_ar_docker_name | CICD Project Artifact Registry Docker Repo Name.                       | `string` |
+| cicd_ar_docker_loc  | CICD Project Artifact Registry Docker Repo Location.                   | `string` |
+| application_name    | The name of the application to containerize.                           | `string` |
diff --git a/examples/cloudbuild-application-cicd/containerize/cloud_build_trigger.tf b/examples/cloudbuild-application-cicd/containerize/cloud_build_trigger.tf
@@ -0,0 +1,42 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+resource "google_cloudbuild_trigger" "containerize_app" {
+  # https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloudbuild_trigger
+  project     = data.google_project.cicd.name
+  name        = "containerize-${var.application_name}"
+  description = "Containerize ${var.application_name} latest"
+  tags        = ["app", "build"]
+  location    = var.region
+
+  substitutions = {
+    _AR_LOC      = var.cicd_ar_docker_loc
+    _AR_REPO     = var.cicd_ar_docker_name
+    _APPLICATION = var.application_name
+  }
+
+  # Github Repo example
+  source_to_build {
+    uri       = "https://github-user-org-name/repository-name"
+    ref       = "refs/heads/main"
+    repo_type = "GITHUB"
+  }
+
+  git_file_source {
+    path      = "cloudbuild/containerize.yaml"
+    uri       = "https://github-user-org-name/repository-name"
+    revision  = "refs/heads/main"
+    repo_type = "GITHUB"
+  }
+}
diff --git a/examples/cloudbuild-application-cicd/containerize/containerize.yaml b/examples/cloudbuild-application-cicd/containerize/containerize.yaml
@@ -0,0 +1,36 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+substitutions:
+  _AR_LOC: us-central1        # Artifact Registry location
+  _AR_REPO: docker            # Artifact Registry repository name (must already exist)
+  _APPLICATION: application   # Application name
+
+
+steps:
+  - id: 'build docker image'
+    name: 'gcr.io/cloud-builders/docker'
+    args: [
+      'build',
+      '-t', '${_AR_LOC}-docker.pkg.dev/$PROJECT_ID/${_AR_REPO}/${_APPLICATION}:$SHORT_SHA',
+      '-t', '${_AR_LOC}-docker.pkg.dev/$PROJECT_ID/${_AR_REPO}/${_APPLICATION}:latest',
+      '.'
+    ]
+
+images:
+  - '${_AR_LOC}-docker.pkg.dev/$PROJECT_ID/${_AR_REPO}/${_APPLICATION}:latest'
+  - '${_AR_LOC}-docker.pkg.dev/$PROJECT_ID/${_AR_REPO}/${_APPLICATION}:$SHORT_SHA'
+
+# Cloud Build logs need to be defined for builds using a GCP service account
+logsBucket: gs://$PROJECT_ID-cloudbuild-logs
diff --git a/examples/cloudbuild-application-cicd/containerize/main.tf b/examples/cloudbuild-application-cicd/containerize/main.tf
@@ -0,0 +1,21 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+provider "google" {
+  project = var.cicd_project_id
+}
+
+data "google_project" "cicd" {
+  project_id = var.cicd_project_id
+}
diff --git a/examples/cloudbuild-application-cicd/containerize/variables.tf b/examples/cloudbuild-application-cicd/containerize/variables.tf
@@ -0,0 +1,40 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+variable "cicd_project_id" {
+  description = "The name of the CICD GCP Project to deply the Cloud Build Triggers to."
+  type        = string
+}
+
+variable "region" {
+  description = "The root region to deploy the Cloud Build Triggers to."
+  type        = string
+}
+
+variable "cicd_ar_docker_name" {
+  description = "CICD Project Artifact Registry Docker Repo Name."
+  type        = string
+  default     = "docker"
+}
+
+variable "cicd_ar_docker_loc" {
+  description = "CICD Project Artifact Registry Docker Repo Location."
+  type        = string
+  default     = "us-central1"
+}
+
+variable "application_name" {
+  description = "The name of the application to containerize."
+  type        = string
+}
diff --git a/examples/cloudbuild-application-cicd/deploy_to_cloud_run/README.md b/examples/cloudbuild-application-cicd/deploy_to_cloud_run/README.md
@@ -0,0 +1,24 @@
+# Getting started
+
+Copy the `deploy_cloud_run.yaml` file into your application's git repository.
+Modify the substitution default values as needed.
+
+# Using the Terraform Example:
+
+This Terraform is an example Cloud Build Triggers and some example dependencies
+that allows you to deploy the latests version of your application to Cloud Run.
+
+Instead of using this as-is, you will want to copy a version of this into your
+existing terraform configurations for your CICD pipelines and applications.
+
+## Terraform Variables
+
+| Name                | Description                                                     | Type     |
+| ------------------- | --------------------------------------------------------------- | -------- |
+| project_id          | The ID of the GCP Project to deply the Cloud Build Triggers to. | `string` |
+| cicd_project_id     | The name of the CICD Project to connect to GCR AR Topic.        | `string` |
+| region              | The root region to deploy the application to.                   | `string` |
+| production          | Production Environment (Hide API Swaggers).                     | `bool`   |
+| cicd_ar_docker_name | CICD Project Artifact Registry Docker Repo Name.                | `string` |
+| cicd_ar_docker_loc  | CICD Project Artifact Registry Docker Repo Location.            | `string` |
+| application_name    | The name of the application to deploy.                          | `string` |
diff --git a/examples/cloudbuild-application-cicd/deploy_to_cloud_run/cloud_build_trigger.tf b/examples/cloudbuild-application-cicd/deploy_to_cloud_run/cloud_build_trigger.tf
@@ -0,0 +1,75 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Artifact Registries GCR
+data "google_pubsub_topic" "cicd_ar_image_pub" {
+  project = var.cicd_project_id
+  name    = "gcr"
+}
+
+# This is a Cloud Build trigger that will deploy the latest version of the
+# PUP Backend app to the Environment.
+resource "google_cloudbuild_trigger" "deploy_app" {
+  # https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloudbuild_trigger
+  project     = data.google_project.cicd.name
+  name        = "app-deploy-to-cloud-run"
+  description = "Deploy Application latest to Cloud Run within Region ${var.region}."
+  tags        = ["app", "deploy", "cloud-run"]
+  location    = var.region
+
+  substitutions = {
+    _ACTION                = "$(body.message.data.action)"
+    _IMAGE_TAG             = "$(body.message.data.tag)"
+    _CICD_PROJECT          = var.cicd_project_id
+    _DEPLOY_PROJECT        = var.project_id
+    _DEPLOY_REGION         = var.region
+    _AR_LOC                = var.cicd_ar_docker_loc
+    _AR_REPO               = var.cicd_ar_docker_name
+    _APPLICATION           = var.application_name
+    _APP_IMAGE_NAME        = var.application_name
+    _SWAGGER               = var.production ? "false" : "true"
+    _SERVICE_ACCOUNT_EMAIL = "${var.application_name}-sa@${var.project_id}.iam.gserviceaccount.com"
+    _LOG_LEVEL             = var.production ? "info" : "debug"
+  }
+
+  # This filter allows this cloud build trigger to only initiate when a new
+  # image is pushed to the Artifact Registry repository for the PUP Backend app.
+  filter = <<EOT
+  _ACTION.matches('INSERT') &&
+  _IMAGE_TAG.matches(
+    '${var.cicd_ar_docker_loc}' +
+    '-docker.pkg.dev/' +
+    '${data.google_project.cicd.name}/' +
+    '${var.cicd_ar_docker_name}/' +
+    '${var.application_name}' +
+    ':latest'
+    )
+  EOT
+
+  pubsub_config {
+    topic = data.google_pubsub_topic.cicd_ar_image_pub.id
+  }
+
+  source_to_build {
+    repo_type = "CLOUD_SOURCE_REPOSITORIES"
+    ref       = "refs/heads/main"
+    uri       = "https://source.developers.google.com/p/${data.google_project.cicd.name}/r/${var.application_name}"
+  }
+
+  git_file_source {
+    path      = "cloudbuild/deploy_cloud_run.yaml"
+    repo_type = "CLOUD_SOURCE_REPOSITORIES"
+    revision  = "refs/heads/main"
+  }
+}
diff --git a/examples/cloudbuild-application-cicd/deploy_to_cloud_run/deploy_cloud_run.yaml b/examples/cloudbuild-application-cicd/deploy_to_cloud_run/deploy_cloud_run.yaml
@@ -0,0 +1,49 @@
+# Copyright 2023 Google LLC
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+substitutions:
+  _CICD_PROJECT: my-cicd-project-id     # GCP Project ID Containing Docker App Images (Must be defined as a Cloud Build Substitution)
+  _DEPLOY_PROJECT: my-deploy-project-id # GCP Project ID to deploy app to (Must be defined as a Cloud Build Substitution)
+  _DEPLOY_REGION: us-central1           # GCP Region to deploy app to
+  _AR_LOC: us-central1                  # Artifact Registry location
+  _AR_REPO: docker                      # Artifact Registry repository name (must already exist)
+  _APPLICATION: application             # Application name
+  _APP_IMAGE_NAME: application          # Application image name
+  _APP_IMAGE_VERSION: latest            # Application image tag/version
+  _SWAGGER: 'true'                      # Enable Swagger UI
+  _LOG_LEVEL: 'info'                    # Log level
+  _SERVICE_ACCOUNT_EMAIL: [email protected]
+
+
+steps:
+  - id: 'push to cloud run'
+    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk:slim'
+    entrypoint: 'gcloud'
+    args: [
+      '--project', '${_DEPLOY_PROJECT}',
+      'run', 'deploy', '${_APPLICATION}',
+      '--service-account', '${_SERVICE_ACCOUNT_EMAIL}',
+      '--image',
+      '${_AR_LOC}-docker.pkg.dev/${_CICD_PROJECT}/${_AR_REPO}/${_APP_IMAGE_NAME}:${_APP_IMAGE_VERSION}',
+      '--platform', 'managed',
+      '--region', '${_DEPLOY_REGION}',
+      '--ingress', 'internal-and-cloud-load-balancing',
+      '--vpc-egress', 'all-traffic',
+      '--set-env-vars','SWAGGER=${_SWAGGER}',
+      '--set-env-vars','LOG_LEVEL=${_LOG_LEVEL}',
+      '--allow-unauthenticated'
+    ]
+
+# Cloud Build logs need to be defined for builds using a GCP service account
+logsBucket: gs://$_CICD_PROJECT-cloudbuild-logs