Add integration tests workflow for self hosted

.github/workflows/integration-self-hosted.yml

@@ -0,0 +1,43 @@
+name: self-hosted-integration
+on:
+  pull_request:
+    paths:
+      - pkg/**
+      - integration/self_hosted/**
+      - main.go
+      - go.mod
+  schedule:
+    - cron: 1 11 * * 0
+
+jobs:
+  integration:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: /usr/src/app/integration/self_hosted
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Docker Compose Up
+        working-directory: /usr/src/app
+        run: docker compose up --build -d materialized materialized_init provider redpanda mysql postgres localstack
+
+      - name: Terraform Init
+        run: |
+          docker exec provider terraform init
+
+      - name: Terraform Apply
+        run: |
+          docker exec provider terraform apply -auto-approve -compact-warnings
+
+      - name: Terraform Plan
+        run: |
+          docker exec provider terraform plan -detailed-exitcode
+
+      - name: Terraform Destroy
+        run: |
+          docker exec provider terraform destroy -auto-approve -compact-warnings
+
+      - name: Docker Compose Down
+        working-directory: /usr/src/app
+        run: docker compose down

CONTRIBUTING.md

@@ -61,14 +61,20 @@ make testacc
 To run the full integration project, set the necessary env variables and start the docker compose similar to the acceptance tests. Then to interact with the provider you can run:
 
 ```bash
-# Run the tests
+# SaaS tests
 docker exec provider terraform init
 docker exec provider terraform apply -auto-approve -compact-warnings
 docker exec provider terraform plan -detailed-exitcode
 docker exec provider terraform destroy -auto-approve -compact-warnings
+
+# Self-hosted tests
+docker exec provider sh -c "cd /usr/src/app/integration/self_hosted && terraform init"
+docker exec provider sh -c "cd /usr/src/app/integration/self_hosted && terraform apply -auto-approve -compact-warnings"
+docker exec provider sh -c "cd /usr/src/app/integration/self_hosted && terraform plan -detailed-exitcode"
+docker exec provider sh -c "cd /usr/src/app/integration/self_hosted && terraform destroy -auto-approve -compact-warnings"
 ```
 
-> Note: You might have to delete the `integration/.terraform`, `integration/.terraform.lock.hcl` and `integration/terraform.tfstate*` files before running the tests.
+> Note: You might have to delete the `integration/.terraform`, `integration/.terraform.lock.hcl` and `integration/terraform.tfstate*` files before running the tests. Or if you are running the self-hosted tests, you will have to delete the `integration/self_hosted/.terraform`, `integration/self_hosted/.terraform.lock.hcl` and `integration/self_hosted/terraform.tfstate*` files.
 
 ### Debugging
 Terraform has detailed logs that you can enable by setting the `TF_LOG` environment variable to any value. Enabling this setting causes detailed logs to appear on `stderr`.

DESIGN.md

@@ -58,4 +58,4 @@ Being the most lightweight the unit tests should cover the wide of SQL variation
 
 **Acceptance tests** use the Terraform `acctest` package to execute Terraform commands in a certain order. These tests are used to ensure the applys, updates and destroys work as expected for each resource. These tests should not cover every SQL permutation but ensure that high level Terraform commands execute against the Materialize image. These tests rely on the docker compose.
 
-**Integration tests** consist of an entire Terraform package in the `integration` directory. This will spin up a docker compose using the `materialized` and surrounding kafka and database dependencies. All resources are applied and destroyed as part of the same terraform project. Any new resources or permutations should be added to the integration tests.
+**Integration tests** consist of an entire Terraform package in the `integration` directory. This will spin up a docker compose using the `materialized` and surrounding kafka and database dependencies. All resources are applied and destroyed as part of the same terraform project. Any new resources or permutations should be added to the integration tests.