release v0.4.61 from PR #814

release v0.4.61 from PR #814

CHANGELOG.md

@@ -7,6 +7,11 @@ Changes to be including in future/planned release notes will be added here.
 
 ## Next
 
+## [0.4.61](https://github.com/Worklytics/psoxy/release/tag/v0.4.61)
+ - added some `columnsToPseudonymizeIfPresent` to survey bulk connectors; these are to avoid PII
+   being sent to Worklytics if these unexpected columns sent, but without errors in usual case, when
+   these are omitted.
+
 ## [0.4.60](https://github.com/Worklytics/psoxy/release/tag/v0.4.60)
  - Calendar rules change to allow OOO-related snippets to be passed through event title fields;
     this is required for proper OOO-analysis in Worklytics Calendar 3.0 methodology.
@@ -360,4 +365,4 @@ Upgrade Notes:
     - eg, `PSOXY_SHARED` and `PSOXY_GCAL`, to allow IAM policies such as "read `PSOXY_SHARED*`" and
       "read+write `PSOXY_GCAL*`" (if shared secrets have common prefix with connector secrets,
       then wildcard policy to read shared also grants read of secrets across all connectors)
-- keys/salts per value kind (PII, item id, etc)
+- keys/salts per value kind (PII, item id, etc)

docs/SUMMARY.md

@@ -37,6 +37,7 @@
   * [API Data Sanitization](configuration/api-data-sanitization.md)
   * [Bulk File Sanitization](configuration/bulk-file-sanitization.md)
   * [JSON Filter](configuration/json-filter.md)
+  * [TLS Version](configuration/tls.md)
 * [Development](development/README.md)
   * [Approaches for Example / Module design](development/terraform-architecture.md)
   * [Create a private fork](development/private-fork.md)
@@ -49,6 +50,9 @@
     * [New Relic Monitoring](development/alpha-features/new-relic-monitoring.md)
 * [Data Sources](sources/README.md)
   * [Asana](sources/asana/README.md)
+  * [Atlassian](sources/atlassian/README.md)
+    * [Jira Cloud](sources/atlassian/jira/README.md)
+    * [Jira Data Center](sources/atlassian/jira/jira-server.md)
   * [Dropbox](sources/dropbox-business/README.md)
   * [GitHub](sources/github/README.md)
   * [Google Workspace](sources/google-workspace/README.md)
@@ -71,9 +75,7 @@
     * [API Call Examples](sources/slack/example-api-calls.md)
   * [Zoom](sources/zoom/README.md)
     * [API Call Examples](sources/zoom/example-api-calls.md)
-  * [Atlassian](sources/atlassian/README.md)
-    * [Jira Cloud](sources/atlassian/jira/README.md)
-    * [Jira Data Center](sources/atlassian/jira/jira-server.md)
+
 * [FAQ](faq-security.md)
 * [Release Notes](https://github.com/Worklytics/psoxy/releases)
 

docs/aws/getting-started.md

@@ -63,6 +63,12 @@ The diagram below provides an architecture overview of the 'API' and 'Bulk' mode
    in place that deny this by default, even if you have an IAM role that allows it at an account
    level.
 
+   NOTE: using AWS API Gateway, VPC, or Secrets Manager (not used by default in our examples) will
+   require additional permissions beyond the above.
+
+   See: [protips.md](protips.md) for guide to create a least-privileged iam policy for provisioning.
+
+
 3. **An authenticated AWS CLI in your provisioning environment**. Your environment (eg, shell/etc
    from which you'll run terraform commands) must be authenticated as an identity that can assume
    that role. (see next section for tips on options for various environments you can use)

docs/aws/protips.md

@@ -142,3 +142,27 @@ For organizations that don't allow use of AWS Managed Policies, you can use the
 `aws_lambda_execution_role_policy_arn` variable to pass in an alternative which will be used INSTEAD
 of the AWS Managed Policy.
 
+## Least-Privileged IAM Policy for Provisioning
+
+YMMV, but we exposed a minimal IAM policy for provisioning in the `psoxy-constants` module, which
+you attach to your desired role to ensure it has sufficient permissions to provision the proxy.
+
+NOTE: using features beyond the default set, such as AWS API Gateway, VPC, or Secrets Manager, may
+require some additional permissions beyond what is provided in the least-privileged policy.
+
+```hcl
+module "psoxy_constants" {
+  source = "git::https://github.com/worklytics/psoxy//infra/modules/psoxy-constants?ref=v0.4.61"
+}
+
+resource "aws_iam_policy" "min_provisioner_policy" {
+    name   = "PsoxyMinProvisioner"
+    policy = module.psoxy_constants.aws_least_privileged_policy
+}
+
+resource "aws_iam_role_policy_attachment" "min_provisioner_policy" {
+    policy_arn         = aws_iam_policy.min_provisioner_policy.arn
+    role               = "{{NAME_OF_YOUR_AWS_PROVISIONER_ROLE}}"
+}
+```
+

docs/configuration/tls.md

@@ -0,0 +1,30 @@
+# Configuring Transport Layer Security (TLS)
+
+By default, proxy from version 0.4.61 will connect to data source APIs using TLS 1.3.
+
+Prior to 0.4.61, the proxy should have negotiated to use 1.3 with all sources that supported it;
+but may have fallen back to 1.2 for some sources.
+
+It will no longer fall back; but you can configure the proxy to use TLS 1.2 for a given source by
+setting the `TLS_VERSION` environment variable on a proxy instance to `TLSv1.2`. As TLS 1.3 offers
+security and performance improvements, we recommend using it whenever possible.
+
+As of Sept 2024, we've confirmed that the following public APIs of various data sources support
+TLS 1.3, either through end-to-end proxy testing OR via openssl negotiation (see next section):
+  - Google Workspace
+  - Microsoft 365 (Microsoft Graph)
+  - GitHub (cloud version)
+  - Asana
+  - Atlassian (JIRA, etc)
+  - Slack
+  - Zoom
+
+## Testing TLS 1.3 Support for a Source API
+
+To test TLS 1.3 support, you can use something like the following command (assuming you have
+`openssl` installed on  a Mac):
+
+```shell
+openssl s_client -connect api.asana.com:443 -tls1_3
+```
+

docs/faq-security.md

@@ -115,3 +115,15 @@ researchers complain about.
 If you remain uncomfortable with DWD, a private Google Marketplace App is a possible alternative,
 albeit more tedious to configure. It requires a dedicated GCP project, with additional APIs enabled
 in the project.
+
+
+### Is this Attribute-based Access Control (ABAC)?
+No. [ABAC](https://en.wikipedia.org/wiki/Attribute-based_access_control) is specifying an access
+control policy predicated on attributes of the object/resource being accessed. The approach of Psoxy
+is better described as Attribute-level Access Control, where the access control policy can be
+written to limit access to specific attritibutes (fields) within an object/resource.
+
+Eg, evaluation of an ABAC policy still results in boolean, allow/deny decision on the request; Psoxy
+policy (rule) evaluation results in a modified response, with specific fields redacted or
+transformed in accordance with the policy.
+

docs/sources/microsoft-365/README.md

@@ -64,6 +64,11 @@ proxy host platform)
 
 ![Microsoft Workload Identity Federation Scenarios](msft-workload-identity-federation-scenarios.png)
 
+The video below explains the general idea for identity federation for Azure AD-gated resources more
+generally, of which your Graph API is an example:
+{% embed url="https://www.youtube.com/watch?v=WIs3IRCJhEo" %}
+
+
 ### Authorization and Scopes
 
 The following Scopes are required for each connector. Note that they are all READ-only scopes.
@@ -138,4 +143,4 @@ this:
 
 Just running `terraform apply` again (and maybe again) usually fixes it. Likely it's something with
 with Azure's clock relative to your machine, plus whatever flight time is required between cert
-generation and it being PUT to Azure.
+generation and it being PUT to Azure.

infra/examples-dev/aws-all/api-gateway-advanced.tf

@@ -0,0 +1,45 @@
+# example of setting up a custom domain for an API Gateway V2 via composition with Psoxy
+# this example is provided for informational purposes only, with no warranty; please refer to AWS
+# and Terraform documentation for the most up-to-date information
+
+# resource "aws_acm_certificate" "cert" {
+#   domain_name       = "example.com"
+#   validation_method = "DNS"
+#
+#   lifecycle {
+#     create_before_destroy = true
+#   }
+# }
+#
+# resource "aws_apigatewayv2_domain_name" "example" {
+#   domain_name = "ws-api.example.com"
+#
+#   domain_name_configuration {
+#     certificate_arn = aws_acm_certificate.cert.arn
+#     endpoint_type   = "REGIONAL"
+#     security_policy = "TLS_1_2" # this is a 'min version'; 'TLS_1_2' allows TLS v1.2 or TLS v1.3 in practice; see https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-custom-domain-tls-version.html
+#   }
+# }
+#
+# # see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/apigatewayv2_api_mapping
+# resource "aws_apigatewayv2_api_mapping" "example" {
+#   api_id      = module.psoxy.api_gateway_v2.id
+#   domain_name = aws_apigatewayv2_domain_name.example.id
+#   stage       = module.psoxy.api_gateway_v2_stage.id
+# }
+#
+# resource "aws_route53_zone" "main" {
+#   name = "example.com"
+# }
+#
+# resource "aws_route53_record" "main" {
+#   name    = aws_apigatewayv2_domain_name.example.domain_name
+#   type    = "A"
+#   zone_id = aws_route53_zone.main.zone_id
+#
+#   alias {
+#     name                   = aws_apigatewayv2_domain_name.example.domain_name_configuration[0].target_domain_name
+#     zone_id                = aws_apigatewayv2_domain_name.example.domain_name_configuration[0].hosted_zone_id
+#     evaluate_target_health = false
+#   }
+# }

infra/examples-dev/aws-all/google-workspace.tf

@@ -8,7 +8,7 @@ provider "google" {
 
 module "worklytics_connectors_google_workspace" {
   source = "../../modules/worklytics-connectors-google-workspace"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-google-workspace?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-google-workspace?ref=v0.4.61"
 
   providers = {
     google = google.google_workspace

infra/examples-dev/aws-all/main.tf

@@ -21,7 +21,7 @@ terraform {
 # general cases
 module "worklytics_connectors" {
   source = "../../modules/worklytics-connectors"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors?ref=v0.4.61"
 
   enabled_connectors               = var.enabled_connectors
   jira_cloud_id                    = var.jira_cloud_id
@@ -101,7 +101,7 @@ locals {
 
 module "psoxy" {
   source = "../../modules/aws-host"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-host?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-host?ref=v0.4.61"
 
   environment_name                     = var.environment_name
   aws_account_id                       = var.aws_account_id
@@ -159,7 +159,7 @@ module "connection_in_worklytics" {
   for_each = local.all_instances
 
   source = "../../modules/worklytics-psoxy-connection-aws"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-psoxy-connection-aws?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-psoxy-connection-aws?ref=v0.4.61"
 
   psoxy_instance_id    = each.key
   worklytics_host      = var.worklytics_host

infra/examples-dev/aws-all/msft-365.tf

@@ -2,7 +2,7 @@
 
 module "worklytics_connectors_msft_365" {
   source = "../../modules/worklytics-connectors-msft-365"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-msft-365?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-msft-365?ref=v0.4.61"
 
   enabled_connectors                         = var.enabled_connectors
   environment_id                             = var.environment_name
@@ -48,7 +48,7 @@ module "cognito_identity_pool" {
   count = local.msft_365_enabled ? 1 : 0 # only provision identity pool if MSFT-365 connectors are enabled
 
   source = "../../modules/aws-cognito-pool"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-cognito-pool?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-cognito-pool?ref=v0.4.61"
 
   developer_provider_name = local.developer_provider_name
   name                    = "${local.env_qualifier}-azure-ad-federation"
@@ -71,7 +71,7 @@ module "cognito_identity" {
   count = local.msft_365_enabled ? 1 : 0 # only provision identity pool if MSFT-365 connectors are enabled
 
   source = "../../modules/aws-cognito-identity-cli"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-cognito-identity-cli?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/aws-cognito-identity-cli?ref=v0.4.61"
 
   aws_region       = data.aws_region.current.id
   aws_role         = var.aws_assume_role_arn
@@ -108,7 +108,7 @@ module "msft_connection_auth_federation" {
   for_each = local.provision_entraid_apps ? local.enabled_to_entraid_object : local.shared_to_entraid_object
 
   source = "../../modules/azuread-federated-credentials"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/azuread-federated-credentials?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/azuread-federated-credentials?ref=v0.4.61"
 
   application_object_id = each.value.connector_id
   display_name          = "${local.env_qualifier}AccessFromAWS"
@@ -136,4 +136,4 @@ locals {
 output "msft_365_api_clients" {
   description = "Map of API client identifiers. Useful for configuration of clients, terraform migration."
   value       = module.worklytics_connectors_msft_365.api_clients
-}
+}

infra/examples-dev/aws-google-workspace/main.tf

@@ -59,7 +59,7 @@ data "google_project" "psoxy-google-connectors" {
 
 module "psoxy" {
   source = "../../modular-examples/aws-google-workspace"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-google-workspace?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-google-workspace?ref=v0.4.61"
 
   aws_account_id                 = var.aws_account_id
   aws_assume_role_arn            = var.aws_assume_role_arn # role that can test the instances (lambdas)

infra/examples-dev/aws-msft-365/main.tf

@@ -51,7 +51,7 @@ provider "azuread" {
 
 module "psoxy" {
   source = "../../modular-examples/aws-msft-365"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-msft-365?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-msft-365?ref=v0.4.61"
 
   aws_account_id                 = var.aws_account_id
   aws_assume_role_arn            = var.aws_assume_role_arn # role that can test the instances (lambdas)

infra/examples-dev/aws/main.tf

@@ -57,7 +57,7 @@ provider "azuread" {
 
 module "psoxy" {
   source = "../../modular-examples/aws"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws?ref=v0.4.61"
 
   aws_account_id                 = var.aws_account_id
   aws_assume_role_arn            = var.aws_assume_role_arn # role that can test the instances (lambdas)

infra/examples-dev/gcp-google-workspace/main.tf

@@ -28,7 +28,7 @@ provider "google" {
 
 module "psoxy" {
   source = "../../modular-examples/gcp-google-workspace"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/gcp-google-workspace?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/gcp-google-workspace?ref=v0.4.61"
 
   gcp_project_id                 = var.gcp_project_id
   environment_name               = var.environment_name

infra/examples-dev/gcp/google-workspace.tf

@@ -8,7 +8,7 @@ provider "google" {
 
 module "worklytics_connectors_google_workspace" {
   source = "../../modules/worklytics-connectors-google-workspace"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-google-workspace?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-google-workspace?ref=v0.4.61"
 
   providers = {
     google = google.google_workspace

infra/examples-dev/gcp/main.tf

@@ -29,7 +29,7 @@ locals {
 # call this 'generic_source_connectors'?
 module "worklytics_connectors" {
   source = "../../modules/worklytics-connectors"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors?ref=v0.4.61"
 
 
   enabled_connectors               = var.enabled_connectors
@@ -81,7 +81,7 @@ locals {
 
 module "psoxy" {
   source = "../../modules/gcp-host"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/gcp-host?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/gcp-host?ref=v0.4.61"
 
   gcp_project_id                    = var.gcp_project_id
   environment_name                  = var.environment_name
@@ -121,7 +121,7 @@ module "connection_in_worklytics" {
   for_each = local.all_instances
 
   source = "../../modules/worklytics-psoxy-connection-generic"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-psoxy-connection-generic?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-psoxy-connection-generic?ref=v0.4.61"
 
   psoxy_host_platform_id = local.host_platform_id
   psoxy_instance_id      = each.key

infra/examples-dev/gcp/msft-365.tf

@@ -2,7 +2,7 @@
 
 module "worklytics_connectors_msft_365" {
   source = "../../modules/worklytics-connectors-msft-365"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-msft-365?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/worklytics-connectors-msft-365?ref=v0.4.61"
 
   enabled_connectors                         = var.enabled_connectors
   environment_id                             = var.environment_name
@@ -34,7 +34,7 @@ module "msft-connection-auth-federation" {
   for_each = module.worklytics_connectors_msft_365.enabled_api_connectors
 
   source = "../../modules/azuread-federated-credentials"
-  # source = "git::https://github.com/worklytics/psoxy//infra/modules/azuread-federated-credentials?ref=v0.4.60"
+  # source = "git::https://github.com/worklytics/psoxy//infra/modules/azuread-federated-credentials?ref=v0.4.61"
 
   application_object_id = each.value.connector.id
   display_name          = "GcpFederation"

infra/examples/aws-google-workspace/main.tf

@@ -59,7 +59,7 @@ data "google_project" "psoxy-google-connectors" {
 
 module "psoxy" {
   # source = "../../modular-examples/aws-google-workspace"
-  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-google-workspace?ref=v0.4.60"
+  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-google-workspace?ref=v0.4.61"
 
   aws_account_id                 = var.aws_account_id
   aws_assume_role_arn            = var.aws_assume_role_arn # role that can test the instances (lambdas)

infra/examples/aws-msft-365/main.tf

@@ -51,7 +51,7 @@ provider "azuread" {
 
 module "psoxy" {
   # source = "../../modular-examples/aws-msft-365"
-  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-msft-365?ref=v0.4.60"
+  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/aws-msft-365?ref=v0.4.61"
 
   aws_account_id                 = var.aws_account_id
   aws_assume_role_arn            = var.aws_assume_role_arn # role that can test the instances (lambdas)

infra/examples/gcp-google-workspace/main.tf

@@ -28,7 +28,7 @@ provider "google" {
 
 module "psoxy" {
   # source = "../../modular-examples/gcp-google-workspace"
-  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/gcp-google-workspace?ref=v0.4.60"
+  source = "git::https://github.com/worklytics/psoxy//infra/modular-examples/gcp-google-workspace?ref=v0.4.61"
 
   gcp_project_id                 = var.gcp_project_id
   environment_name               = var.environment_name