-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #582 from Worklytics/rc-v0.4.41
v0.4.41
- Loading branch information
Showing
220 changed files
with
2,579 additions
and
1,791 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # Authentication and Authorization in AWS Deployments of Psoxy | ||
|
|
||
| This page provides an overview of how proxy authenticates and confirms authorization of clients | ||
| (Worklytics tenants). | ||
|
|
||
| ## Authentication | ||
|
|
||
| Each Worklytics tenant operates as a unique GCP service account within Google Cloud. GCP issues | ||
| an identity token for this service account to processes running in the tenant, which the tenant then | ||
| uses to authenticate against AWS. | ||
|
|
||
| This is [OIDC](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc.html) based | ||
| identity federation (aka "web identity federation" or "workload identity federation"). | ||
|
|
||
| No secrets or keys need to be exchanged between Worklytics and your AWS instance. The integrity of | ||
| the authentication is provided by the signature of the identity token provided by GCP, which AWS | ||
| verifies against Google's public certificates. | ||
|
|
||
| ## Authorization | ||
|
|
||
| Within your AWS account, you create an IAM role, with a role assumption policy that allows your | ||
| Worklytics tenant's GCP Service Account (identified by a numeric ID you obtain from the Worklytics | ||
| portal) to assume the role. | ||
|
|
||
| This assumption policy will have a statement similar to the following, where the value of the `aud` | ||
| claim is the numeric ID of your Worklytics tenant's GCP Service Account: | ||
| ```json | ||
| { | ||
| "Effect": "Allow", | ||
| "Principal": { | ||
| "Federated": "accounts.google.com" | ||
| }, | ||
| "Action": "sts:AssumeRoleWithWebIdentity", | ||
| "Condition": { | ||
| "StringEquals": { | ||
| "accounts.google.com:aud": "12345678901234567890123456789" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Colloquially, this allows a web identity federated from `accounts.google.com` where Google has | ||
| asserted the claim that `aud` == `12345678901234567890123456789` to assume the role. | ||
|
|
||
| Then you use this AWS IAM role as the principal in AWS IAM policies you define to authorize to | ||
| invoke your proxy instances via their function URLs (API connectors) or to read from their sanitized | ||
| output buckets (bulk data connectors) | ||
|
|
||
| See: https://github.com/Worklytics/psoxy/blob/v0.4.40/infra/modules/aws/main.tf#L81-L102 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Releases | ||
|
|
||
|
|
||
| ## Prepare Release Candidate | ||
|
|
||
| From `main`: | ||
|
|
||
| ```shell | ||
| ./tools/release/prep.sh v0.4.15 rc-v0.4.16 | ||
| ``` | ||
|
|
||
| - follow steps output by that tool | ||
| - if need interim testing, create a "branch" of the release (eg, branch `v0.4.16` instead of tag), | ||
| and trigger `gh workflow run ci-terraform-examples-release.yaml` | ||
|
|
||
| ## Release | ||
|
|
||
| On `rc-`: | ||
|
|
||
| ```shell | ||
| ./tools/release/prep.sh rc-v0.4.16 v0.4.16 | ||
| ``` | ||
|
|
||
| QA aws, gcp dev examples by running `terraform apply` for each, and testing various connectors. | ||
|
|
||
| ```shell | ||
| ./tools/release/rc-to-release.sh v0.4.16 | ||
| ``` | ||
|
|
||
| After merged to `main`: | ||
| ```shell | ||
| ./tools/release/publish.sh v0.4.16 | ||
| ``` | ||
|
|
||
| ## Java 8 Library Binaries | ||
|
|
||
| ```shell | ||
| cd java/gateway-core | ||
| mvn clean install -P java8 | ||
| cd ../core | ||
| mvn clean install -P java8 | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # FAQ - Security | ||
|
|
||
|
|
||
| ## Can Psoxy invocation be locked to a set of known IP addresses? | ||
|
|
||
| No, but this is not necessary, as requests from your Worklytics tenant to your Psoxy instances are | ||
| authenticated via identity federation (OIDC) and authorized by your Cloud providers IAM policies. | ||
|
|
||
| Your Worklytics tenant is a process running in GCP, personified by a unique GCP service account. You | ||
| simply use your cloud's IAM to grant that service account access to your psoxy instance. | ||
|
|
||
| This is functionally equivalent to how access is authenticated and authorized to within and between | ||
| any public cloud infrastructure. Eg, access to your S3 buckets is authorized via a policy you specify | ||
| in AWS IAM. | ||
|
|
||
| Remember that Psoxy is, in effect, a drop-in replacement for a data sources API; in general, these | ||
| APIs, such as for Google Workspace, Slack, Zoom, and Microsoft 365, are already accessible from | ||
| anywhere on the internet without IP restriction. Psoxy exposes only a more restricted view of the | ||
| source API - a subset of its endpoints, http methods (read-only), and fields - with field values that | ||
| contain PII redacted or pseudonymized. | ||
|
|
||
| See [AWS Authentication and Authorization](aws/authentication-authorization.md) for more details. | ||
|
|
||
| See [GCP Authentication and Authorization](gcp/authentication-authorization.md) for more details. | ||
|
|
||
| ## Can Psoxy instances be deployed behind an AWS API Gateway? | ||
|
|
||
| Yes - and prior to March 2022 this was necessary. But AWS has released [Lambda function urls](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html) | ||
| , which provide a simpler and more direct way to securely invoke lambdas via HTTP. As such, the | ||
| Worklytics-provided Terraform modules use function URLs rather than API gateways. | ||
|
|
||
| API gateways provide a layer of indirection that can be useful in certain cases, but is overkill for | ||
| psoxy deployments - which do little more than provide a transformed, read-only view of a subset of | ||
| endpoints within a data source API. The indirection provides flexibility and control, but at the | ||
| cost of complexity in infrastructure and management - as you must provision a gateway, route, stage, | ||
| and extra IAM policies to make that all work, compared to a function URL. | ||
|
|
||
| That said, the payload lambdas receive when invoked via a function URL is equivalent to the payload | ||
| of API Gateway v2, so the proxy itself is compatible with either API Gateway v2 or function urls. | ||
|
|
||
| ## Can I deploy a WAF in front of my Psoxy instances? | ||
|
|
||
| Sure, but why? Psoxy is itself a rules-based layer that validates requests, authorizes them, and | ||
| then sanitizes the response. It is a drop-in replacement for the API of your data source, which in | ||
| many cases are publicly exposed to the internet and likely implement their own WAF. | ||
|
|
||
| Psoxy never exposes *more* data than is in the source API itself, and in the usual case it provides | ||
| read-only access to a small subset of API endpoints and fields within those endpoints. | ||
|
|
||
| Psoxy is stateless, so all requests must go to the source API. Psoxy does not cache or store any | ||
| data. There is no database to be vulnerable to SQL injections. | ||
|
|
||
| A WAF could make sense if you are using Psoxy to expose an on-prem, in-house built tool to | ||
| Worklytics that is otherwise not exposed to the internet. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Authentication and Authorization in GCP Deployments of Psoxy | ||
|
|
||
| This page provides an overview of how psoxy authenticates and confirms authorization of clients | ||
| (Worklytics tenants) to access data for GCP-hosted deployments. | ||
|
|
||
| ## Authentication | ||
|
|
||
| As Worklytics tenants run inside GCP, they are implicitly authenticated by GCP. No secrets or keys | ||
| need be exchanged between your Worklytics tenant and your Psoxy instance. GCP can verify the | ||
| identity of requests from Worklytics to your instance, just as it does between any process and | ||
| resource within GCP. | ||
|
|
||
|
|
||
| ## Authorization | ||
|
|
||
| Invocations of your proxy instances are authorized by the IAM policies you define in GCP. For API | ||
| connectors, you grant the Cloud Function Invoker role to your Worklytics tenant's GCP service account | ||
| on the Cloud Function for your instance. | ||
|
|
||
| For the bulk data case, you grant the Storage Object Viewer role to your Worklytics tenant's GCP | ||
| service account on the sanitized output bucket for your connector. | ||
|
|
||
| You can obtain the identity of your Worklytics tenant's GCP service account from the Worklytics | ||
| portal. | ||
|
|
||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.