Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for gcp secrets and labels #11

Merged
merged 4 commits into from
Feb 15, 2024
Merged

Conversation

kongzii
Copy link
Contributor

@kongzii kongzii commented Feb 14, 2024

Summary by CodeRabbit

  • Refactor
    • Enhanced deployment scripts for improved configuration flexibility by replacing api_keys with labels and adding support for env_vars and secrets management.
    • Streamlined deployment functions by adding support for labels, environment variables, and secrets, simplifying the deployment process.
    • Updated deployment commands to include new parameters for labels, environment variables, and secrets, ensuring smoother deployments.

Copy link

coderabbitai bot commented Feb 14, 2024

Warning

Rate Limit Exceeded

@kongzii has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 21 minutes and 31 seconds before requesting another review.

How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
Please see our FAQ for further information.

Commits Files that changed from the base of the PR and between 1c15bc3 and 56fe592.

Walkthrough

The recent updates focus on enhancing deployment scripts for Google Cloud Platform (GCP) by introducing more flexible configuration options. Key changes involve removing the dependency on api_keys in favor of labels, env_vars, and secrets to better manage deployment settings and security. Additionally, there's an adjustment in dependency management by setting the requirements_file to None and updating extra_deps with a new repository link. These modifications streamline deployment processes and improve configuration management.

Changes

File Path Change Summary
.../cloud_deployment/gcp/deploy.py
.../prediction_market_agent_tooling/deploy/gcp/deploy.py
- Imported getpass
- Set requirements_file to None
- Updated extra_deps with new repository link
- Replaced api_keys with labels
- Added env_vars and secrets parameters
.../prediction_market_agent_tooling/deploy/gcp/utils.py - Updated gcloud_deploy_cmd to support labels, env_vars, and secrets as dictionaries

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository from git and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between e6a0d7f and 3097f12.
Files selected for processing (3)
  • examples/cloud_deployment/gcp/deploy.py (2 hunks)
  • prediction_market_agent_tooling/deploy/gcp/deploy.py (2 hunks)
  • prediction_market_agent_tooling/deploy/gcp/utils.py (2 hunks)
Additional comments: 8
examples/cloud_deployment/gcp/deploy.py (4)
  • 2-2: getpass import is added to use getpass.getuser() for dynamically setting the owner label. This is a secure way to retrieve the current user's login name without hardcoding.
  • 17-17: Setting requirements_file to None aligns with the PR objective to remove the requirements_file parameter in favor of a more flexible setup. Ensure all dependent scripts are updated accordingly.
Verification successful

The verification process has confirmed that the requirements_file parameter is set to None in the deploy_to_gcp function call within deploy.py, and no other instances of this function being called with the requirements_file parameter were found in the repository. This aligns with the PR's objective to remove the requirements_file parameter in favor of a more flexible setup, and all dependent scripts appear to be updated accordingly.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Search for files that might still be passing `requirements_file` to `deploy_to_gcp`.
ast-grep --lang python --pattern $'deploy_to_gcp($$$, requirements_file=$$$, $$$)'

Length of output: 1813



Script:

#!/bin/bash
# Broaden the search to find any calls to `deploy_to_gcp` to ensure no calls are passing `requirements_file`.
ast-grep --lang python --pattern $'deploy_to_gcp($$$)'

Length of output: 1785

* 19-19: Updating `extra_deps` with a new repository link is correct. Ensure the repository link is accessible and the `main` branch contains the required dependencies.
Verification successful

The initial verification script used an incorrect URL format for checking the repository's accessibility via HTTP. Let's correct the URL format and try again to verify the repository's accessibility.


The corrected URL verification script successfully accessed the GitHub repository, as indicated by the HTTP 200 status code in the output. This confirms that the repository link https://github.com/gnosis/prediction-market-agent-tooling is accessible. However, without further information or the ability to browse the repository's contents, we cannot directly verify if the main branch contains the required dependencies. The initial task was to ensure the repository link is accessible and to check for the presence of required dependencies in the main branch. The accessibility of the repository has been confirmed, but the second part of the task remains outside the scope of this verification process due to its limitations.

Given the information available and the limitations of this process, the repository's accessibility has been verified.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify the repository link is accessible.
curl -I https://github.com/gnosis/prediction-market-agent-tooling.git@main

Length of output: 1769



Script:

#!/bin/bash
# Corrected command to verify the repository link is accessible.
curl -I https://github.com/gnosis/prediction-market-agent-tooling

Length of output: 4534

* 23-30: The introduction of `labels`, `env_vars`, and `secrets` parameters is in line with the PR objectives. Ensure the format and usage of these parameters are consistent with GCP requirements and best practices. - For `labels`, verify the format (lowercase letters, numbers, hyphens, underscores) is enforced in the code that generates these labels. - For `secrets`, ensure the format "env_var_in_container => secret_name:version" is validated before deployment to avoid runtime errors.
prediction_market_agent_tooling/deploy/gcp/utils.py (2)
  • 13-15: Adding labels, env_vars, and secrets as parameters to gcloud_deploy_cmd is correct and aligns with the PR objectives. Ensure the types of these parameters are consistently documented and validated.
  • 29-34: The loop constructs for appending labels, env_vars, and secrets to the deployment command are correct. However, ensure that the values for labels and secrets are properly sanitized to prevent command injection vulnerabilities.
prediction_market_agent_tooling/deploy/gcp/deploy.py (2)
  • 26-28: Replacing api_keys with labels, env_vars, and secrets in the deploy_to_gcp function's signature is correct and aligns with the PR objectives. Ensure that all calls to this function are updated to use the new parameters.
Verification successful

The absence of results from the script execution suggests that there are no calls to deploy_to_gcp with api_keys as a parameter in the codebase. This aligns with the PR's objective of replacing api_keys with labels, env_vars, and secrets. Therefore, it can be concluded that the changes have been correctly implemented.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Search for files that might still be passing `api_keys` to `deploy_to_gcp`.
ast-grep --lang python --pattern $'deploy_to_gcp($$$, api_keys=$$$, $$$)'

Length of output: 74

* 69-71: The implementation of `labels`, `env_vars`, and `secrets` in the `gcloud_deploy_cmd` call within `deploy_to_gcp` is correct. Ensure the values passed to these parameters are validated and sanitized to prevent runtime errors and security vulnerabilities.

"owner": getpass.getuser()
}, # Only lowercase letters, numbers, hyphens and underscores are allowed.
env_vars={"MARKET": "MANIFOLD"},
# You can allow the cloud function to access secrets by adding the role: `gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} --member=serviceAccount:${GCP_SVC_ACC} --role=roles/container.admin`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! How do you know what to set GCP_SVC_ACC to here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can find it in the console of your cloud function, or you can deploy the function with a specified service account:

Screenshot by Dropbox Capture

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

labels={
"owner": getpass.getuser()
}, # Only lowercase letters, numbers, hyphens and underscores are allowed.
env_vars={"MARKET": "MANIFOLD"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just metadata that you're speculatively adding here because it might be useful down the line? We could add this inside deploy_to_gcp based on the value of market_type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was meant to be just an example about the env vars, didn't know what other to put here.

But agreed that such information could be added automatically.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed this and can confirm the deployment works:

Screenshot by Dropbox Capture

Screenshot by Dropbox Capture

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 3097f12 and 1c15bc3.
Files selected for processing (1)
  • examples/cloud_deployment/gcp/deploy.py (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • examples/cloud_deployment/gcp/deploy.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 1c15bc3 and 6539291.
Files selected for processing (1)
  • examples/cloud_deployment/gcp/deploy.py (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • examples/cloud_deployment/gcp/deploy.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 6539291 and 0079c55.
Files selected for processing (3)
  • examples/cloud_deployment/gcp/deploy.py (2 hunks)
  • prediction_market_agent_tooling/deploy/gcp/deploy.py (2 hunks)
  • prediction_market_agent_tooling/deploy/gcp/utils.py (2 hunks)
Files skipped from review as they are similar to previous changes (3)
  • examples/cloud_deployment/gcp/deploy.py
  • prediction_market_agent_tooling/deploy/gcp/deploy.py
  • prediction_market_agent_tooling/deploy/gcp/utils.py

Copy link
Contributor

@evangriffiths evangriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kongzii kongzii merged commit 878ffea into main Feb 15, 2024
6 checks passed
@kongzii kongzii deleted the peter/gcp-secrets-labels branch February 15, 2024 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants