OpenAI PR Agent Setup

[somesylvie]

OpenAI PR Agent Setup

• Added a new PR agent AI reviewer.
• Our model is using ChatGPT-4 currently as it was recommended as a good text reading LLM. We can always use a different LLM with Azure OpenAI or run different LLMs as a back up.
• Using mostly default settings, PR Agent is set to Auto-Review/Auto-Improve.
  • We removed auto-describe as we want to encourage engineers to review their stories as part of the PR review process when writing up a PR Description.
• See https://qodo-merge-docs.qodo.ai/tools/ for all the manual triggering features.
• See https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/ for all configuration options.
  • Config was added in another PR because the agent only reads config files that are merged into main

Issue

#1357

Checklist

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit e50f233)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1357 - Partially compliant Fully compliant requirements: Implement an AI PR review option using a third-party library or service. Remove auto-labelling from the PR Agent. Not compliant requirements: Remove the trigger: pull_request_review_comment. Test the PR Agent with example PRs in TI and SFTP Ingestion Service. Create new cards for migration of Azure LLM hosting and setup of SFTP Ingestion Service with PR Agent.

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Configuration Inconsistency The PR still includes the trigger for pull_request_review_comment which was supposed to be removed according to the ticket requirements.

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Best practice	Standardize the environment variable keys to use consistent casing and formatting Ensure that the environment variable keys in the YAML file use consistent casing and formatting. Currently, some keys use dots while others use underscores, which might lead to confusion or errors in usage. .github/workflows/pr_agent.yml [25-29] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} OPENAI_API_VERSION: '2023-03-15' AZURE_API_VERSION: '2023-03-15-preview' Suggestion importance[1-10]: 9 Why: The suggestion correctly identifies an inconsistency in the environment variable naming convention, which could lead to errors or confusion. Standardizing the format improves readability and reduces potential bugs.	9
Security	Enable security labels by default to enhance the security review process Set enable_review_labels_security to true by default to ensure security issues are automatically flagged, enhancing the security practices of the PR review process. .pr_agent.toml [25] -enable_review_labels_security=false # If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true. +enable_review_labels_security=true # If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true. Suggestion importance[1-10]: 8 Why: Enabling security labels by default is a critical enhancement for security practices in PR reviews. This suggestion correctly identifies a discrepancy in the default settings and proposes a change that significantly improves security awareness.	8
Enhancement	Add a failure handling step to improve the robustness and maintainability of the workflow Consider adding a failure step to handle errors gracefully in the workflow. This can help in debugging and maintaining the workflow by providing specific error messages or actions when something goes wrong. .github/workflows/pr_agent.yml [19-23] steps: - name: PR Agent action step id: pragent uses: Codium-ai/pr-agent@main env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ... + - name: Handle failure + if: ${{ failure() }} + run: echo "Workflow failed at PR Agent action step" Suggestion importance[1-10]: 7 Why: Adding a failure handling step is a good practice for workflows, enhancing error management and maintainability. This suggestion is relevant and improves the robustness of the GitHub Actions workflow.	7
Maintainability	Remove redundant or clarify the purpose of model configuration lines to avoid confusion It's recommended to remove the redundant model configuration lines or clarify their purpose if they are intended for different uses, as it currently appears to be a mistake or oversight. .pr_agent.toml [2-4] -model="gpt-4" # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo) -model_turbo="gpt-4" # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo) -fallback_models=["gpt-4"] # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo) +model="gpt-4" # the OpenAI model you've deployed on Azure +fallback_models=["gpt-4"] # Fallback models in case the primary model fails Suggestion importance[1-10]: 6 Why: The suggestion to remove or clarify redundant model configurations is valid and improves the maintainability of the configuration file by reducing confusion and potential errors.	6

[github-actions]

Persistent review updated to latest commit e50f233

[github-actions]

PR Description updated to latest commit (e50f233)

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1357 - Partially compliant Fully compliant requirements: Implement an AI PR review option Add GitHub action Add secrets Not compliant requirements: Fix Github Manual PR Agent Triggers on PR Comments Test PR Agent with example PR in TI and SFTP Ingestion Service Create new Cards for migration and setup tasks

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Trigger Configuration The trigger for 'pull_request_review_comment' should be removed as per ticket requirements.

[github-actions]

Remove the trigger for 'pull_request_review_comment' to align with the ticket requirements. This can be done by deleting lines 5 and 6 from the workflow file. [important]

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Best practice	Standardize environment variable naming conventions for consistency Ensure that the environment variable keys in the YAML file follow a consistent naming convention. Currently, some keys use dots (.) while others use underscores (_). Standardizing these will improve readability and reduce the risk of errors. .github/workflows/pr_agent.yml [25-27] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 7 Why: The suggestion correctly identifies an inconsistency in the naming convention of environment variables, which could lead to confusion or errors. Standardizing the format to use underscores improves readability and consistency.	7
Enhancement	Add a repository checkout step to ensure the workflow can interact with the repository contents Consider adding a step to check out the repository code if any operations in subsequent steps require access to the repository contents. This is often necessary for workflows that interact with the repository. .github/workflows/pr_agent.yml [18-21] steps: + - name: Checkout + uses: actions/checkout@v2 - name: PR Agent action step id: pragent uses: Codium-ai/pr-agent@main Suggestion importance[1-10]: 6 Why: Adding a checkout step is a valid suggestion if the workflow requires interaction with the repository's contents. However, without specific details on the operations performed by 'pr-agent', it's uncertain if this step is necessary, hence a moderate score.	6

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1357 - Partially compliant Fully compliant requirements: Implement an AI PR review system using a GitHub action. Remove auto-labelling from the PR Agent. Not compliant requirements: Remove the trigger: pull_request_review_comment. Test the PR Agent with example PRs in TI and SFTP Ingestion Service. Create new cards for migration and setup tasks.

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Configuration Error The trigger for 'pull_request_review_comment' should be removed as per ticket requirements, but it is still present in the workflow file.

[github-actions]

Remove the trigger for 'pull_request_review_comment' to comply with the ticket requirements. This can be done by removing lines 5-6 from the workflow file. [important]

[github-actions]

Title

OpenAI PR Agent Setup

---

User description

OpenAI PR Agent Setup

• Added a new PR agent AI reviewer.
• Our model is using ChatGPT-4 currently as it was recommended as a good text reading LLM. We can always use a different LLM with Azure OpenAI or run different LLMs as a back up.
• Using mostly default settings, PR Agent is set to Auto-Review/Auto-Improve.
  • We removed auto-describe as we want to encourage engineers to review their stories as part of the PR review process when writing up a PR Description.
• See https://qodo-merge-docs.qodo.ai/tools/ for all the manual triggering features.
• See https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/ for all configuration options.
  • Feel free to look at configurations you think we may need. We use mostly default for now but it can be adjusted accordingly.

Issue

#1357

Checklist

---

PR Type

enhancement, configuration changes

---

Description

• Introduced a new GitHub Actions workflow named 'Perry the PR Agent Platypus' to automate PR reviews using Azure OpenAI's GPT-4.
• Configured the workflow to trigger on pull requests and specific types of comments (pull request review comments and issue comments).
• Set up necessary environment variables for Azure OpenAI integration, ensuring the use of the latest API versions for compatibility.
• Defined permissions for the workflow to write to issues, pull requests, and repository contents, enabling comprehensive automation capabilities.

---

Changes walkthrough 📝

Relevant files

Configuration changes

pr_agent.yml Setup GitHub Actions Workflow for OpenAI PR Agent                .github/workflows/pr_agent.yml Introduced a new GitHub Actions workflow named 'Perry the PR Agent Platypus'. Configured triggers for pull requests and comments to activate the workflow. Set up environment variables for integration with Azure OpenAI using GPT-4. Specified permissions for the workflow to interact with issues, pull requests, and repository contents. +29/-0

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud