OpenAI PR Agent Setup

[pluckyswan]

User description

OpenAI PR Agent Setup

• Added a new PR agent AI reviewer.
• Our model is using ChatGPT-4 currently as it was recommended as a good text reading LLM. We can always use a different LLM with Azure OpenAI or run different LLMs as a back up.
• Using mostly default settings, PR Agent is set to Auto-Review/Auto-Improve.
  • We removed auto-describe as we want to encourage engineers to review their stories as part of the PR review process when writing up a PR Description.
• See https://qodo-merge-docs.qodo.ai/tools/ for all the manual triggering features.
• See https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/ for all configuration options.
  • Feel free to look at configurations you think we may need. We use mostly default for now but it can be adjusted accordingly.

Issue

#1357

Checklist

Note: You may remove items that are not applicable

---

PR Type

enhancement, configuration changes

---

Description

• Added a new GitHub Actions workflow to integrate OpenAI's PR Agent, named 'Perry the PR Agent Platypus', which triggers on pull requests and comments.
• Configured the PR Agent to use Azure's OpenAI API with specific version settings to ensure compatibility.
• Created a TOML configuration file for the PR Agent, specifying model settings, review behaviors, and other operational settings.
• The PR Agent is set to automatically review PRs, provide code suggestions, and manage PR descriptions without auto-labeling.

---

Changes walkthrough 📝

Relevant files

Configuration changes

pr_agent.yml Setup GitHub Actions Workflow for OpenAI PR Agent                .github/workflows/pr_agent.yml Added a new GitHub Actions workflow named 'Perry the PR Agent Platypus'. Configured triggers for pull requests, pull request review comments, and issue comments. Set up environment variables for the PR Agent using GitHub secrets. Specified the use of Azure's OpenAI API with specific version settings. +29/-0    .pr_agent.toml Configure PR Agent Settings in TOML File                                  .pr_agent.toml Defined configurations for the PR Agent including model settings and GitHub action behaviors. Configured PR reviewer settings such as the number of code suggestions and inline comments. Set up PR description and code suggestion behaviors. +43/-0

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

[github-actions]

Failed to generate code suggestions for PR

[github-actions]

Failed to generate code suggestions for PR

[github-actions]

Failed to generate code suggestions for PR

[saquino0827]

/describe

[saquino0827]

@CodiumAI-Agent /review

[CodiumAI-Agent]

PR Reviewer Guide 🔍

(Review updated until commit 94d1509)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1357 - Partially compliant Fully compliant requirements: Implement an AI PR review system using a third-party library or service. Remove auto-labelling from the PR Agent. Not compliant requirements: Remove the trigger for pull_request_review_comment. Test the PR Agent with example PRs in Trusted Intermediary and SFTP Ingestion Service. Create new cards for migration of Azure LLM hosting and setup of SFTP Ingestion Service with PR Agent.

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Configuration Issue The trigger for pull_request_review_comment is still present, which should have been removed according to the ticket requirements.

[CodiumAI-Agent]

Persistent review updated to latest commit 595737d

[github-actions]

Failed to generate code suggestions for PR

[CodiumAI-Agent]

Persistent review updated to latest commit 595737d

[github-actions]

Failed to generate code suggestions for PR

[github-actions]

PR Description updated to latest commit (4448436)

[somesylvie]

/review

[github-actions]

PR Code Suggestions ✨

Latest suggestions up to 94d1509

Explore these optional code suggestions:

Category	Suggestion	Score
Possible issue	Correct the syntax of environment variable keys in the GitHub Actions YAML file Ensure that the environment variable keys in the GitHub Actions YAML file follow the correct syntax. Replace the incorrect dot notation with underscores. .github/workflows/pr_agent.yml [25-27] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 10 Why: The suggestion correctly identifies and fixes a syntax error in the environment variable keys, changing from dot notation to underscores. This is crucial as the dot notation would not be correctly recognized in the GitHub Actions environment, potentially causing the workflow to fail or behave unexpectedly.	10

---

Previous suggestions

Suggestions up to commit c9f1a28

Category	Suggestion	Score
Best practice	Standardize environment variable naming convention to use underscores Ensure that the environment variable names follow a consistent format. The current format mixes dots and underscores which might lead to confusion or errors in usage. .github/workflows/pr_agent.yml [25-27] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 8 Why: The suggestion correctly identifies an inconsistency in the environment variable naming convention and provides a standardized format using underscores, which improves readability and reduces potential errors in usage.	8
Maintainability	Update configuration setting descriptions to accurately reflect defaults and enhance clarity Correct the default values and descriptions for configuration settings to accurately reflect the actual defaults and enhance clarity. .pr_agent.toml [18-24] -inline_code_comments = true # If set to true, the tool will publish the code suggestions as comments on the code diff. Default is false. -enable_review_labels_security=false # If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true. +inline_code_comments = true # Default is true. If set to true, the tool will publish the code suggestions as comments on the code diff. +enable_review_labels_security=false # Default is false. If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Suggestion importance[1-10]: 7 Why: Correcting default values and descriptions in the configuration improves clarity and ensures that the documentation matches the actual behavior, which is crucial for maintainability and user understanding.	7
Enhancement	Simplify model configuration settings Remove redundant model configurations if they are not necessary, as it simplifies configuration management and reduces potential errors. .pr_agent.toml [2-4] model="gpt-4" -model_turbo="gpt-4" fallback_models=["gpt-4"] Suggestion importance[1-10]: 6 Why: Removing redundant model configurations simplifies the configuration file and reduces potential errors, making the suggestion valuable for maintainability.	6
Possible issue	Confirm API version compatibility with deployed services Verify the compatibility of the specified API versions with the deployed services to avoid potential integration issues. .github/workflows/pr_agent.yml [28-29] -OPENAI_API_VERSION: '2023-03-15' -AZURE_API_VERSION: '2023-03-15-preview' +OPENAI_API_VERSION: '2023-03-15' # Ensure this version is supported by the deployed OpenAI service +AZURE_API_VERSION: '2023-03-15-preview' # Ensure this version is supported by the deployed Azure service Suggestion importance[1-10]: 5 Why: This suggestion is a precautionary measure to verify API version compatibility. It's a good practice but not a direct improvement or bug fix, hence the moderate score.	5

Suggestions up to commit 4448436

Category	Suggestion	Score
Possible issue	Replace periods in environment variable keys with underscores to ensure compatibility and prevent issues Ensure that the environment variable keys do not contain periods as it might not be supported or could lead to unexpected behavior. Replace OPENAI.API_TYPE, OPENAI.API_BASE, and OPENAI.DEPLOYMENT_ID with underscore-separated keys. .github/workflows/pr_agent.yml [25-27] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 9 Why: Using periods in environment variable keys can lead to compatibility issues or unexpected behavior in certain systems. The suggestion correctly identifies and fixes this potential issue by replacing periods with underscores, which is a best practice in environment variable naming.	9
	Align the configuration setting with the documented default for security label publishing Correct the inconsistency in the configuration for enable_review_labels_security. The comment suggests the default is true, but it is set to false. Align the setting with the intended default value or adjust the comment accordingly. .pr_agent.toml [29] -enable_review_labels_security=false # If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true. +enable_review_labels_security=true # If set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true. Suggestion importance[1-10]: 8 Why: The suggestion correctly identifies a discrepancy between the configuration setting and its documentation comment regarding security label publishing. Aligning these ensures clarity and consistency, which is crucial for maintaining accurate documentation and expected behavior.	8

Suggestions up to commit 4f5f36a

Category	Suggestion	Score
Possible issue	Correct the environment variable keys to use underscores instead of dots Ensure that the environment variable keys in the GitHub Actions YAML file use correct formatting. Replace the incorrect dot notation with underscores. .github/workflows/pr_agent.yml [22-24] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 9 Why: The suggestion correctly identifies a syntax error in the environment variable keys within the GitHub Actions YAML file. Using dots instead of underscores could lead to misinterpretation or errors during the parsing of these variables, potentially causing the workflow to fail. This correction is crucial for the proper functioning of the GitHub Actions workflow.	9

Suggestions up to commit add342d

Category	Suggestion	Score
Best practice	Modify environment variable names to use underscores for better compatibility and standardization Ensure that the environment variable names follow the standard convention of using underscores instead of dots for separation to avoid potential parsing issues. .github/workflows/pr_agent.yml [22-24] -OPENAI.API_TYPE: 'azure' -OPENAI.API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} -OPENAI.DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} +OPENAI_API_TYPE: 'azure' +OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_BASE_URL }} +OPENAI_DEPLOYMENT_ID: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_ID }} Suggestion importance[1-10]: 9 Why: The suggestion correctly identifies a common best practice in environment variable naming, which enhances compatibility and avoids parsing issues. This is a critical improvement for ensuring the workflow functions correctly.	9
Performance	Refine the trigger condition for 'issue_comment' to enhance workflow efficiency Use a more specific trigger condition for the 'issue_comment' event to avoid unnecessary workflow runs. .github/workflows/pr_agent.yml [5] issue_comment: + types: [created] Suggestion importance[1-10]: 7 Why: Specifying the type of 'issue_comment' events can prevent unnecessary workflow triggers, thus saving resources and improving efficiency. This is a relevant and beneficial enhancement.	7
Enhancement	Enable the generation of code suggestions in PR reviews by adjusting the configuration Set 'num_code_suggestions' to a positive value to enable the generation of code suggestions during PR reviews. .pr_agent.toml [21] -num_code_suggestions=0 +num_code_suggestions=3 Suggestion importance[1-10]: 6 Why: Setting 'num_code_suggestions' to a positive value activates a useful feature for automated code suggestions during PR reviews, which can significantly enhance the review process.	6
Maintainability	Simplify the configuration by removing redundant model specifications Remove the redundant model specification for 'model_turbo' if it is intended to be the same as 'model'. .pr_agent.toml [2-3] model="gpt-4" # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo) -model_turbo="gpt-4" # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo) Suggestion importance[1-10]: 4 Why: Removing redundant entries in configuration files can improve maintainability and clarity. However, this change is relatively minor and does not impact functionality directly.	4

[github-actions]

Persistent review updated to latest commit c9f1a28

[github-actions]

Persistent review updated to latest commit 94d1509

[github-actions]

Persistent review updated to latest commit 94d1509

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud