Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI Actions improvements #478

Merged
merged 4 commits into from
Oct 4, 2024
Merged

AI Actions improvements #478

merged 4 commits into from
Oct 4, 2024

Conversation

belmai
Copy link
Contributor

@belmai belmai commented Oct 4, 2024

PR Type

enhancement, tests


Description

  • Removed additionalInstructions field and related logic from multiple files, simplifying the message generation process.
  • Enhanced system prompts to include disclaimer messages and improved language handling.
  • Cleaned up console log statements across various files to improve code readability.
  • Updated test cases to reflect changes in functionality, including the removal of additionalInstructions and the addition of disclaimer handling.
  • Adjusted titles and keys for actions to improve consistency and clarity.

Changes walkthrough 📝

Relevant files
Enhancement
14 files
categorizeMessage.ts
Remove console log statements from categorizeMessage         

extensions/shelly/actions/categorizeMessage/categorizeMessage.ts

  • Removed console log statements for category and explanationHtml.
  • +0/-2     
    categorizeMessageWithLLM.ts
    Clean up console log statements in categorizeMessageWithLLM

    extensions/shelly/actions/categorizeMessage/lib/categorizeMessageWithLLM/categorizeMessageWithLLM.ts

  • Removed console log statements for prompt, result, matchedCategory,
    and explanation.
  • +1/-5     
    fields.ts
    Remove additionalInstructions field from generateMessage configuration

    extensions/shelly/actions/generateMessage/config/fields.ts

    • Removed additionalInstructions field from the configuration.
    +0/-9     
    generateMessage.ts
    Remove additionalInstructions handling in generateMessage

    extensions/shelly/actions/generateMessage/generateMessage.ts

  • Removed additionalInstructions from the payload validation and
    processing.
  • +1/-4     
    constants.ts
    Update system prompt constants for generateMessageWithLLM

    extensions/shelly/actions/generateMessage/lib/generateMessageWithLLM/constants.ts

  • Removed references to Additional Instructions from the system prompt.
  • +3/-11   
    generateMessageWithLLM.ts
    Simplify generateMessageWithLLM by removing additionalInstructions

    extensions/shelly/actions/generateMessage/lib/generateMessageWithLLM/generateMessageWithLLM.ts

  • Removed additionalInstructions from the function parameters and
    processing.
  • +2/-7     
    summarizeCareFlowWithLLM.ts
    Clean up console log statements in summarizeCareFlowWithLLM

    extensions/shelly/actions/summarizeCareFlow/lib/summarizeCareFlowWithLLM/summarizeCareFlowWithLLM.ts

    • Removed console log statements for prompt and summary.
    +1/-2     
    summarizeCareFlow.ts
    Update title and clean up logs in summarizeCareFlow           

    extensions/shelly/actions/summarizeCareFlow/summarizeCareFlow.ts

  • Updated title to 'Summarize Care Flow'.
  • Removed console log statement for htmlSummary.
  • +1/-2     
    fields.ts
    Simplify field descriptions in summarizeForm configuration

    extensions/shelly/actions/summarizeForm/config/fields.ts

    • Simplified descriptions for summaryFormat and language fields.
    +2/-2     
    summarizeForm.ts
    Update title and disclaimer handling in summarizeForm       

    extensions/shelly/actions/summarizeForm/summarizeForm.ts

  • Updated title to 'Summarize Form'.
  • Adjusted handling of disclaimer message.
  • +3/-5     
    summarizeFormsInStep.ts
    Update key, title, and disclaimer handling in summarizeFormsInStep

    extensions/shelly/actions/summarizeFormsInStep/summarizeFormsInStep.ts

  • Updated key and title to 'Summarize Forms in Step'.
  • Adjusted handling of disclaimer message.
  • +5/-7     
    getFormResponseText.ts
    Clean up console log statements in getFormResponseText     

    extensions/shelly/lib/getFormResponseText/getFormResponseText.ts

    • Removed unnecessary console log statements.
    +1/-2     
    constants.ts
    Enhance system prompts with disclaimer and language handling

    extensions/shelly/lib/summarizeFormWithLLM/constants.ts

  • Added disclaimer message handling in system prompts.
  • Updated language handling instructions.
  • +43/-30 
    summarizeFormWithLLM.ts
    Implement disclaimer message handling in summarizeFormWithLLM

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts

    • Added handling for disclaimer message in summarization.
    +6/-3     
    Tests
    9 files
    generateMessage.test.ts
    Update generateMessage tests to exclude additionalInstructions

    extensions/shelly/actions/generateMessage/generateMessage.test.ts

  • Updated test cases to remove references to additionalInstructions.
  • +4/-9     
    generateMessageRealOpenAI.test.ts
    Update real OpenAI tests for generateMessage                         

    extensions/shelly/actions/generateMessage/generateMessageRealOpenAI.test.ts

  • Updated test descriptions and payloads to reflect removal of
    additionalInstructions.
  • +2/-4     
    generateMessageWithLLM.test.ts
    Adjust generateMessageWithLLM tests for removed field       

    extensions/shelly/actions/generateMessage/lib/generateMessageWithLLM/generateMessageWithLLM.test.ts

  • Updated tests to remove additionalInstructions from the test payloads.
  • +2/-5     
    generateMessageWithLLMRealOpenAI.test.ts
    Update real OpenAI tests for generateMessageWithLLM           

    extensions/shelly/actions/generateMessage/lib/generateMessageWithLLM/generateMessageWithLLMRealOpenAI.test.ts

    • Updated test cases to remove additionalInstructions.
    +2/-5     
    summarizeForm.test.ts
    Adjust summarizeForm tests for disclaimer handling             

    extensions/shelly/actions/summarizeForm/summarizeForm.test.ts

    • Updated test expectations to match new disclaimer handling.
    +10/-7   
    summarizeFormRealOpenAI.test.ts
    Add test for Spanish bullet-point format in summarizeForm

    extensions/shelly/actions/summarizeForm/summarizeFormRealOpenAI.test.ts

    • Added a new test case for Spanish bullet-point format.
    +66/-1   
    summarizeFormsInStep.test.ts
    Adjust summarizeFormsInStep tests for disclaimer handling

    extensions/shelly/actions/summarizeFormsInStep/summarizeFormsInStep.test.ts

    • Updated test expectations to match new disclaimer handling.
    +10/-8   
    summarizeFormsInStepRealOpenAI.test.ts
    Adjust real OpenAI tests for summarizeFormsInStep               

    extensions/shelly/actions/summarizeFormsInStep/summarizeFormsInStepRealOpenAI.test.ts

    • Updated test expectations to match new disclaimer handling.
    +25/-26 
    summarizeFormWithLLM.test.ts
    Update summarizeFormWithLLM tests with disclaimer message

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.test.ts

    • Added disclaimer message to test cases.
    +6/-3     

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    @belmai belmai marked this pull request as draft October 4, 2024 10:49
    @github-actions github-actions bot added enhancement New feature or request tests Adds or updates tests for an extension Review effort [1-5]: 3 labels Oct 4, 2024
    Copy link

    github-actions bot commented Oct 4, 2024

    PR Reviewer Guide 🔍

    (Review updated until commit 9893462)

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Refactoring Needed
    The removal of additionalInstructions from the function parameters and logic simplifies the code but may reduce flexibility in message customization. Consider if alternative mechanisms for customization are needed or if this simplification aligns with the system's goals.

    Test Coverage
    Ensure that the updated tests adequately cover the new logic and paths introduced by the changes, particularly the handling of default values and the removal of additionalInstructions.

    Hardcoded Values
    The introduction of hardcoded phrases like "We hope this message finds you well" being refrained could limit the flexibility of message formats. Ensure that such changes are configurable or well-documented if they are necessary for compliance or style guidelines.

    Copy link

    github-actions bot commented Oct 4, 2024

    PR Code Suggestions ✨

    Latest suggestions up to 9893462
    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Security
    Sanitize or validate input to prevent security risks

    Ensure that the disclaimerMessage is properly sanitized or validated before being
    used in the systemPrompt.format method to prevent potential security risks such as
    injection attacks.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [23]

    -disclaimerMessage,
    +sanitizedDisclaimerMessage,
    Suggestion importance[1-10]: 8

    Why: Sanitizing or validating the disclaimerMessage is crucial for preventing potential security risks such as injection attacks. This suggestion addresses a significant security concern, making it highly relevant and impactful.

    8
    Best practice
    Implement error handling for asynchronous operations

    Add error handling for the asynchronous systemPrompt.format and
    ChatModelGPT4o.invoke calls to manage potential failures or exceptions.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [20-24]

    -const prompt = await systemPrompt.format({
    -  language,
    -  input: formData,
    -  disclaimerMessage,
    -})
    +let prompt;
    +try {
    +  prompt = await systemPrompt.format({
    +    language,
    +    input: formData,
    +    disclaimerMessage,
    +  });
    +} catch (error) {
    +  // Handle error appropriately
    +}
    Suggestion importance[1-10]: 7

    Why: Adding error handling for asynchronous calls is a best practice that improves the robustness of the code by managing potential failures or exceptions. This suggestion enhances the code's reliability and maintainability.

    7
    Enhance type safety and code readability by explicitly defining return types

    Consider adding type definitions for the return values of systemPrompt.format and
    ChatModelGPT4o.invoke to enhance type safety and readability.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [20-24]

    -const prompt = await systemPrompt.format({
    +const prompt: PromptType = await systemPrompt.format({
       language,
       input: formData,
       disclaimerMessage,
     })
    Suggestion importance[1-10]: 5

    Why: Adding type definitions for return values enhances type safety and code readability. While not critical, it is a good practice that contributes to clearer and more maintainable code.

    5
    Maintainability
    Improve code readability and maintainability by using a mapping object for conditional logic

    Replace the ternary operations with a more readable switch-case or mapping object
    for systemPrompt selection based on summaryFormat.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [17-19]

    -const systemPrompt = summaryFormat === 'Bullet-points' ? systemPromptBulletPoints : 
    -                     summaryFormat === 'Text paragraph' ? systemPromptTextParagraph :
    -                     systemPromptBulletPoints; // Default to bullet points if unknown format
    +const promptTypes = {
    +  'Bullet-points': systemPromptBulletPoints,
    +  'Text paragraph': systemPromptTextParagraph,
    +  'default': systemPromptBulletPoints,
    +};
    +const systemPrompt = promptTypes[summaryFormat] || promptTypes['default'];
    Suggestion importance[1-10]: 6

    Why: Using a mapping object for selecting systemPrompt improves code readability and maintainability by simplifying the conditional logic. This suggestion is beneficial for understanding and maintaining the code.

    6

    Previous suggestions

    Suggestions up to commit a787101
    CategorySuggestion                                                                                                                                    Score
    Security
    Sanitize or validate the disclaimerMessage to enhance security

    Ensure that the disclaimerMessage is properly sanitized or validated before being
    used to format the system prompt to prevent potential security risks such as
    injection attacks.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [8]

    -disclaimerMessage,
    +disclaimerMessage: sanitize(disclaimerMessage),
    Suggestion importance[1-10]: 9

    Why: Sanitizing or validating disclaimerMessage is crucial for preventing injection attacks, which are significant security risks. This suggestion directly addresses a potential vulnerability, making it highly relevant and impactful.

    9
    Best practice
    Implement error handling for asynchronous operations to improve reliability and error management

    Add error handling for the asynchronous operations within the function to manage
    exceptions and provide more robust error feedback.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [26]

    -const summaryMessage = await ChatModelGPT4o.invoke(prompt)
    +let summaryMessage;
    +try {
    +  summaryMessage = await ChatModelGPT4o.invoke(prompt);
    +} catch (error) {
    +  console.error('Error invoking ChatModelGPT4o:', error);
    +  throw error;
    +}
    Suggestion importance[1-10]: 8

    Why: Adding error handling for asynchronous operations enhances the robustness and reliability of the function by ensuring that exceptions are managed properly. This is a best practice that significantly improves error feedback and resilience.

    8
    Improve type safety by verifying the type of summaryMessage.content before asserting it as a string

    Use TypeScript's type assertion more effectively by ensuring that the type of
    summaryMessage.content is checked before the assertion, to avoid runtime errors.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [29]

    -const summary = summaryMessage.content as string
    +if (typeof summaryMessage.content !== 'string') {
    +  throw new TypeError('Expected string type for summaryMessage.content');
    +}
    +const summary = summaryMessage.content as string;
    Suggestion importance[1-10]: 7

    Why: Checking the type of summaryMessage.content before asserting it as a string prevents potential runtime errors, improving type safety and reliability. This is a valuable best practice for ensuring type correctness.

    7
    Maintainability
    Refactor the assignment of systemPrompt to use a switch statement for clearer handling of different cases and defaults

    Consider adding a default case in the conditional logic for systemPrompt assignment
    to handle unexpected summaryFormat values more explicitly.

    extensions/shelly/lib/summarizeFormWithLLM/summarizeFormWithLLM.ts [17-19]

    -const systemPrompt = summaryFormat === 'Bullet-points' ? systemPromptBulletPoints : 
    -                     summaryFormat === 'Text paragraph' ? systemPromptTextParagraph :
    -                     systemPromptBulletPoints; // Default to bullet points if unknown format
    +let systemPrompt;
    +switch (summaryFormat) {
    +  case 'Bullet-points':
    +    systemPrompt = systemPromptBulletPoints;
    +    break;
    +  case 'Text paragraph':
    +    systemPrompt = systemPromptTextParagraph;
    +    break;
    +  default:
    +    systemPrompt = systemPromptBulletPoints; // Default to bullet points if unknown format
    +    console.warn('Unknown summaryFormat:', summaryFormat);
    +    break;
    +}
    Suggestion importance[1-10]: 7

    Why: Using a switch statement for systemPrompt assignment improves code readability and maintainability by clearly handling different cases and providing explicit warnings for unknown formats. This refactoring enhances clarity without altering functionality.

    7

    @belmai belmai marked this pull request as ready for review October 4, 2024 10:56
    Copy link

    github-actions bot commented Oct 4, 2024

    Persistent review updated to latest commit 9893462

    @belmai belmai self-assigned this Oct 4, 2024
    @belmai belmai merged commit 8debd9c into main Oct 4, 2024
    2 checks passed
    @belmai belmai deleted the ai_actions_improvements branch October 4, 2024 11:15
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    enhancement New feature or request Review effort [1-5]: 3 tests Adds or updates tests for an extension
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants