2_SentimentAnalysis

Module 2: Sentiment Analysis using Comprehend

In this module you'll use Amazon API Gateway, AWS Lambda and Amazon Comprehend to analyze the feedback that users enter through the web page. The browser application that you deployed in the first module allows users to enter their name and feedback. The JavaScript running in the browser invokes an API Gateway method, which executes a Lambda function to persist the information entered in a DynamoDB table.

In order to activate “Predict Sentiment” functionality, you will add another method in your API Gateway and a corresponding Lambda function. The Lambda function will invoke Amazon Comprehend Service API to predict the sentiment expressed in user's feedback.

If you're already comfortable with invoking Amazon Comprehend API from a Lambda function, or just want to skip ahead and start working with custom model training and hosting on SageMaker, you can launch one fo these AWS CloudFormation templates in the region of your choice to build out the sentiment prediction functionality automatically.

Region	Launch
US East (N. Virginia)
US East (Ohio)
US West (Oregon)
EU (Ireland)

CloudFormation Launch Instructions (expand for details)

1. Click the Launch Stack link above for the region of your choice.

2. Click Next on the Select Template page.

3. On the Options page, leave all the defaults and click Next.

4. On the Review page, click Create.

5. Wait for the nlp-workshop-voc-comprehend stack to reach a status of CREATE_COMPLETE.

6. With the nlp-workshop-voc-comprehend stack selected, click on the Outputs tab and verify that Rest API ID output value is the same as that you specified in the previous module. You should however see a new value for Deployment Id, indicating the API have been redeployed with the new method added..

7. Verify that you can select one or more feedbacks from VOC application home page and click on Predict Sentiment, and that the page refreshes to show the sentiment of the chosen feedback as one of POSITIVE, NEGATIVE, NEUTRAL or MIXED. Move on to the next module NLP Classifier.

Architecture Overview

The architecture for this module is composed of AWS Lambda function that leverage the sentiment analysis capabilities of Amazon Comprehend. Feedback entered by the user through the web page is persisted in a DynamoDB table. Upon the request from the web page, API Gateway invokes the Lambda function, which sends an API call to Comprehend to do the sentiment analysis. The result is saved to the DynamoDB table and the web page gets refreshed to show the result of the analysis, both through the Lambda function and API Gateway.

Implementation Instructions

Each of the following sections provide an implementation overview and detailed, step-by-step instructions. The overview should provide enough context for you to complete the implementation if you're already familiar with the AWS Management Console or you want to explore the services yourself without following a walkthrough.

If you're using the latest version of the Chrome, Firefox, or Safari web browsers the step-by-step instructions won't be visible until you expand the section.

1. Create a Lambda Function

Background

AWS Lambda will run your code in response to events in this case from API Gateway (which we will cover in the next section). In this step you'll build the core function that will process API requests (via API Gateway) from the web application to detect a sentiment of a feedback text. In the next section you'll use Amazon API Gateway to create a RESTful API that will expose an HTTP endpoint that can be invoked from your users' browsers. You'll then connect the Lambda function you create in this step to that API in order to create a fully functional backend for your web application.

High-Level Instructions

Use the AWS Lambda console to create a new Lambda function called PredictFeedbackSentiment that will process the requests.

Make sure to configure your function to use the VOCLambdaRole IAM role you created in the previous section.

Step-by-step instructions (expand for details)

1. Choose on Services then select Lambda in the Compute section.

2. Click Create function.

3. Keep the default Author from scratch card selected.

4. Enter PredictFeedbackSentiment in the Name field.

5. Select Python 3.6 for the Runtime.

6. Ensure Choose an existing role is selected from the Role dropdown.

7. Select VOCLambdaRole from the Existing Role dropdown.

8. Click on Create function.

9. Scroll down to the Function code section and replace the exiting code in the lambda_function.py code editor with the contents of predictfeedbacksentiment.py.

   

10. Scroll down to the "Environment sariables" section and add an environment variable with Key table_name and Value - UnicornCustomerFeedback.

11. Click "Save" in the upper right corner of the page.

2. Add Policy to Lambda Execution Role

Background

In Module 1, you have created Lambda execution role VOCLambdaRole. In order for your Lambda function you created in the step 1 to call the Comprehend API, you will need to add policy to grands your Lambda function permission to detect sentiment.

High-Level Instructions

Use the IAM Management Console to add a policy to the exisitng role. Go to the Lambda execution role you created in Module 1. You'll need to create a custom inline policy for the role that allows the comprehend:DetectSentiment action.

Step-by-step instructions (expand for details)

1. Go to the IAM Management Console, and look for the VOCLambdaRole role you created in the Module 1.

2. Select the VOCLambdaRole role that you created in the Module 1.

3. On the Permissions tab, choose the Add inline policy link in the lower right corner to create a new inline policy.

4. Select Choose a service.

5. Begin typing Comprehend into the search box labeled Find a service and select Comprehend when it appears.

6. Choose Select actions.

7. Begin typing DetectSentiment into the search box labeled Filter actions and check the box next to DetectSentiment when it appears.

8. Choose Review Policy.

9. Enter ComprehendDetectSentiment for the policy name and choose Create policy.

Implementation Validation

For this section you will test the function that you built using the AWS Lambda console. In the next section you will add a REST API with API Gateway so you can invoke your function from the browser-based application that you deployed in the first module.

1. From the main edit screen for your function, select Configure test event from the the Select a test event... dropdown.

2. Keep Create new test event selected.

3. Enter TestAnalyzeSentiment in the Event name field

4. Insert JSON request body similar to the following into the test event editor:

   {
       "ID": "271605fb-f9f0-4d63-b3fd-578a4a5d8c04,f92921dc-bed1-4af0-98ec-eb128add7cf3"
   }

   Replace the IDs specified with the IDs of the item you have in the DynamoDB table - UnicornCustomerFeedback. You can find the items inserted into DynamoDB table flom DynamoDB console, by looking inside the table UnicornCustomerFeedback.

   Items will get inserted there as you keep using the web application created in Module-1. You can insert as many items and use as many of those IDs as you like. In case of multiple IDs, you have to separate the entries with comma(,).

5. Click Create.

6. On the main function edit screen click Test with TestAnalyzeSentiment selected in the dropdown.

7. Scroll to the top of the page and expand the Details section of the Execution result section.

8. Verify that the execution succeeded and that the function result looks similar to the following (actual results will vary depedning on the feedbacks records you used):

   [
   "271605fb-f9f0-4d63-b3fd-578a4a5d8c04 - POSITIVE",
   "f92921dc-bed1-4af0-98ec-eb128add7cf3 - NEGATIVE"
   ]

   Status of Execution result should display as "succeeded", and expanding the section should show the summary and log output as shown below.

After you have successfully tested your new function using the Lambda console, you can move on to the next section.

3. Add a Resource and Method to the existing REST API

API Gateway calls the Lambda function we created at the previous step. Create a GET method that invokes the Lambda function and passes the IDs of each user feedback as an event.

Create a new resource called /predictsentiment within your API. Then create a GET method for that resource and configure it to use a Lambda proxy integration backed by the PredictFeedbackSentiment function you created in the previous section.

Step-by-step instructions (expand for details)

1. In the left nav, click on Resources under your NLPWorkshopAPI.

2. From the Actions dropdown select Create Resource.

3. Enter predictsentiment as the Resource Name.

4. Ensure the Resource Path is set to predictsentiment.

5. Select Enable API Gateway CORS for the resource.

6. Click Create Resource.

   

7. With the newly created /predictsentiment resource selected, from the Action dropdown select Create Method.

8. Select GET from the new dropdown that appears, then click the checkmark.

   

9. Select Lambda Function for the integration type.

10. Select the Region you are using for Lambda Region.

11. Enter the name of the function you created in the previous module, PredictFeedbackSentiment, for Lambda Function.

12. Choose Save. Please note, if you get an error that you function does not exist, check that the region you selected matches the one you used in the previous module.

    

13. When prompted to give Amazon API Gateway permission to invoke your function, choose OK.

14. Click to open the Method Request card.

15. Open the URL Query String Parameters, and click on Add query string link. Enter Name as Id and then click the checkmark.

    

16. Go back to the previous screen and choose Integration Request card.

17. Scroll down to see Body Mapping Templates and open the section by clicking on the arrow on the left side.

18. Click on Add mapping template link and type application/json in the textbox.

    

19. When you save it by clicking on the checkmark, it will bring up a text area. Type { "ID": "$input.params('Id')" } into it and press the save button.

    

4. Deploy Your API

From the Amazon API Gateway console, choose Actions, Deploy API. You'll be prompted to create a new stage. You can use prod for the stage name.

Step-by-step instructions (expand for details)

1. In the Actions drop-down list select Deploy API.

2. Select the deployment stage named vocweb created in the previous modules in the Deployment stage drop-down list. This will add the additional method to the same deployment, so that the endpoint base URL stays the same.

   

3. Choose Deploy.

4. Note the Invoke URL, it should stay the same as the one created during the previous module. The Javascript file under js/config.js in your website config bucket should already have the same URL specified for invokeUrl.

Implementation Validation

Note: It's possible that you will see a delay between updating the config.js file in your S3 bucket and when the updated content is visible in your browser. You should also ensure that you clear your browser cache before executing the following steps.

1. Visit /enterfeedback.html under your website domain.

2. Enter a test First Name, Last Name and Feedback.

3. Click Submit button. It will redirect to a list of customer names and feedback.

4. Choose checkbox of the feedback you want to detect sentiment.

5. Click on Predict Sentiment button. It will refresh the page and load the result of the sentiment detection.

Congratulations, you have completed the module 2 of the NLP Workshop! You are ready to move on to the next module NLP Classifier.