feat: link LTI Provider launches to authenticated users

[tecoholic]

Description

The LTI Provider feature of the platform currently allows accessing content from other LMS systems only as an anonymous user. Technically, there is support for using LTI Authentication, which can be used for non-anonymous use case. However, this seems to have been built with Blackboad integration in mind (see section "Authentication & User Provisioning") and doesn't work with Canvas when tested.

Furthermore, the LTI Launch view and the auth mechanism do not consider the existing logged-in user and switch over to the anonymous user account when accessing content over LTI.

For organizations that use multiple LMS, this creates the issue of the grades and outcomes of users not being linked to their accounts despite having an account on the Open edX Platform.

This PR tries to bridge the gap by introducing an opt-in flag that can be set on the LtiConsumer configurations that will allow linking existing users to the actions performed via LTI. The auto-linking of the edx_user and the lti_user is performed when the following conditions are met:

• The LTI Consumer performing the launch has the Require user account checkbox enabled.
• The LTI Consumer sends the lis_person_contact_email_primary during the LTI launch and an user account with the same email already exists in the platform.

Who does this change affect?

• Learners - The data generated via LTI will now be tied to their actual account on the platform instead of an anonymous account

Screenshots
A new flag is now added to the Admin UI of the LTI Consumer model.

Configuration Changes

Since the LTI Provider is already behind the ENABLE_LTI_PROVIDER feature flag, and the auto-linking introduced here is only enabled with an explicit flag included in the PR, no other configuration has been introduced/changed.

Supporting information

• Discourse Discussion about the issue
• LTI 1.1 Specification - Refer this for the lis_person_contact_email_primary optional parameter used in this PR to perform auto-linking.

Testing instructions

NOTE: This has been tested only with Canvas LMS. Technically, it should be possible to test this with any LTI Consumer that sends the user's email using the lis_person_contact_email_primary parameter in the LTI launch request's POST data.

Setting up the platform

1. Pull and switch to the PR branch in your devstack or Tutor dev setup.
2. Enable the LTI Provider functionality
3. Create a new consumer and note down the client id and secret. IMPORTANT: Do NOT check the "Auto link users using email" check box.
4. Create 2 learner accounts
   • [email protected] - We will use this to test the existing functionality
   • [email protected] - We will use this to test the changes introduced in the PR
5. Now open any course on the LMS and grab the course ID and usage ID (attr data-usage-id) of an XBlock.

Setting up Canvas

• Follow the Docker automated setup instructions and setup Canvas LMS (requires cloning the repo, use --depth=1 while cloning to save some time and space)
• Login to Canvas and create a new course
• Go to the Course Settings > Apps and click + App to add a new app.
• In the form fill in the following details:
  • Configuration Type: Manual Entry
  • Name: Any name you would like
  • Consumer Key, Shared Secret: LTI Consumer configuration values generated by the platform
  • Launch URL: Using the course and usage IDs noted before, enter the URL in the format: http(s)://<your-domain>/lti_provider/courses/<course_id>/<usage_id>
  • Privacy: Email only or Public
• Add a new block to the course and choose External Tool. This should list the tool added in the previous step. It might take a couple of minutes or a forced refresh to get it listed as shown here. See Electric Circuit Tool.

• Publish the block and the course
• Add 2 users with the same email as the platform accounts as Students of the course from the People section of the settings. You will have to select the "User details" from the 3-dot menu -> Select "more details" -> "Add Login" under Login information -> set the username and password for the logins to work.

Testing the changes

Without auto-linking (existing functionality)

• Go to the Open edX Platform's admin site -> Users and see the existing user list.
• In a private window, log into Canvas as the first test user test_one and accept the course assignment.
• Click the LTI Block in the course and verify that the embeded XBlock loads without any issue. (A warning or error message may flash for a couple of seconds before the iframe is loaded)
• Go to Open edX Admin Users list and refresh the page, there should be a new user with a random username like this:

With auto-linking (introduced in this PR)

• In Open edX admin, go to LTI Provider > Lti Consumers, change the LTI Consumer created above and check the "Require user accounts" checkbox and save the configuration.
• Open another private window and log into Canvas using the second test user test_two and accept the course assignment, open the section with the XBlock.
• An message informing that the content is available only for signed in users will be displayed.
• Folow the instructions of the displayed message and verify that the XBlock loads as expected. Example:
  
• Now switch to the Open edX admin and go to the Users list. Note that no new users with random ID have been added as before.
• Open the LMS Shell and run ./manage.py lms shell
• Execute the following code block to verify that the second LTI User has been actually linked to the existing user account.

>> from lms.djangoapps.lti_provider.models import LtiUser
>> lti_users = LtiUser.objects.all()
>> print([lti_user.edx_user.email for lti_user in lti_users])
['[email protected]', '[email protected]']

Deadline

"None" if there's no rush, or provide a specific date or event (and reason) if there is one.

Other information

• At the outset, this kind of auto-linking using an OPTIONAL field lis_person_contact_primary_email feels like unnecessary as addressing the issues with the broken LTI Authentication mechanism mentioned earlier in the description should have this working out of the box. However, it was discovered during the investigation that the LtiUser in only ever created from the lti_launch view (also noted by @ztraboo in the forum comment here). This means, in order for the auto-linking to work, on top of fixing the LTI Authentication, we will still need to modify the LtiUser creation logic, which this PR handles.
• Related PR for updating the documentation

[openedx-webhooks]

Thanks for the pull request, @tecoholic! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[Cup0fCoffee]

Launch URL: Using the course and usage IDs noted before, enter the URL in the format: http(s)://<your-domain>/lti_provider/courses/<course_id>/<usage_id>/

Didn't work for me with the trailing slash. After I removed, it worked.

[Cup0fCoffee]

Open the LMS Shell and run ./manage.py shell

./manage.py lms shell

[Cup0fCoffee]

👍 Once the other comments are addressed. Also, is there someone this can be documented for admins?

• I tested this: followed the testing instructions.
• I read through the code
• [-] I checked for accessibility issues
• Includes documentation

[tecoholic]

@Cup0fCoffee I have added the documentation to the relevant section and create a PR.

[Agrendalath]

@tecoholic, it took me a while to set up Canvas and learn the way around it. Two main issues I encountered while testing this:

1. Permission errors after starting Canvas. (solution)
2. Canvas generates identical GUIDs for the Open edX LTI Consumers. The GUID has precedence over the client_id while retrieving the correct consumer, so I had to manually remove this value from the consumer after each LTI launch.

I like this solution - it's clean and straightforward. But...

Assumption: none of your Canvas users has staff or superuser permissions in Open edX.
Let's say somebody gains unauthorized access to your admin account of the Canvas instance. They can modify the email address to be anything without any confirmation. This way, they can send any value in the lis_person_contact_email_primary, thus impersonating any user in the Open edX.

This might be an acceptable risk for instances managed by a single organization, but it can be dangerous for multi-org instances.

For a comparison, when you try to use SAML or OAuth2 with an email already bound to an Open edX account, you are redirected to the login page. When you log in (with your password or an already confirmed TPA), the UserSocialAuth account is linked to your main account.
Could we utilize the LTIProviderConfig from the third_party_auth app to use this existing mechanism?

[tecoholic]

@cmltaWt0 Hi, we were trying to get this into Quince. But discovered a potential security issue and are working on updating the PR to handle it. I will create a backport and add it to the backport list once this goes through. Just wanted to give you a heads-up and see if you have any concerns.

[tecoholic]

@Agrendalath Hi, after exploring multiple avenues, I think I have a fair solution here. The important changes are:

1. The LtiConsumer object is editable – the auto-linking switch can be toggled. This should get rid of the issue of creating duplicate LtiConsumers from the same instance, which were causing the instance GUID clashes.
2. The auto-linking is done only when request.user.email and the lis_person_contact_email_primary match. If the user is not-authenticated or authenticated with a user account with different email, the session will be cleared, and an error message is shown.
3. If an LtiUser object already exists with an anonymous edX User, it will be switched over to the “real account” with the matching email. This is done to maintain the lti_user_id + lti_consumer unique constraint. One side effect of this is, once the LtiUser and the edX user are linked, going back to an anonymous user is not possible on the same LtiConsumer, a new LtiConsumer will have to be created.
4. Finally, the auto-linking will work only if the SESSION_COOKIE_SAMESITE value is set to 'None' and SESSION_COOKIE_SECURE is set to True as the iframes on the LTI Consumer will need it to share the session. If it is not desirable, then the LTI Launch should happen in a new tab instead of the IFrame.

Testing Tips:

1. Use Chrome or Chromium - localhost sites behave like HTTPS and the SameSite=None;Secure cookies are forwarded as expected in a prod-env with HTTPS. I lost a good full day trying to make Firefox do the same, but didn't succeed.
2. Use devstack instead of Tutor, as tutor serves from local.overhang.io and making it run in HTTPS on dev setup can be challenging. Using a http proxy in front of it, didn't work as the OAuth signature validation kept failing. Maybe there is a way to overcome both here, but devstack provides a more straightforward solution.
3. My lms/envs/private.py used for testing:

from .common import FEATURES, INSTALLED_APPS, AUTHENTICATION_BACKENDS

FEATURES['ENABLE_LTI_PROVIDER'] = True
INSTALLED_APPS.append('lms.djangoapps.lti_provider.apps.LtiProviderConfig')
AUTHENTICATION_BACKENDS.append('lms.djangoapps.lti_provider.users.LtiBackend')

SESSION_COOKIE_SAMESITE = 'None'
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_SAMESITE_FORCE_ALL = True
CSRF_COOKIE_SECURE = True
CSRF_COOKIE_SAMESITE = 'None'
# Needed for showing pages in iframe
X_FRAME_OPTIONS = "ALLOW-FROM canvas.docker"

# These are probably not needed
ENABLE_CORS_HEADERS = True
ENABLE_CROSS_DOMAIN_CSRF_COOKIE = True

DCS_SESSION_COOKIE_SAMESITE = 'None'
DCS_SESSION_COOKIE_SECURE = True
DCS_SESSION_COOKIE_SAMESITE_FORCE_ALL = True

Kindly review and see if the security concerns are addressed. I might have inadvertently created more with the cookie permissions, but they can be mitigated by launching on new tab. But everything have been tested in iframe to ensure we are not forcing a new tab on the learner for anything apart from the "Sign in".

[kaustavb12]

Sandbox deployment successful.

Sandbox LMS is available at pr-33310-139931.staging.do.opencraft.hosting
Sandbox Studio is available at studio.pr-33310-139931.staging.do.opencraft.hosting

[open-craft-grove]

Sandbox deployment successful.

Sandbox LMS is available at pr-33310-139931.staging.do.opencraft.hosting
Sandbox Studio is available at studio.pr-33310-139931.staging.do.opencraft.hosting

[Agrendalath]

@tecoholic, I'm trying to understand why we need to change these cookie settings now. The previous approach worked correctly without these changes. What did we change that requires this? Does it mean that the user switching functionality has not been working correctly since the default cookie behavior was altered in the browsers?

didn't work as the OAuth signature validation kept failing.

Maybe it's not the best approach, but I usually just turn this validation off by making the validation functions return True in the devstack.

[open-craft-grove]

Sandbox update request received. Deployment will start soon.

[tecoholic]

@Agrendalath

@tecoholic, I'm trying to understand why we need to change these cookie settings now. The previous approach worked correctly without these changes. What did we change that requires this? Does it mean that the user switching functionality has not been working correctly since the default cookie behavior was altered in the browsers?

The user-switching was working before because, the LtiUser was fetched with just the user_id from the LTI Launch param and then the anonymous edx_user was authenticated using the LtiBackend - which only required that the lti_user_id had an object with a connected edx_user. So even when the session cookie wasn't being sent due to the new browser implementations in the requests, the responses had an established session inside the Iframe. So, in a way the switch_user was both working and not working.

• It was working by switching the AnonymousUser with an actual edx_user
• It wasn't working as in, it didn't switch 2 different edx_users

We need to change the cookie settings now, as we need to be able to authenticate the user who logged in outside the Iframe.

[open-craft-grove]

Sandbox deployment failed.

Check failure logs here https://grove-stage-build-logs.nyc3.digitaloceanspaces.com/34602668-5445942825.log

Please check the settings and requirements and retry deployment by updating the pull request or posting a /grove sandbox update comment.

[open-craft-grove]

Sandbox deployment started.

[Agrendalath]

@tecoholic, let's assume we have Alice and Bob who are the learners. Is the following scenario correct?

1. Alice has already used LTI, so her Canvas and Open edX accounts are linked.
2. Bob used a shared computer in a classroom and left his Open edX account logged in.
3. Alice uses the same shared computer, logs into Canvas, and goes to a Canvas page with the LTI content from Open edX.
4. We have one of the following outcomes:
   1. If Bob had used LTI before, this raises an IntegrityError (due to the 'lti_consumer', 'lti_user_id' uniqueness violation).
   2. Otherwise, Bob's account is accidentally "hijacked" by Alice.
      1. Bob's account will get re-linked again (to the correct account) the next time he uses the LTI, but it is still a security concern.

In this scenario, we want to switch the Open edX user to the one linked to the LtiUser (with the switch_user function).
The only real-world usage I can think of for re-linking the accounts is moving the existing (automatically created) LTI users to "real" Open edX accounts after enabling an existing provider's require_user_account option. If this is our goal, we should verify if the User's email ends with settings.LTI_USER_EMAIL_DOMAIN. However, all existing progress of such users will be lost this way, so we should be sure that we want to do this (instead of, e.g., changing the emails of their automatically created accounts).

[tecoholic]

@Agrendalath First up, I am just astounded at how many of the issues I miss and you catch. And I entirely understand that's the function of the work-review cycle and really appreciate it.

And yes, the only real reason to switch the user here is to make sure we don't have a mix of real users and anonymous users on a client if/when the flag is turned on. I had my reservations about it, but this is something I totally missed

However, all existing progress of such users will be lost this way, so we should be sure that we want to do this

So, I think it is best to remove the re-linking part now. I will update the PR.

[Agrendalath]

Great work, @tecoholic!

👍

• I tested this: linking existing Open edX accounts with the ones from LTI
• I read through the code
• I checked for accessibility issues: n/a
• Includes documentation: docs: adds note about the auto linking flag in LTI Consumer configuration edx-documentation#2190

[openedx-webhooks]

@tecoholic 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.