Merge pull request #4 from openedx/sameeramin/ENT-9811

feat: add integrated_channel subapp with utils

channel_integrations/README.md

@@ -0,0 +1,74 @@
+# Integrated Channels
+
+## Overview
+
+An integrated channel is an abstraction meant to represent a third-party system
+which provides an API that can be used to transmit EdX data to the third-party
+system. The most common example of such a third-party system is an enterprise-level
+learning management system (LMS). LMS users are able to discover content made available
+by many different content providers and manage the learning outcomes that are produced
+by interaction with the content providers. In such a scenario, EdX would be the content
+provider while a system like SAP SuccessFactors would be the integrated channel.
+
+The integrated channel subsystem was developed as a means for consolidating common code
+needed for integrating the EdX ecosystem with any integrated channel and minimizing the
+amount of code that would need to be written in order to add a new integrated channel.
+
+## Integration Phases
+
+The subsystem organizes the integration into two separate phases:
+
+1. The *export* phase is where data is collected from the EdX ecosystem
+   and transformed into the schema expected by the integrated channel.
+2. The *transmission* phase is where the exported data is transmitted to
+   the integrated channel by whatever means is provided by the third-party
+   system, usually an HTTP-based API. 
+
+There are [base implementations](https://github.com/openedx/edx-enterprise/tree/master/integrated_channels/integrated_channel)
+for each of these phases which can be extended for
+channel-specific customizations needed for integrating with a given integrated channel.
+Channel-specific implementation code should be placed in a new directory adjacent to
+the base implementation code.
+
+For example:
+
+* [degreed](https://github.com/openedx/edx-enterprise/tree/master/integrated_channels/degreed)
+* [sap_success_factors](https://github.com/openedx/edx-enterprise/tree/master/integrated_channels/sap_success_factors)
+
+## Integration Points
+
+There are currently two integration points supported for integrated channels:
+
+1. *Content metadata* - Metadata (e.g. titles, descriptions, etc.) related to EdX content (e.g.     courses, programs) can be exported and transmitted to an integrated channel to assist content    discovery in the third-party system.
+2. *Learner data* - Specifically, learner outcome data for each content enrollment can be
+   exported and transmitted to the integrated channel.
+
+Additional integration points may be added in the future.
+
+## Integrated Channel Configuration
+
+There is a many-to-many relationship between integrated channels and enterprise customers.
+Configuration information related to an enterprise-specific integrated channel is stored in
+the database by creating a concrete implementation of the abstract
+[EnterpriseCustomerPluginConfiguration](https://github.com/openedx/edx-enterprise/blob/master/integrated_channels/integrated_channel/models.py) model. Fields can be added to the concrete
+implementation to store values such as the API credentials for a given enterprise customer's
+instance of an integrated channel.
+
+Configuration that is common for all instances of integrated channel regardless of enterprise
+customer should be persisted by implementing a model within the channel-specific implementation
+directory, (e.g. [SAPSuccessFactorsGlobalConfiguration](https://github.com/openedx/edx-enterprise/blob/master/integrated_channels/sap_success_factors/models.py)).
+
+## Content Metadata Synchronization
+
+The set of content metadata transmitted for a given integrated channel instance is defined by the
+EnterpriseCustomerCatalogs configured for the associated EnterpriseCustomer. In order to ensure that the content metadata transmitted to an integrated channel is synchronized with the content made available by the EnterpriseCustomer's catalogs, each content metadata item transmission is persisted using the [ContentMetadataItemTransmission](https://github.com/openedx/edx-enterprise/blob/master/integrated_channels/integrated_channel/models.py) model. ContentMetadataItemTransmission records are created, updated, and deleted as EnterpriseCustomerCatalogs are modified and modified sets of content metadata are exported and transmitted to the integrated channel.
+
+## Implementing a new Integrated Channel
+
+Here is the general implementation plan for creating a new integrated channel.
+
+1. Obtain API documentation for the third-party service which the integrated
+   channel will represent.
+2. Implement a concrete subclass of [IntegratedChannelApiClient](https://github.com/openedx/edx-enterprise/blob/master/integrated_channels/integrated_channel/client.py).
+3. Copy/paste one of the existing integrated channel directories and modify the implementation
+   as needed for the new integrated channel.

channel_integrations/catalog_service_utils.py

@@ -0,0 +1,27 @@
+"""
+A utility collection for calls from integrated_channels to Catalog service
+"""
+from enterprise.api_client.discovery import get_course_catalog_api_service_client
+
+
+def get_course_id_for_enrollment(enterprise_enrollment):
+    """
+    Fetch course_id for a given enterprise enrollment
+    Returns None if no course id found for course_id associated with input enterprise_enrollment
+    """
+    course_catalog_client = get_course_catalog_api_service_client(
+        site=enterprise_enrollment.enterprise_customer_user.enterprise_customer.site
+    )
+    return course_catalog_client.get_course_id(enterprise_enrollment.course_id)
+
+
+def get_course_run_for_enrollment(enterprise_enrollment):
+    """
+    Fetch course_run associated with the enrollment
+    Returns empty dict {} if no course_run found for enterprise_enrollment.course_id
+    """
+    course_catalog_client = get_course_catalog_api_service_client(
+        site=enterprise_enrollment.enterprise_customer_user.enterprise_customer.site
+    )
+    course_run = course_catalog_client.get_course_run(enterprise_enrollment.course_id)
+    return course_run

channel_integrations/exceptions.py

@@ -0,0 +1,14 @@
+"""
+Integrated channel custom exceptions.
+"""
+
+
+class ClientError(Exception):
+    """
+    Indicate a problem when interacting with an integrated channel.
+    """
+    def __init__(self, message, status_code=500):
+        """Save the status code and message raised from the client."""
+        self.status_code = status_code
+        self.message = message
+        super().__init__(message)

channel_integrations/integrated_channel/__init__.py

@@ -0,0 +1,5 @@
+"""
+Base Integrated Channel application for specific integrated channels to use as a starting point.
+"""
+
+__version__ = "0.1.0"

channel_integrations/integrated_channel/admin/__init__.py

@@ -0,0 +1,142 @@
+"""
+Admin site configurations for integrated channel's Content Metadata Transmission table.
+"""
+
+from django.contrib import admin
+
+from channel_integrations.integrated_channel.models import (
+    ApiResponseRecord,
+    ContentMetadataItemTransmission,
+    IntegratedChannelAPIRequestLogs,
+)
+from channel_integrations.utils import get_enterprise_customer_from_enterprise_enrollment
+
+
+class BaseLearnerDataTransmissionAuditAdmin(admin.ModelAdmin):
+    """
+    Base admin class to hold commonly used methods across integrated channel admin views
+    """
+    def enterprise_customer_name(self, obj):
+        """
+        Returns: the name for the attached EnterpriseCustomer or None if customer object does not exist.
+        Args:
+            obj: The instance of Django model being rendered with this admin form.
+        """
+        ent_customer = get_enterprise_customer_from_enterprise_enrollment(obj.enterprise_course_enrollment_id)
+        return ent_customer.name if ent_customer else None
+
+
+@admin.action(description='Clear remote_deleted_at on ContentMetadataItemTransmission item(s)')
+def clear_remote_deleted_at(modeladmin, request, queryset):  # pylint: disable=unused-argument
+    queryset.update(remote_deleted_at=None)
+
+
+@admin.register(ContentMetadataItemTransmission)
+class ContentMetadataItemTransmissionAdmin(admin.ModelAdmin):
+    """
+    Admin for the ContentMetadataItemTransmission audit table
+    """
+    list_display = (
+        'enterprise_customer',
+        'integrated_channel_code',
+        'content_id',
+        'remote_deleted_at',
+        'modified'
+    )
+
+    search_fields = (
+        'enterprise_customer__name',
+        'enterprise_customer__uuid',
+        'integrated_channel_code',
+        'content_id'
+    )
+
+    raw_id_fields = (
+        'enterprise_customer',
+    )
+
+    readonly_fields = [
+        'api_record',
+        'api_response_status_code',
+        'friendly_status_message',
+    ]
+
+    actions = [
+        clear_remote_deleted_at,
+    ]
+
+    list_per_page = 1000
+
+
+@admin.register(ApiResponseRecord)
+class ApiResponseRecordAdmin(admin.ModelAdmin):
+    """
+    Admin for the ApiResponseRecord table
+    """
+    list_display = (
+        'id',
+        'status_code'
+    )
+
+    search_fields = (
+        'id',
+        'status_code'
+    )
+
+    readonly_fields = (
+        'status_code',
+        'body'
+    )
+
+    list_per_page = 1000
+
+
+@admin.register(IntegratedChannelAPIRequestLogs)
+class IntegratedChannelAPIRequestLogAdmin(admin.ModelAdmin):
+    """
+    Django admin model for IntegratedChannelAPIRequestLogs.
+    """
+
+    list_display = [
+        "endpoint",
+        "enterprise_customer_id",
+        "time_taken",
+        "status_code",
+    ]
+    search_fields = [
+        "enterprise_customer__name__icontains",
+        "enterprise_customer__uuid__iexact",
+        "enterprise_customer_configuration_id__iexact",
+        "endpoint__icontains",
+    ]
+    readonly_fields = [
+        "status_code",
+        "enterprise_customer",
+        "enterprise_customer_configuration_id",
+        "endpoint",
+        "time_taken",
+        "response_body",
+        "payload",
+    ]
+    list_filter = ('status_code',)
+
+    list_per_page = 20
+
+    def get_queryset(self, request):
+        """
+        Optimize queryset by selecting related 'enterprise_customer' and limiting fields.
+        """
+        queryset = super().get_queryset(request)
+        return queryset.select_related('enterprise_customer').only(
+            'id',
+            'endpoint',
+            'enterprise_customer_id',
+            'time_taken',
+            'status_code',
+            'enterprise_customer__name',
+            'enterprise_customer__uuid',
+            'enterprise_customer_configuration_id'
+        )
+
+    class Meta:
+        model = IntegratedChannelAPIRequestLogs

channel_integrations/integrated_channel/apps.py

@@ -0,0 +1,14 @@
+"""
+Enterprise Integrated Channel Django application initialization.
+"""
+
+from django.apps import AppConfig
+
+
+class IntegratedChannelConfig(AppConfig):
+    """
+    Configuration for the Enterprise Integrated Channel Django application.
+    """
+    name = 'channel_integrations.integrated_channel'
+    verbose_name = "Enterprise Integrated Channels"
+    label = 'channel_integration'

channel_integrations/integrated_channel/channel_settings.py

@@ -0,0 +1,19 @@
+"""
+Channel level settings (global for all channels).
+"""
+
+
+class ChannelSettingsMixin:
+    """
+    Mixin for channel settings that apply to all channels.
+    Provides common settings for all channels. Each channels is free to override settings at their
+    Exporter or Transmitter or Client level, as needed
+
+    Important: If you add a setting here, please add a test to cover this default, as well as
+    any overrides you add on a per channel basis. See this test for an example:
+    tests/test_integrated_channels/test_sap_success_factors/test_transmitters/test_learner_data.py
+    """
+
+    # a channel should override this to False if they don't want grade changes to
+    # cause retransmission of completion records
+    INCLUDE_GRADE_FOR_COMPLETION_AUDIT_CHECK = True