NZSL-77 ValidationResult model (#138)

* Document ValidationResult model approach * Add ValidtionRecord model and model admin The model admin contains a input search filter for the gloss, modeled after the example found here https://hakibenita.com/how-to-add-a-text-filter-to-django-admin * Adjust model fields * Extend validation result model documentation * Minor model adjustment --------- Co-authored-by: Nora Schneider <[email protected]>
ODNZSL · Feb 4, 2024 · baac24c · baac24c
1 parent 2261ff4
commit baac24c
Show file tree

Hide file tree

Showing 6 changed files with 268 additions and 7 deletions.
diff --git a/docs/qualitrics-screenshot.png b/docs/qualitrics-screenshot.png
diff --git a/docs/validation_result_model.md b/docs/validation_result_model.md
@@ -0,0 +1,136 @@
+# Qualitrics exported CSV data and ValidationResult model
+
+Qualitrics gives the ability to export survey data in several formats, 
+the one relevant for NZSL Signbank is CSV.
+
+## Example CSV snippet
+
+| StartDate	                                             | EndDate                                              | Status                | IPAddress                | Progress                | Duration (in seconds)   | Finished                | RecordedDate                                              | ResponseId               | RecipientLastName                | RecipientFirstName                | RecipientEmail                | ExternalReference                    | LocationLatitude                | LocationLongitude                | DistributionChannel                | UserLanguage                | Q_BallotBoxStuffing                | 1_Q1_1                                                                                           | 1_Q2                                                                                            | 1_Q2_5_TEXT                                                                                  |
+|--------------------------------------------------------|------------------------------------------------------|-----------------------|--------------------------|-------------------------|-------------------------|-------------------------|-----------------------------------------------------------|--------------------------|----------------------------------|-----------------------------------|-------------------------------|--------------------------------------|---------------------------------|----------------------------------|------------------------------------|-----------------------------|------------------------------------|--------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
+| Start Date                                             | End Date                                             | Response Type         | IP Address               | Progress                | Duration (in seconds)   | Finished                | Recorded Date                                             | Response ID              | Recipient Last Name              | Recipient First Name              | Recipient Email               | External Data Reference              | Location Latitude               | Location Longitude               | Distribution Channel               | User Language               | Q_BallotBoxStuffing                | 1_Q1 - https://vuw.qualtrics.com/CP/File.php?F=F_78nY3cJ9AWK0XtA - Have seen it or use it myself | Comment - https://vuw.qualtrics.com/CP/File.php?F=F_78nY3cJ9AWK0XtA - Comment - Selected Choice | Comment - https://vuw.qualtrics.com/CP/File.php?F=F_78nY3cJ9AWK0XtA - Write a comment - Text |
+| {"ImportId":"startDate","timeZone":"Pacific/Auckland"} | {"ImportId":"endDate","timeZone":"Pacific/Auckland"} | {"ImportId":"status"} | {"ImportId":"ipAddress"} | {"ImportId":"progress"} | {"ImportId":"duration"} | {"ImportId":"finished"} | {"ImportId":"recordedDate","timeZone":"Pacific/Auckland"} | {"ImportId":"_recordId"} | {"ImportId":"recipientLastName"} | {"ImportId":"recipientFirstName"} | {"ImportId":"recipientEmail"} | {"ImportId":"externalDataReference"} | {"ImportId":"locationLatitude"} | {"ImportId":"locationLongitude"} | {"ImportId":"distributionChannel"} | {"ImportId":"userLanguage"} | {"ImportId":"Q_BallotBoxStuffing"} | {"ImportId":"1_QID10_1"}                                                                         | {"ImportId":"1_QID7"}                                                                           | {"ImportId":"1_QID7_5_TEXT"}                                                                 |
+| 9/11/2022 17:54                                        | 9/11/2022 18:44                                      | 0                     |                          | 100                     | 2987                    | 1                       | 9/11/2022 18:44                                           | R_UMhF6SuJzvtZE2t        | Doe                              | Joe                               |                               |                                      |                                 |                                  | email                              | EN-GB                       |                                    | Not sure                                                                                         | Write a comment	                                                                                | couldn't see the video                                                                       |
+| 10/11/2022 18:23                                       | 11/11/2022 8:57                                      | 0                     |                          | 100                     | 52428                   | 1                       | 11/11/2022 8:57                                           | R_3qqXgb2jvWPRPbR        | Name                             | Random                            |                               |                                      |                                 |                                  | email                              | EN-GB                       |                                    | Not sure                                                                                         | Write a comment,I want to talk about this sign in NZSL - contact me                             | video not showing?                                                                           |
+| 10/11/2022 21:53                                       | 11/11/2022 20:49                                     | 0                     |                          | 100                     | 82586                   | 1                       | 11/11/2022 20:49                                          | R_3MrPivulGQ6TJmk        | Someone                          | Else                              |                               |                                      |                                 |                                  | email                              | EN-GB                       |                                    | Yes                                                                                              | Write a comment	                                                                                | I think only one sign for abbreviation - ticked yes but can't see video                      |
+
+## [Qualitrics][qualitrics-data] documentation
+### [Format basics][format-basics]
+
+CSV and TSV files come with 3 rows of headers. The first header is the internal Qualtrics ID of 
+the field (e.g., EndDate, Q1, Q2, and so on). The second header is the field’s name or text 
+(e.g., End Date, How satisfied are you with Qualtrics?). The third header has import IDs. All 3 
+of these headers are included because they are needed to upload the data to a survey. Respondent 
+data starts on the fourth row of the file.  
+
+### [Respondent information][respondent-information]
+The first several columns pertain to information about the respondent. For the purpose of 
+importing the data into Signbank most of these columns can be ignored.  
+There are a number of different statuses a response can have, supplied in the `Status` column. 
+Their values can be:
+
+- `0 / IP Address`: A normal response
+- `1 / Survey Preview`: A preview response 
+- `2 / Survey Test`: A test response 
+- `4 / Imported`: An imported response 
+- `8 / Spam`: A possible spam response 
+- `9  / Preview Spam`: A possible spam response submitted through the preview link 
+- `16 / Offline`: A Qualtrics Offline App response 
+- `17 / Offline Preview`: Previews submitted through the Qualtrics Offline App. This feature is deprecated in latest versions of the app
+
+Based on these values it might be worth ignoring anything but 0 and 4
+
+Furthermore, we should take note of the `ResponseId`, `RecipientLastName` and `RecipientFirstName` columns to 
+store the name of the validator. Potentially even the `RecipientEmail` column.
+
+For each gloss there are three column headers. In the example provided their names are of the 
+format 
+1. (`{number}_Q1_1`,`{number}_Q1 - {video_url} - Have seen it or use it myself` , `ImportId {number}_QID10_1`)
+2. (`{number}_Q2`, `Comment - {video_url} - Comment - Selected Choice` , `ImportId {number}_QID7`)
+3. (`{number}_Q2_5_TEXT`,`Comment - {video_url} - Write a comment - Text`, `ImportId {number}_QID7_5_TEXT`) 
+
+It seems tricky to put the idgloss into the column headers instead of the question number for the 
+export, so the idgloss will have to be extracted form the url instead.
+
+The first column corresponds to the question `Have seen it or use it myself` with possible answers 
+`Yes`, `No`, `Not sure`. 
+In a previous version of the CSV export these columns were filled with values (at a glance) 1,2,4.  
+Assumption: 1-Yes, 2-No, 4-Not sure
+
+The second and third columns correspond to a Comment the respondent can leave. The second column  
+represents the comment choice, presumably the tick box in the screen shot. 
+A newer version of the CSV file shows which tick boxes have been selected, eg. 
+`Write a comment,I want to talk about this sign in NZSL - contact me`.
+The previous version of the CSV showed the values 5 and 7. This meant that 5 refers to the `write a comment` tickbox, and 7 refers to 
+the `I wans to talk to NZSL about this comment` tickbox.  
+Column 3 represents the comment itself.
+
+![screenshot][qualitrics-screenshot]
+
+# Proposed model
+
+While there is an option to have a JSON field on a model and store all recorded answers for a 
+gloss in that field it might be cleaner to just have a record per respondent per gloss.
+
+```python
+from django.db import models
+
+
+class ValidationRecord(models.Model):
+    class SignSeenChoices(models.TextChoices):
+        YES = "yes", "Yes"
+        NO = "no", "No"
+        NOT_SURE = "not_sure", "Not sure"
+    gloss = models.ForeignKey(Gloss, related_name="validation_records", on_delete=models.CASCADE)
+    sign_seen = models.CharField(
+        max_length=50, choices=SignSeenChoices.choices,
+        help_text="Result of the survey question 'Have seen it or use it myself'"
+    )
+    response_id = models.CharField(
+        max_length=255, help_text="Identifier of specific survey result in Qualitrics"
+    )  # can potentially make this unique
+    respondent_first_name = models.CharField(
+        max_length=255, default="", help_text="Survey respondents first name"
+    )
+    respondent_last_name = models.CharField(
+        max_length=255, default="", help_text="Survey respondents last name"
+    )
+    respondent_email = models.EmailField(default="", help_text="Survey respondents email")
+    comment = models.TextField(
+        default="", help_text="Optional comment the survey respondent can leave about the gloss"
+    )
+    contact_with_nzsl_requested = models.BooleanField(
+        default=False,
+        help_text=(
+            "Boolean value that indicates if the survey respondent would like to be contacted by "
+            "NZSL to discuss the gloss further"
+        )
+    )
+
+```
+
+## ShareValidationAggregation model
+
+As part of NZSL-74 a model has been introduced to capture the amount of people agreeing and 
+disagreeing with a gloss from Share. This model is populated during the CSV import of glosses 
+from Share. These aggregated results will be displayed along the aggregated results of the 
+ValidationRecords imported from Qualitrics, part of NZSL-78.
+
+```python
+from django.db import models
+
+
+class ShareValidationAggregation(models.Model):
+    """
+    Captures how many people on NZSL Share agree or disagree with a gloss
+    """
+    gloss = models.ForeignKey(Gloss, related_name="share_validation_aggregations",
+                              on_delete=models.CASCADE)
+    agrees = models.PositiveIntegerField()
+    disagrees = models.PositiveIntegerField()
+```
+
+<!-- Links and resources -->
+[qualitrics-data]: https://www.qualtrics.com/support/survey-platform/data-and-analysis-module/data/download-data/export-data-overview/#UnderstandingDataSet
+[format-basics]: https://www.qualtrics.com/support/survey-platform/data-and-analysis-module/data/download-data/understanding-your-dataset/#Basics
+[respondent-information]: https://www.qualtrics.com/support/survey-platform/data-and-analysis-module/data/download-data/understanding-your-dataset/#RespondentInformation
+[qualitrics-screenshot]: ./qualitrics-screenshot.png
diff --git a/signbank/dictionary/admin.py b/signbank/dictionary/admin.py
@@ -18,7 +18,7 @@
 from .models import (AllowedTags, Dataset, Dialect, FieldChoice, Gloss, Lemma,
                      GlossRelation, GlossTranslations, GlossURL, Language,
                      ShareValidationAggregation,
-                     SignLanguage, Translation)
+                     SignLanguage, Translation, ValidationRecord)
 from ..video.admin import GlossVideoInline
 
 
@@ -281,7 +281,6 @@ class FieldChoiceAdmin(admin.ModelAdmin):
     list_display = ('field', 'english_name', 'machine_value',)
 
 
-
 class AssignedGlossInline(admin.StackedInline):
     model = Gloss
     fk_name = 'assigned_user'
@@ -295,6 +294,44 @@ def has_add_permission(self, request, obj=None):
         return False
 
 
+class InputFilter(admin.SimpleListFilter):
+    template = 'admin/input_filter.html'
+
+    def lookups(self, request, model_admin):
+        # Dummy, required to show the filter.
+        return ((),)
+
+    def choices(self, changelist):
+        # Grab only the "all" option.
+        all_choice = next(super().choices(changelist))
+        all_choice['query_parts'] = (
+            (k, v)
+            for k, v in changelist.get_filters_params().items()
+            if k != self.parameter_name
+        )
+        yield all_choice
+
+
+class GlossFilter(InputFilter):
+    parameter_name = 'gloss'
+    title = _('Gloss')
+
+    def queryset(self, request, queryset):
+        if self.value() is not None:
+            gloss = self.value()
+
+            return queryset.filter(
+                models.Q(gloss__idgloss__contains=gloss)
+            )
+
+
+class ValidationRecordAdmin(admin.ModelAdmin):
+    model = ValidationRecord
+    list_display = ("gloss", "response_id", "sign_seen")
+    search_fields = ["response_id"]
+    list_filter = [GlossFilter, "sign_seen"]
+
+
 class UserAdmin(AuthUserAdmin):
     inlines = [AssignedGlossInline]
 
@@ -307,7 +344,8 @@ class UserAdmin(AuthUserAdmin):
 admin.site.register(GlossRelation, GlossRelationAdmin)
 admin.site.register(AllowedTags, AllowedTagsAdmin)
 admin.site.register(Lemma)
-admin.site.register(ShareValidationAggregation,ShareValidationAggregationAdmin)
+admin.site.register(ShareValidationAggregation, ShareValidationAggregationAdmin)
+admin.site.register(ValidationRecord, ValidationRecordAdmin)
 
 admin.site.unregister(User)
 admin.site.register(User, UserAdmin)

diff --git a/signbank/dictionary/migrations/0045_validation_record_model.py b/signbank/dictionary/migrations/0045_validation_record_model.py
@@ -0,0 +1,28 @@
+# Generated by Django 3.2.23 on 2024-01-25 03:07
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('dictionary', '0044_share_validation_aggregtion_model'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='ValidationRecord',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('sign_seen', models.CharField(choices=[('yes', 'Yes'), ('no', 'No'), ('not_sure', 'Not sure')], help_text="Result of the survey question 'Have seen it or use it myself'", max_length=50)),
+                ('response_id', models.CharField(help_text='Identifier of specific survey result in Qualitrics', max_length=255)),
+                ('respondent_first_name', models.CharField(default='', help_text='Survey respondents first name', max_length=255)),
+                ('respondent_last_name', models.CharField(default='', help_text='Survey respondents last name', max_length=255)),
+                ('respondent_email', models.EmailField(default='', help_text='Survey respondents email', max_length=254)),
+                ('comment', models.TextField(default='', help_text='Optional comment the survey respondent can leave about the gloss')),
+                ('contact_with_nzsl_requested', models.BooleanField(default=False, help_text='Boolean value that indicates if the survey respondent would like to be contacted by NZSL to discuss the gloss further')),
+                ('gloss', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='validation_records', to='dictionary.gloss')),
+            ],
+        ),
+    ]
diff --git a/signbank/dictionary/models.py b/signbank/dictionary/models.py
@@ -2,7 +2,6 @@
 """Models for the Signbank dictionary/corpus."""
 from __future__ import unicode_literals
 
-import json
 import re
 from collections import OrderedDict
 from itertools import groupby
@@ -14,8 +13,7 @@
 from django.urls import reverse
 from django.utils.translation import ugettext_lazy as _
 from tagging.models import Tag
-from tagging.registry import AlreadyRegistered
-from tagging.registry import register as tagging_register
+from tagging.registry import AlreadyRegistered, register as tagging_register
 
 
 class Dataset(models.Model):
@@ -792,11 +790,47 @@ class ShareValidationAggregation(models.Model):
     """
     Captures how many people on NZSL Share agree or disagree with a gloss
     """
-    gloss = models.ForeignKey(Gloss, related_name="share_validation_aggregations", on_delete=models.CASCADE)
+    gloss = models.ForeignKey(Gloss, related_name="share_validation_aggregations",
+                              on_delete=models.CASCADE)
     agrees = models.PositiveIntegerField()
     disagrees = models.PositiveIntegerField()
 
 
+class ValidationRecord(models.Model):
+    """Record Qualitrics validation result for a gloss """
+
+    class SignSeenChoices(models.TextChoices):
+        YES = "yes", "Yes"
+        NO = "no", "No"
+        NOT_SURE = "not_sure", "Not sure"
+
+    gloss = models.ForeignKey(Gloss, related_name="validation_records", on_delete=models.CASCADE)
+    sign_seen = models.CharField(
+        max_length=50, choices=SignSeenChoices.choices,
+        help_text="Result of the survey question 'Have seen it or use it myself'"
+    )
+    response_id = models.CharField(
+        max_length=255, help_text="Identifier of specific survey result in Qualitrics"
+    )  # can potentially make this unique
+    respondent_first_name = models.CharField(
+        max_length=255, default="", help_text="Survey respondents first name"
+    )
+    respondent_last_name = models.CharField(
+        max_length=255, default="", help_text="Survey respondents last name"
+    )
+    respondent_email = models.EmailField(default="", help_text="Survey respondents email")
+    comment = models.TextField(
+        default="", help_text="Optional comment the survey respondent can leave about the gloss"
+    )
+    contact_with_nzsl_requested = models.BooleanField(
+        default=False,
+        help_text=(
+            "Boolean value that indicates if the survey respondent would like to be contacted by "
+            "NZSL to discuss the gloss further"
+        )
+    )
+
+
 # Register Models for django-tagging to add wrappers around django-tagging API.
 models_to_register_for_tagging = (Gloss, GlossRelation,)
 for model in models_to_register_for_tagging: