v5.0.0

Breaking Changes

• Change dependency manager from pipenv to pdm in this release for improved git dependency handling and faster lock and sync operations. (#278)

Other Changes

• Only import R when needed to prevent the logs containing lots of R warnings when R isn't being used. (#275)
• Fix test pipeline crashing due to using an outdated constituencies code_scheme (#273)
• Make all docker scripts executable (#272)
• Add basic map configuration option, for controlling which locations get drawn. (#274)
• Upgrade Firestore filters to utilize the new 'filter' keyword, addressing warnings in the repository: 'UserWarning: Detected filter using positional arguments. Prefer using the 'filter' keyword argument instead.' This enhancement ensures compatibility with the latest practices and resolves the associated warnings. (#276)
• Changed loop to process values instead of keys in the 'messages_map' dictionary for improved efficiency and clarity. (#277)
• Add optional region_filter argument to MapConfiguration in analysis (#280)
• Add optional legend_position argument to MapConfiguration (#281)
• Set Exit Code to 1 for Usage Errors (#282)

Full Changelog: v4.1.0...v5.0.0

v4.1.0

Rapid Pro -> Engagement DB Sync:

• Skip updating Rapid Pro -> Engagement DB contacts cache if contacts haven't changed (#268)

Engagement DB <-> Coda Sync:

• Automatically fix WS cycles (#269)

v4.0.0

Breaking Changes

• Updates some Kenya constituency codes to reflect the constituency names used in the 2022 Kenyan general election. The renamed codes are IGAMBANG_OMBE -> CHUKA_IGAMBANG_OMBE, MBITA -> SUBA_NORTH, and SUBA -> SUBA_SOUTH. (#267)

Other Changes
Rapid Pro -> Engagement DB

• Phone numbers with prefix +25277 are now classified as Hormuud rather than NC (#267).

Engagement DB -> Analysis:

• (Experimental) Add proof-of-concept implementations of complete case regression, on hard-coded variables (#260, #261). Enable by setting feature flag enable_experimental_regression_analysis in the analysis configuration.
• (Experimental) Add initial minimal implementation of analysis dashboard support (#266). Enable by setting feature flag analysis_dashboard_upload in the analysis configuration.

Features marked "experimental" may be subject to breaking changes in future minor releases.

3.3.0

Common:

• x86 images are now smaller (#257).

Rapid Pro -> Engagement DB Sync:

• Greatly reduces the likelihood of the sync cache corrupting when quitting this stage while it is running (#258).

Google Forms -> Engagement DB Sync:

• Supports merging answers from multiple questions into a single message. For example, questions that asked a "yes/no" question followed by a separate "reasons" question could now have the two responses combined into a single message for analysis (#233).
• Add optional --dry-run flags (#234).

v3.2.0

Common:

• Documents pipeline configuration spec (#248)

Rapid Pro -> Engagement DB Sync:

• Optimises fetching of runs to automatically start at the flow creation date time rather than the beginning of time (#254).

Coda <-> Engagement DB Sync:

• Adds optional --skip-updating-coda-users-and-code-schemes flag (#215).

v3.1.0

Common:

• Improve documentation (#247).

Engagement DB <-> Coda Sync:

• Checks that all the requested ws_match_values exist in the WS - Correct Dataset code scheme when loading the configuration (#252).
• Document configuration (#246).

Engagement DB -> Rapid Pro Sync:

• Handles duplicate scheme ids by automatically merging the affected code schemes into a single code scheme. This automatic merging applies to this stage only (#251).

v3.0.0

Breaking changes

• Updates Google Form sync to handle more form structures (adds support for linking multiple question answers into a single message, and for handling question groups - questions which all share the same possible answers such as a grid of ratings) (#231). To upgrade pipeline configurations to be compatible with this release, replace instances of question_title: str with question_titles: list of str in QuestionConfiguration objects.

Other changes
Common:

• Fixes a bug where stages that fetch all the messages from a few datasets were crashing if a message changed dataset during the fetch (#243).
• Fix docker build not working in some environments due to a missing trailing '/' in a docker ADD command (#244).

CSV -> Engagement Database Sync:

• Handles a new datetime format in the CSVs (#245).

Engagement Database -> Analysis:

• Fixes this stage crashing when a time-range filter is set, due to creating invalid TracedData (#242).

v2.2.0

Rapid Pro -> Engagement DB:

• Fixes this stage crashing due to its cache not resetting correctly when the configuration has changed (#238).
• Add an optional time-range filter to each flow result configuration. Results created outside this range will not be added to the engagement database (#239).

Coda Sync:

• Fixes the error message given when the Coda configuration doesn't contain an entry with a ws_code_match_value found in the labelled data (#230).

Engagement DB -> Rapid Pro Sync:

• Asserts that each label is for a code scheme in the project when syncing consent withdrawn. This fixes AttributeError: "NoneType' object has no attribute 'get_code_with_code_id' (#217).
• Fixes this stage crashing due to a typo in the uuid_prefix check (#236).
• Fixes contact field creation not respecting the dry-run flag (#237).

Engagement DB -> Analysis:

• Fixes the exported traced data not being deserializable (#235), and adds a check to the export function that prevents this issue from reoccurring in future (#241).
• Documents the configuration for this stage (#232).

v2.1.0

Common:

• Fetches engagement database messages in batches, preventing crashes on slow networks (#226).
• Allows Message.origin_id to be a str or a list of str (#221).
• Fix lockfile (#222).

Rapid Pro -> Engagement DB Sync:

• Fixes undefined variable errors (#219, #220).
• Fixes this stage crashing when no rapid_pro_sources are provided (#228).
• Log which fields have relevant run results in db sync (#229).

Google Forms Sync:

• Supports setting the uuid from responses to an optional Kenyan mobile number question (#208).

Coda Sync:

• Fixes an error message (now reports ws match_values instead of string_values) (#216)

Analysis:

• Supports partial location imputation (#225).

Engagement DB -> Rapid Pro Sync:

• weekly_advert_contact_field configuration is now optional (#227).
• Fixes sync crashing if code_schemes are organised into subdirectories (#223).
• Don't sync ids that aren't in the format used by the uuid table e.g. don't sync participant uuids that represent anonymous google form responses (#224).

Full Changelog: v2.0.0...v2.1.0

v2.0.0

• Get coda dataset configuration by WS code match value #205

Full Changelog: v1.3.0...v2.0.0