Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize attachment xform migration #2599

Merged
merged 2 commits into from
May 24, 2024

Conversation

kelvin-muchiri
Copy link
Contributor

@kelvin-muchiri kelvin-muchiri commented May 20, 2024

Changes / Features implemented

Run migration as SQL query to optimize performance when number of records to be updated is huge. The migration SQL is

WITH logger_attachment_instance AS (
  SELECT 
    logger_attachment.id, 
    logger_instance.xform_id, 
    logger_instance.user_id 
  FROM 
    logger_attachment 
    INNER JOIN logger_instance ON logger_attachment.instance_id = logger_instance.id 
    INNER JOIN logger_xform T4 ON logger_instance.xform_id = T4.id 
  WHERE 
    logger_attachment.xform_id IS NULL 
    AND T4.deleted_at IS NULL
) 
UPDATE 
  logger_attachment 
SET 
  xform_id = logger_attachment_instance.xform_id, 
  user_id = logger_attachment_instance.user_id 
FROM 
  logger_attachment_instance 
WHERE 
  logger_attachment.id = logger_attachment_instance.id;

The reverse migration SQL query is

WITH logger_attachment_xform AS (
  SELECT 
    logger_attachment.id 
  FROM 
    logger_attachment 
    INNER JOIN logger_xform T4 ON logger_attachment.xform_id = T4.id 
  WHERE 
    logger_attachment.xform_id IS NOT NULL 
    AND T4.deleted_at IS NULL
) 
UPDATE 
  logger_attachment 
SET 
  xform_id = NULL, 
  user_id = NULL 
FROM 
  logger_attachment_xform 
WHERE 
  logger_attachment.id = logger_attachment_xform.id;

Steps taken to verify this change does what is intended

Side effects of implementing this change

Before submitting this PR for review, please make sure you have:

  • Included tests
  • Updated documentation

Closes #

@kelvin-muchiri kelvin-muchiri changed the title Convert python code into SQL query Convert populate attachment xform migration python code into SQL query May 20, 2024
Run migration as SQL query to optimize performance when number of records huge
@kelvin-muchiri kelvin-muchiri force-pushed the optimize-populate-attachment-xform branch from e77e6ef to 3949b38 Compare May 23, 2024 14:13
@kelvin-muchiri kelvin-muchiri marked this pull request as ready for review May 24, 2024 09:43
@kelvin-muchiri kelvin-muchiri requested a review from ukanga May 24, 2024 09:43
@kelvin-muchiri kelvin-muchiri changed the title Convert populate attachment xform migration python code into SQL query Convert populate attachment xform migration into SQL query May 24, 2024
@kelvin-muchiri kelvin-muchiri changed the title Convert populate attachment xform migration into SQL query Optimize attachment xform migration May 24, 2024
@kelvin-muchiri kelvin-muchiri merged commit 76f8408 into main May 24, 2024
12 checks passed
@kelvin-muchiri kelvin-muchiri deleted the optimize-populate-attachment-xform branch May 24, 2024 12:59
@KipSigei KipSigei mentioned this pull request Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants