-
Notifications
You must be signed in to change notification settings - Fork 322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: wrong terminal counts calculated during migration check #5400
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5400 +/- ##
==========================================
+ Coverage 74.74% 74.84% +0.10%
==========================================
Files 440 440
Lines 61668 61668
==========================================
+ Hits 46093 46155 +62
+ Misses 13029 12968 -61
+ Partials 2546 2545 -1 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the startup, what kind of clean up we do? and if for a job a terminal status already exists, why do we append more?
Just appends an aborted status without checking for the status already present.
|
Lets change the base to master and release it in 1.40 |
d40ff5e
to
5f1f0db
Compare
Description
Wrong count calculated as part of migration check that happens every 30 seconds.
We fetch counts of terminal statuses in the status table, but since at startup we "cleanup" old jobs just by appending another aborted status irrespective of it's state, that above query can count more than one terminal status per job.
Understandably while actually migrating the jobs, we see that more jobs than we expect have been moved because we were expecting a lesser number.
This issue happened now because archival tables have a default retention of 24 hours. so on successive restarts, more and more statuses were being appended for the same job. And we expect the following expression number of jobs to be migrated:
Due to the cleanup at startup even when
a
remains same,b
increases based on the retention duration effectively decreasinge
. And server panics when it actually migrated more jobs thane
.Now with this fix: we change
b
tonumber of jobIDs with terminal status in the status table
and it's bound to remain the same even if we append more statuses for the same job.Linear Ticket
Resolves PIPE-1827
Security