You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi! After resolved #1276 I builded and installed aptly version 1.5.0+207+g372ce3c4
And now some big packages (>900 Mb) are lost with concurent builds on Jenkins.
When uploading and publishing the package I get status 2, but the package is not in the repository.
For example, the package testfile_4.4.5.7667 was lost.
Please, help me understand, why this is happening?
root@testrepo:~# aptly repo show -with-packages testrepo
testfile_4.4.5.7665_amd64
testfile-archive-_4.4.5.7667_amd64
testfile-long-data_4.4.5.7667_amd64
The Jenkins output looks ok to me... this is just one of the parallel runs ?
some more questions:
Does each run use a unique upload dir (tmp.U5FXf4EUJcvNTWu) ?
Are the 3 files uploaded concurrently ?
What do you mean by the package is not in the repository ?
yes, I use upload_folder=$(mktemp -d tmp.XXXXXXXXXXXXXXX)
I removed some of the packages from the output. There are actually 11 packages loaded sequentially in this build and another 11 packages loaded in a parallel build on Jenkins;
I can see from the output that the package has been successfully downloaded (status 2), but when aptly repo show -with-packages testrepo outputs the package of the version I need is not there, there is no package on the repository web page, and when apt install testfile=4.4.5.7667 there is no such package either.
I think I found the race condition: async background operation was not implemented to do all database operation inside a synchronized block, so for example adding a directory to a repo would read the repo contents, and then run the background tasks. If 2 run at the same time, the could both read a emptry repo, and then in the background operation, on of them wins.
since you are compiling aptly, could you try commit 309450d from the branch fix/concurrent-repo-add ?
this should fix most race conditions (except maybe deleting the repo while adding files)
Hi! After resolved #1276 I builded and installed
aptly
version1.5.0+207+g372ce3c4
And now some big packages (>900 Mb) are lost with concurent builds on Jenkins.
When uploading and publishing the package I get status 2, but the package is not in the repository.
For example, the package
testfile_4.4.5.7667
was lost.Please, help me understand, why this is happening?
In aptly access.log file:
In Jenkins console output:
The text was updated successfully, but these errors were encountered: