Fixed concurrent read/write map issue

This release attempts to fix an error caused by a concurrent read and write to a map of download URLs. It is compatible with previous versions so you may swap out an older version of the download agent with this one and restart downloads with no problem.

MD5sum of tarball: ebe066ac5e1447d18c546ef1142ed8e3

Resolved backoff issue

Resolved an issue where a nil pointer exception tends to appear. This will result in fewer attempts to restart downloading a part for long downloads and cleaner logs. This version is compatible with previous versions of the download agent so you are welcome to interrupt a download and use this executable instead for an improved experience.

MD5sum for this tarball: d00b61803747735029b1a599d31fa59d

Improved handling of retries when downloading parts

This release is more graceful when retrying downloads when the entire internet connection is down for minutes or more. The download agent is designed to restart where it left off if the user terminates the connection. If there are any errors or issues you notice, feel free to terminate and restart the process and you should be up and running again.

Clean up log output for intermittent failures

When downloading with many threads, log output for intermittent network failures can be overwhelming. This release contains a modification to prevent many threads from complaining about an intermittent failure simultaneously.

This release also contains a fix to 'ls_manifest' where a '/' was missing between the folder path and file name for subdirectories in a project.

MD5sum for tarball: 8f68f0d4b61ae2ec3e534dfcd5aabf4e

Additional cosmetic improvements

Changed the output message to more accurately show "MB/s estimated ver the last 60s".

MD5sum of tarball for this release: e815735b399ecd88c697c84e0234636e

Cosmetic improvements

Output is consistently using 1024 KB in a MB, etc for the moment. This may change to IEC in the future.
Carriage returns are likely not to interfere with each other.

MD5sum of tarball: 77810eac20730b953660de931f3fa5d1

Note: The output when reporting net throughput written to disk should read something like "XXX MB/s written to disk estimated over the last 60s" as opposed to "MB" (no "per second") as in this release. This will be fixed in the next incremental release.

Introduce log file framework

This release cleans up output of the download agent by placing detailed messages and errors in a log file of the form <manifest-file-name>.download.log. The idea behind this file is that if an error occurs the user can consult this as a historical log (it includes timestamps) and can forward it to DNAnexus support for further debugging. In the future we plan to include more detailed information about the client's networking setup to help further debug issues.

Increased resilience and cosmetic improvements

• Obtain a mutex to query database for progress updates (in the future this will be managed with a single connection)
• Graceful retries of downloading parts are now done perpetually to deal with intermittent cases where retryablehttp obtains a nil pointer (we are investigating why the pointer is appearing nil in this library)
• Summary output has been cleaned up to be clearer about net throughput in a particular time period and updated every time a part is downloaded

Improved resilience to 503s returned by the platform

This release adds support to perpetually retry on 503s. The wait time to retry uses the retry-after header when available. Otherwise, it exponentially backs off up to 30 minutes in the absence of the retry-after header.

Increased robustness to flaky connections

Added a retry loop to provide increased robustness. Changes tested in an environment that throttles aggressively.