You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
See validate() func in validate stage.
It opens a new connection and only closes it at the end of it's work.
But in single process mode, it calls process_hash_list() and process_hash_list() opens it's own connection!
It needs to do this in multiple process mode, so each process has it's own psql connection.
But in single mode, we have an unused connection that is idle and may timeout.
BUT in validate() function, the connection is not used after getUnvalidatedDatasets is called. Can we close it straight away, before any actual work is done?
This may apply to other stages
Severity
Low - it results in one extra connection per stage so not a big deal, but the unused connections may hang due to idleness (see #317 )
The text was updated successfully, but these errors were encountered:
Notes
See validate() func in validate stage.
It opens a new connection and only closes it at the end of it's work.
But in single process mode, it calls process_hash_list() and process_hash_list() opens it's own connection!
It needs to do this in multiple process mode, so each process has it's own psql connection.
But in single mode, we have an unused connection that is idle and may timeout.
BUT in validate() function, the connection is not used after getUnvalidatedDatasets is called. Can we close it straight away, before any actual work is done?
This may apply to other stages
Severity
Low - it results in one extra connection per stage so not a big deal, but the unused connections may hang due to idleness (see #317 )
The text was updated successfully, but these errors were encountered: