cadc-artifact-sync: Uncaught exception in validate mode

[severingaudet]

Could have been caused by having validate run at the same time as discover. The log file: ~cadcops/artifact-validate-HST-20200117121108.log:

2020-01-17 14:33:17.292 [main] ERROR Main - uncaught exception org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL []; ERROR: duplicate key value violates unique constraint "harvestskipuri_i1" Detail: Key (source, cname, skipid)=(tardis.mast.caom2?HST, Artifact, mast:HST/product/lbry03qrq_corrtag_a.fits) already exists.; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "harvestskipuri_i1" Detail: Key (source, cname, skipid)=(tardis.mast.caom2?HST, Artifact, mast:HST/product/lbry03qrq_corrtag_a.fits) already exists. at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:228) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:607) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:792) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:815) at ca.nrc.cadc.caom2.harvester.state.HarvestSkipURIDAO.put(HarvestSkipURIDAO.java:147) at ca.nrc.cadc.caom2.artifactsync.ArtifactValidator.checkAddToSkipTable(ArtifactValidator.java:459) at ca.nrc.cadc.caom2.artifactsync.ArtifactValidator.compareMetadata(ArtifactValidator.java:261) at ca.nrc.cadc.caom2.artifactsync.ArtifactValidator.run(ArtifactValidator.java:194) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at ca.nrc.cadc.caom2.artifactsync.Validate.execute(Validate.java:138) at ca.nrc.cadc.caom2.artifactsync.Main.main(Main.java:119) Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "harvestskipuri_i1" Detail: Key (source, cname, skipid)=(tardis.mast.caom2?HST, Artifact, mast:HST/product/lbry03qrq_corrtag_a.fits) already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2310) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2023) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166) at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:137) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:798) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:591) ... 10 more 2020-01-17 14:33:17.295 [Thread-0] ERROR Caom2ArtifactSync - terminating with exit status 2

[yeunga]

Since both discover and validate update the skip table, and since both run asynchronously of each other, it's possible that both were processing the same artifact with discover being slightly ahead as follows:

1. discover checked that there was no entry in the skip table for the artifact
2. validate checked that there was no entry in the skip table for the same artifact
3. discover added an entry into the skip table for the artifact
4. validate attempted to add an entry into the skip table for the same artifact

The window is small but is still possible.