Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFS solution with problem deleting failed task (debug mode enabled) #10

Open
thiagoyeds opened this issue Nov 13, 2020 · 0 comments
Open
Labels
bug Something isn't working

Comments

@thiagoyeds
Copy link
Contributor

When performing the tests of the NFS solution in ufcg-lsd/saps-engine/issues/135, the real algorithms are executed for two tasks that are added, a LANDSAT 7 and another LANDSAT 8. The LANDSAT 8 was correctly executed, going through all the phases successfully and the results with the md5sum also expected, however, LANDSAT 7 failed (as predicted), except that the Archiver routine for failed tasks is generating an error that should occur, but not be launched, but rather be treated so that it does not loop. In this case, the error below is occurring in every routine for the same task fails for reasons that it does not have the preprocessing folder, but fails in the previous phase of inputdownloading, it should be treated, because the files in NFS temporary storage are not being deleted and this may cause disk space problems in the future.

2020-05-25 20:30:30 INFO  CatalogUtils:41 - [Retry Catalog function] Trying gets tasks with failed state using 5 seconds with time sleep
2020-05-25 20:30:30 INFO  Archiver:149 - Deleting temp data from task [fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d]
2020-05-25 20:30:30 INFO  NfsPermanentStorage:82 - Archiving task [fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d] to permanent storage.
2020-05-25 20:30:30 DEBUG NfsPermanentStorage:143 - Copying [/local/exports/fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d/inputdownloading] into [/nfs/trash/fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d]
2020-05-25 20:30:30 DEBUG NfsPermanentStorage:143 - Copying [/local/exports/fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d/preprocessing] into [/nfs/trash/fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d]
2020-05-25 20:30:30 ERROR Archiver:162 - Error while delete task [fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d] files from disk:
java.io.FileNotFoundException: Source '/local/exports/fdd12b4e-cf7c-4225-bb34-1da2e6e0b85d/preprocessing' does not exist
        at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1300)
        at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1193)
        at org.apache.commons.io.FileUtils.copyDirectoryToDirectory(FileUtils.java:1134)
        at org.fogbowcloud.saps.engine.core.archiver.storage.nfs.NfsPermanentStorage.copyDirToDir(NfsPermanentStorage.java:144)
        at org.fogbowcloud.saps.engine.core.archiver.storage.nfs.NfsPermanentStorage.archive(NfsPermanentStorage.java:98)
        at org.fogbowcloud.saps.engine.core.archiver.Archiver.deleteTempData(Archiver.java:158)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at org.fogbowcloud.saps.engine.core.archiver.Archiver.gc(Archiver.java:80)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant