Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indication of whether a task should be kept around or not. #216

Open
claymcleod opened this issue Sep 24, 2024 · 2 comments
Open

Indication of whether a task should be kept around or not. #216

claymcleod opened this issue Sep 24, 2024 · 2 comments

Comments

@claymcleod
Copy link

In many executors, it's going to be more performant to clean up job records after the job has been completed and the output has been collected. That being said, opting out of this behavior is important when debugging jobs, for instance. It would be great to provide a flag that allows job cleanup (at the discretion of the executor) by default but then be able to specify when you want a job to stick around (at the cost of performance).

@vsmalladi
Copy link
Contributor

Can you clarify what you mean @claymcleod ? TES is fundamentally dealing with individual tasks and inputs and outputs. It’s my opinion that users, or engines would need to deal with cleanup the outputs. As for performance can you clarify. The job is done, so the only performance is storage.

@uniqueg
Copy link
Contributor

uniqueg commented Sep 28, 2024

I would also appreciate a clarification, @claymcleod. Apart from what @vsmalladi asked, I am also wondering what exactly you mean by "job records" and by "executors" - do you mean TES tasks or actual _executor_s as per the spec (of which there can be more than one per TES task).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants