Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

Job cancellation when training time is not sufficient for steps requested. #79

Open
cmacfarl opened this issue Aug 6, 2021 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@cmacfarl
Copy link
Member

cmacfarl commented Aug 6, 2021

image

For the above job, I specified 20 minutes and 2000 steps. There should be some indication that your training job will be cancelled if you don't allocate enough time for all the steps. This indication should appear before the user starts training.

@lizlooney lizlooney self-assigned this Aug 20, 2021
@lizlooney
Copy link
Collaborator

I'd like to store a running average of training steps per minute, keyed by number of steps total (because it isn't linear), so we can estimate how long a job will take.

@uvidyadharan uvidyadharan added the enhancement New feature or request label Dec 2, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants