Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding dynamic progress based starting of plots. #1091

Open
wants to merge 5 commits into
base: development
Choose a base branch
from
Open

Adding dynamic progress based starting of plots. #1091

wants to merge 5 commits into from

Conversation

NPutting
Copy link

@NPutting NPutting commented Jun 3, 2021

Adding dynamic progress percent based starting. This waits until the average percent is greater than
(running + 1)/2 * 1/max_concurrent such as for a max concurrent of 8 it would use these values:

current running average %
0 0.00
1 12.50
2 18.75
3 25.00
4 31.25
5 37.50
6 43.75
7 50.00
8 56.25
9 62.50

such as if there were 7 already running, the average would have to be 50% or greater such as,
12.5%, 25%, 37.5%, 50%, 62.5%, 75%, and 87.5 % as they average out to 50%

This should help in two cases:
First, when starting up the manager and plots are going faster than the average time fixed stagger minutes waste time.
Second, the user shouldn't have to try and guess/calculate the correct stagger minutes just their max parallel plots.

This change is completely passive, defaulting to false both in new and old configs without the option. All other config options such as max concurrent, max concurrent with early start, stagger minutes, and max for phase 1 still apply as well. I've documented the option in the config.yaml.default, however, I haven't added anything to the primary README.md yet as I wasn't sure if this would be a default option in the future.

I've been trying this out for about a day so far and plots seem to stagger well. I used a 10-minute minimum stagger so it doesn't delay extra and the average spacing is close to the 48-50 minutes I was using before once steady state is reached.

Info on not met percentages is also included in the INFO level logs similar to other config options and looks like this:
2021-06-02 21:39:23 [INFO]: Minimum percent not met, 51.97 vs 56.25

@NPutting NPutting changed the base branch from main to development June 3, 2021 02:53
@NPutting
Copy link
Author

NPutting commented Jun 3, 2021

Changed the PR to the development branch as that seems to be what other PRs were doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants