Plot Delay + Stagger unexpected behavior!? (Windows 10)

[giordanidev]

Phase 1 plots limit wasn't working, but it is working now. The problem is plot timers (delay+stagger) are not working properly. See other comments below. (Updated)

[torrayne]

Which version of plotng are you running? Could you post your config file?

[giordanidev]

Seems to be working now, EXCEPT that they started 7 mins apart and not 90 (I changed Stagger and Delay to 120 after starting the manager to test the next plots) as in the config.

[giordanidev]

Something is off...

The config file is as I post before. Seems like now the app used both variables times 2h+2h (Stagger + Delay) for a total of 4h delay.

It didn't wait for the delay I had set to start the second plot though. I'll change Stagger back to 5 and see if the next plot starts exactly 2h or 2h05min after the last.

[giordanidev]

Definitely not waiting for the timers I set on the config file.

[markkundinger]

i'm noticing related but different behavior.
if I have MaxActivePlotPerPhase1 set to 2 and have "DelaysBetweenPlot" set to something big like 2 hours, then I will have a case where way more than 2 hours will separate plot times. It seems like maybe plotng is waiting until phase1 opens up, even if that takes hours, and then adding a few hours on after it. I would expect it to start as soon as phase1 ended, as long as it was as least as long as the delay.

[giordanidev]

i'm noticing related but different behavior.
if I have MaxActivePlotPerPhase1 set to 2 and have "DelaysBetweenPlot" set to something big like 2 hours, then I will have a case where way more than 2 hours will separate plot times. It seems like maybe plotng is waiting until phase1 opens up, even if that takes hours, and then adding a few hours on after it. I would expect it to start as soon as phase1 ended, as long as it was as least as long as the delay.

yeah, something is really off with the timers

[giordanidev]

Look at those Duration times.

The 4th plot (counting up>down) was the only plot in Phase 1 for at least 30 mins while there were only another 3 plots running.

This is my config file:

But if you notice the 1st and 2nd plot times, you will see that they started much sooner than 2h50min delay of the 5th plot.

2nd plot started 1h50min after 1st. 3rd started only 1h16min after 2nd. 4th started 2h9min after 3rd.

The main "issue" here is that phase 1 only had 1 plot for a loooong time while the 5 plots limit was not reached.

For reference, here are my archived plot times:

[maded2]

looking at the config you posted above. You can see that you have 2 entries for "PlotSize", which means that this is an invalid configuration. I don't think PlotNG will be able to parse this configuration file properly. Can you check the PlotNG server output to see if there's any error/warning messages?

[giordanidev]

looking at the config you posted above. You can see that you have 2 entries for "PlotSize", which means that this is an invalid configuration. I don't think PlotNG will be able to parse this configuration file properly. Can you check the PlotNG server output to see if there's any error/warning messages?

Hey! I didn't even noticed the warning from VSC lol. I have removed it now but no, no errors regardless of the 2nd PlotSize option (my guess is that it would replace the 1st entry with the 2nd, which would still be 32 anyways, no?).

Anyways, back to the delay issues: My drive became full and it stoped plotting (as expected). I added a new drive 40mins ago, added it in the config, it laded the newly saved config file (as shown below, it recognized the new drive), and it didn't started plotting yet.

Config is set to 35-35 for Delay and Stagger still. Idk what might be happening. =/

---

Edit:
It started plotting 45mins~~ after it loaded the config with the new drive, and there were zero plots running. =/

[markkundinger]

It seems to me like the combination of a phase 1 check and a delay/stagger are working in a different order than I would expect. And it's more noticeable with long delays.

If I have only delay/stagger, and it checks for one hour, it seems to kick off plots every hour
If I have only "max phase 1" then it seems to cap out at phase 1 and then kick off plots a minute or two later.
But if I have both specified, then I see messages in the log like "waiting until 12:30" So it's maybe something like this:

1. plotng is checking every hour
2. plotng notes that phase 1 is maxed out.
3. plotng waits another hour to even check.

While instead, I would have expected plotng to check every minute, and make sure that each plot was spaced by at least an hour, and that phase1 was enforced, but otherwise, plots would fire every minute.

So if I specified a delay of a hour, but phase 1 was full for 1:30, then my next plot would be expected at 1:31.

I run staggers of like 3 hours so it's very noticeable.

[giordanidev]

That makes a lot of sense. It definitely seems to be working that way.

[maded2]

Maybe if I explain the logic then this will help (or uncover a bug)

1. if no temp / target directories set it will skip this cycle (a cycle is 1min)
2. if current time is before targetDelayStartTime then skip this cycle
3. if PlotNG reached the end of the Target list then set the targetDelayStartTime with current time + StaggeringDelay
4. if MaxActivePlotPerPhase1 is set then check if we've reached this limit. If so then skip this cycle
5. if MaxActivePlotPerTemp is set and we've reached this limit then skip this cycle
6. if MaxActivePlotPerTarget is set and we've reached this limit then skio this cycle
7. set targetDelayStartTime with current time + DelaysBetweenPlot
8. if DiskSpaceCheck is set and there's no space with the target then skip this cycle
9. start a new plot

[giordanidev]

So every 1 min it checks plots. That blows away one of out theories. Something is still wrong, though.

1. Ok. Makes sense.
2. targetDelayStartTime is measured by pickiing up running time of latest plot, right? At least it should be.
3. Adding delay should only happen if the plot is started successfuly, after step 9. Also, the only reason I could think of adding delay on each target is if you have really really slow usb 2.0 external targets that would require 1+ hour to receive every plot from temp.
4. Ok.
5. Ok.
6. Ok.
7. Adding delay should only happen if the plot is started successfuly, after step 9.
8. DiskSpaceCheck should be the 2nd check imo. It would skip everything else if that target is full.
9. Ok.

So the optimal configuration to only have delay in between plots being started and a limit of 5 running plots and max of 2 plots at Phase 1 is as follows:

The expected behavior of the config I posted above would be:

1. Check MaxActivePlotPerTarget/Temp/Phase1 (every cycle aka 1 min). Start 1st plot.
2. 90 mins delay.
3. Check MaxActivePlotPerTarget/Temp/Phase1 (every cycle aka 1 min). Start 2nd plot.
4. 90mins delay.
5. Check MaxActivePlotPerTarget/Temp/Phase1 (every cycle aka 1 min). Start 3rd plot.
6. 90mins delay.
7. Check MaxActivePlotPerTarget/Temp/Phase1 (every cycle aka 1 min). Start 4th plot.
8. 90mins delay.
9. Check MaxActivePlotPerTarget/Temp/Phase1 (every cycle aka 1 min). Start 5th plot.
10. Repeat.

No delays for targets. Is that correct? Every plot should start immediately after 90 mins, or 1min~~ (cycle timer) after the Phase1 limit has been released.

[maded2]

Let me think about it. I think your conclusions is very good. As the delays and checks was added over time from user requests so I guess a bit of refactoring is needed.

[giordanidev]

Also, just to add to that, when loading a new config the Delay timers should not "reset". For example, if I had a config file with Delay set to 90 mins and I changed my mind and want to set it to 60mins, and the last plot started 70mins ago, just start a new one immediately (because the 60mins delay has already passed since the last plot started) and then set the next one to start after 60mins.

My point is, ALWAYS use the last running plot timer as the base do the Delays.

Was that clear?

Be sure to also make DiskSpaceCheck to consider already running plots for that target.

[markkundinger]

Maybe if I explain the logic then this will help (or uncover a bug)

1. if no temp / target directories set it will skip this cycle (a cycle is 1min)

2. if current time is before targetDelayStartTime then skip this cycle

3. if PlotNG reached the end of the Target list then set the targetDelayStartTime with current time + StaggeringDelay

4. if MaxActivePlotPerPhase1 is set then check if we've reached this limit.  If so then skip this cycle

5. if MaxActivePlotPerTemp  is set and we've reached this limit then skip this cycle

6. if MaxActivePlotPerTarget is set and we've reached this limit then skio this cycle

7. set targetDelayStartTime with current time + DelaysBetweenPlot

8. if DiskSpaceCheck is set and there's no space with the target then skip this cycle

9. start a new plot

okay let's say targetDelayStartTime gets set in the future a few hours away because stagging.
When that time finally comes, but the system is already busy with something, what's the new targetDelayStartTime ? Is it set forward an additional couple hours? or just pushed forward 1 minute continuously to check for the blocking conditions to free up.

[giordanidev]

I also don't think that this is supposed to happen. I only have 3 plots running with this config, after one of the disks got full. In my understanding, a better way to approach this would be to skip that target and run all 5 plots on the disk that still has space.

Config:

Running plots:

Yes, it's still within the 90min delay BUT I have been watching throughout the day and there were always only 3 plots running. It never got to 4 or 5 plots.

As soon as I removed the full directory from the config file another plot started immediately, which shouldn't have happened since the 90min delay had not been reached yet. (62 mins)