Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler Odd behavior #35

Closed
thoven87 opened this issue Oct 24, 2024 · 7 comments
Closed

Scheduler Odd behavior #35

thoven87 opened this issue Oct 24, 2024 · 7 comments
Labels
bug Something isn't working

Comments

@thoven87
Copy link
Contributor

I have been testing SwiftJobs for the last few weeks on GCP and noticed a funny behavior with the onMinutes scheduler. I think this is true for all the on functions because the implement the same API.

If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted. Most folks will not see this issue if there is at least one Job with the every interval scheduled (i.e .everyMinute(), .everyHour e.t.c)

2024-10-21T22:01:08-0400 debug SwiftJobService : JobName=StatsJob JobTime=2024-10-18 21:25:00 +0000 [Jobs] Next Scheduled Job

value in the Postgres Database

_jobScheduleLastDate:  373530393739323235

After deleting the persisted schedule and restarted the container, the following is printed

2024-10-21T22:16:03-0400 debug SwiftJobService : JobName= StatsJob JobTime=2024-10-22 02:17:00 +0000 [Jobs] Next Scheduled Job

The schedule is as follows:

.init(
    job: StatsJob(),
   schedule: .onMinutes([0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55])
)

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.

I think popping the first value in the queue is what's causing last issue from this list. I am trying to debug what's causing the job to never catch up.

@thoven87 thoven87 added the bug Something isn't working label Oct 24, 2024
@adam-fowler
Copy link
Member

If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted.

Im not totally sure what you mean here. Is it if the jobs are not being cleared fast enough eventually it stops working? And what do you mean by the "schedule is removed"?

After deleting the persisted schedule and restarted the container

Did you delete the database? The last scheduled date is stored in the database.

@thoven87
Copy link
Contributor Author

thoven87 commented Oct 24, 2024

If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted.

Im not totally sure what you mean here. Is it if the jobs are not being cleared fast enough eventually it stops working?

And what do you mean by the "schedule is removed"?
I described "schedule is removed" below. I deleted the last scheduled entry in the Postgres table. The "StatsJob" itself takes about 100ms to complete from start to finish.

After deleting the persisted schedule and restarted the container

Did you delete the database? The last scheduled date is stored in the database.

Yes, I had to delete the last scheduled entry in the database for the job to start running again. The last scheduled date never got updated even though it was in the past.

@adam-fowler
Copy link
Member

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.

Do you not mean `2024-10-22 02:20:00 +0000 given it is every 5 minutes and not 5 seconds?

Can you try and replicate this with #36 and see if the output gives us a better idea what is going on.

@thoven87
Copy link
Contributor Author

thoven87 commented Oct 24, 2024

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.

Do you not mean `2024-10-22 02:20:00 +0000 given it is every 5 minutes and not 5 seconds?

Yes, 2024-10-22 02:20:00 +0000 is the correct time. Thanks for pointing that out.

Can you try and replicate this with #36 and see if the output gives us a better idea what is going on.

I will try later and will report back. Thank You!

@adam-fowler
Copy link
Member

Worked out what this is. Should be fixed by #38

@thoven87
Copy link
Contributor Author

Worked out what this is. Should be fixed by #38

I apologize I have been active as of late. I have been trying to figure out the best way to reproduce this bug. It looks like you just opened a PR to fix it.

@adam-fowler
Copy link
Member

Fixed in beta5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants