Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot limit the workers #2

Open
jmriego opened this issue Dec 6, 2021 · 1 comment
Open

cannot limit the workers #2

jmriego opened this issue Dec 6, 2021 · 1 comment

Comments

@jmriego
Copy link

jmriego commented Dec 6, 2021

Hi! sorry if I'm misunderstanding this completely but I'm trying to create a way to queue jobs in my Raspberry Pi and only have them running sequentially. My test code is like this:

# test.py
from queick import JobQueue
from jobfunc import function
from time import time

q = JobQueue()
for i in range(1,50):
    q.enqueue(function, args=(f"hello{i}",), priority=i, max_workers=1)
# jobfunc.py
def function(arg):
    time.sleep(5)
    print(arg)

I was expecting this code would show me one hello{i} every 5 seconds as I'm only passing max_workers as 1 and all jobs have a different priority. Instead, after 5 seconds all jobs finish at the same time.
Any idea of what am I doing wrong? Thanks!

@jmriego
Copy link
Author

jmriego commented Dec 7, 2021

I think I found the issue in the code. The QueueManager is creating a new executor for every job received so each job has its own executor with as many cores as the Raspberry Pi has. The ThreadPoolExecutor should only be created once before starting the watch loop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant