You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have checked that my issue does not already have a solution in the FAQ
I have searched the existing issues and didn't find my bug already reported there
I have checked that my bug is still present in the latest release
Version
APScheduler==3.10.4
What happened?
Using this simple code:
import asyncio
from multiprocessing import SimpleQueue, set_start_method
from apscheduler.executors.pool import ThreadPoolExecutor, ProcessPoolExecutor
from apscheduler.schedulers.background import BackgroundScheduler
from random import random
import time
import uuid
from confluent_kafka import Consumer, KafkaException
# Function to be run as a daemon
def daemon_task_1(**kwargs):
print("Daemon Task 1 is running")
while True:
queue.put(random())
def daemon_task_2(**kwargs):
print("Daemon Task 2 is running")
while True:
data = queue.get()
print(data)
def main():
executors = {
'default': ProcessPoolExecutor(max_workers=8,)
}
job_defaults = {
'coalesce': False,
'max_instances': 3,
'misfire_grace_time': None
}
# Scheduler for the daemon tasks
scheduler = BackgroundScheduler(
executors=executors,
job_defaults=job_defaults,
daemonic=True,
daemon=True
)
scheduler.add_job(daemon_task_1, trigger='date')
scheduler.add_job(daemon_task_2, trigger='date')
global queue
queue = SimpleQueue()
# Start the scheduler
scheduler.start()
try:
while True:
time.sleep(1)
except (KeyboardInterrupt, SystemExit):
pass
finally:
scheduler.shutdown()
if __name__ == '__main__':
main()
When I run it with python 3.8 it works fine:
$ python -V;python aps-mixed.py | more
Python 3.8.18
Daemon Task 2 is running
0.7147972065089533
0.9088275181513735
0.770493801418164
0.0914193457803868
0.7720603694858935
0.48302319930441084
...
But with 3.11 I get this error, why?
Python 3.11.11
Job "daemon_task_1 (trigger: date[2025-01-29 16:47:01 CET], next run at: 2025-01-29 16:47:01 CET)" raised an exception
Traceback (most recent call last):
File "/home/davidp/.pyenv/versions/mlserver2/lib/python3.11/site-packages/apscheduler/executors/base.py", line 131, in run_job
retval = job.func(*job.args, **job.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/davidp/gitlab/geoapps-ta-study/kafka/aps-mixed.py", line 15, in daemon_task_1
queue.put(random())
^^^^^
NameError: name 'queue' is not defined
Job "daemon_task_2 (trigger: date[2025-01-29 16:47:01 CET], next run at: 2025-01-29 16:47:01 CET)" raised an exception
Traceback (most recent call last):
File "/home/davidp/.pyenv/versions/mlserver2/lib/python3.11/site-packages/apscheduler/executors/base.py", line 131, in run_job
retval = job.func(*job.args, **job.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/davidp/gitlab/geoapps-ta-study/kafka/aps-mixed.py", line 20, in daemon_task_2
data = queue.get()
^^^^^
NameError: name 'queue' is not defined
How can we reproduce the bug?
Already described in the issue
The text was updated successfully, but these errors were encountered:
davidp1404
changed the title
SimpleQueue sharing acroos multiprocessing differ across python versions
SimpleQueue use differs across python versions
Jan 29, 2025
Things to check first
I have checked that my issue does not already have a solution in the FAQ
I have searched the existing issues and didn't find my bug already reported there
I have checked that my bug is still present in the latest release
Version
APScheduler==3.10.4
What happened?
Using this simple code:
When I run it with python 3.8 it works fine:
But with 3.11 I get this error, why?
How can we reproduce the bug?
Already described in the issue
The text was updated successfully, but these errors were encountered: