[Bug]: Workers functioning locally but not with Cloud Redis instance

[romain-hasseveldt]

Version

v5.16.0

Platform

NodeJS

What happened?

Hi there, I'm a bit lost 😅.

I’m using BullMQ in my back-end (Strapi) to manage sending reminders. I have two queues (time-reminders and cron-reminders), each with its own worker. Everything works perfectly when I'm pointing to my local Redis instance.

The problem arises when I switch to the Redis Cloud instance I’ve set up. The workers stop functioning, and I’m not sure what’s going wrong.

I’m happy to provide more details if needed.

Any guidance or ideas would be greatly appreciated!

How to reproduce.

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[manast]

When you say "stop working" what do you mean? they work for some time, 1 minute, 1 day? and then stop processing jobs, or they never processed jobs at all? Have you checked that the workers are connected to Redis?

[romain-hasseveldt]

Hello @manast,

I'm following up on my initial message now that I have more details.

I have two queues:

export const cronRemindersQueue = new Queue(CRON_REMINDERS_QUEUE_NAME, {
  connection: CONNECTION,
});

export const timeRemindersQueue = new Queue(TIME_REMINDERS_QUEUE_NAME, {
  connection: CONNECTION,
});

And two associated workers:

const processor: Processor<
  CronReminder<NotificationKey> | TimeReminder<NotificationKey>
> = async (job) => {
  sendNotification(job.data.params);
};

const options: WorkerOptions = {
  connection: CONNECTION,
  autorun: false,
};

export const cronReminderWorker = new Worker(
  CRON_REMINDERS_QUEUE_NAME,
  processor,
  options
);

export const timeReminderWorker = new Worker(
  TIME_REMINDERS_QUEUE_NAME,
  processor,
  options
);

The time reminder queue handles simple jobs, and I just realized that locally or in production, it doesn't cause any issues.

export const enqueueTimeReminder = async <T extends NotificationKey>(
  reminder: TimeReminder<T>
) => {
  try {
    await timeRemindersQueue.add("send-time-reminder", reminder, {
      jobId: reminder.id,
      delay: reminder.time.getTime() - Date.now(),
      attempts: 3,
    });
  } catch (err) {
    console.log(err);
  }
};

export const dequeueTimeReminder = async (id: string) => {
  try {
    await timeRemindersQueue.remove(id);
  } catch (err) {
    console.log(err);
  }
};

However, the cron reminder queue handles job schedulers, and while it works perfectly locally, it causes issues when pointing to my Cloud Redis instance. I'm unsure if nothing is happening or if the cron expressions are causing the problem in production but not locally?

export const enqueueCronReminder = async <T extends NotificationKey>(
  reminder: CronReminder<T>
) => {
  try {
    await cronRemindersQueue.upsertJobScheduler(
      reminder.id,
      { pattern: reminder.cron },
      { name: "send-cron-reminder", data: reminder, opts: { attempts: 3 } }
    );
  } catch (err) {
    console.log(err);
  }
};

export const dequeueCronReminder = async (id: string) => {
  try {
    await cronRemindersQueue.removeJobScheduler(id);
  } catch (err) {
    console.log(err);
  }
};

Let me know if you'd like any further info.

[manast]

What does "it causes issues" mean?

[romain-hasseveldt]

The worker is not triggered when it should be. I don't know if it ever will be or if it will be at some point, but definitely not at the right time.

[manast]

Check on the delayed jobs to see if there are any and when they have been scheduled.

[romain-hasseveldt]

Okay, so two things:

First, I wasn’t setting the time zone correctly on the scheduler, which caused a mismatch between when I expected the worker to run and when it was actually scheduled.
Second, I think I might have been handling the placement of my queues and workers incorrectly within the Node.js processes, but my knowledge is too limited to say for sure.

But in the end, everything is working properly now.

[manast]

Happy that it did work at the end, I will put more effort in documenting the timezone parameter.