[Bug]: Prioritized jobs doesn't process before non-prioritized ones

[BastienGenin]

Version

v3.5.6

Platform

NodeJS

What happened?

Hello, I've encountered a problem with prioritized jobs that I can't figure out.

Let's say I have a queue full of many jobs waiting to be processed by the same worker in the same queue, all these jobs have a priority of 2.

When I add a new job to the queue with a non-specified priority (the highest priority possible if I've read the documentation correctly), it doesn't run immediately after the end of the process of a job with a priority of 2. Could you explain why this is?

Is there a connection with the concurrency settings of my worker? I didn't specify a concurrency when I created it.

// Queue
new Queue(
  "test",
  { connection: { ... } }
)

// Worker
new Worker(
  "test",
  () => { /* my processor code here */ },
  { 
    connection: { ... },
    lockDuration: 300000,
    maxStalledCount: 3,
    stalledInterval: 60000,
  }
)

Should I just upgrade to bullMQ 4/5.X.X ?

Thanks in advance.

How to reproduce.

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[manast]

A job without priority should process before jobs with priorities. I do not know if there is something specific with your code that breaks this behaviour or a bug in your version of BullMQ. You should definitely upgrade as newer versions have a much better implementation of prioritised jobs, however if you have queues in production you may want to use new queues as your version is quite old and there may have incompatibilities with the newer ones.

[BastienGenin]

Thanks for your quick response.

I will bump BullMQ to v5, write some test on my codebase and keep you updated if I encounter this again.

[roggervalf]

hi @BastienGenin, prioritized state was separated from wait state since v4 https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/changelogs/changelog-v4.md#400-2023-06-21, the behavior that you are facing is normal in old versiones from 1 to 3