You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I need to migrate Bull jobs from one Redis instance to another. There are two parts to this problem:
Pause processing of these jobs.
There will be multiple queues, and only some jobs in each queue should be paused, the rest should be processed as usual. The reason being these jobs are associated with a specific user that we want to migrate to a new data center. Is there a way to prevent a job from being processed without pausing the entire queue? I am thinking to identify these jobs with a unique id, e.g. toMigrate-someUUID. Maybe when processing, the job could be marked as permanently failed?
Add the job to an existing Bull queue on another server, connected to a different Redis instance.
Get the job data and queue name for all paused jobs, then run addJob on the appropriate queues in the new data center.
I'm guessing we can't use a tool like https://redis.io/commands/migrate/ to migrate each job by key as there are other associated Redis data structures that Bull needs to preserve?
Any help would be greatly appreciated! Thanks for all your work ❤️
This discussion was converted from issue #2541 on February 07, 2023 17:02.
Heading
Bold
Italic
Quote
Code
Link
Numbered list
Unordered list
Task list
Attach files
Mention
Reference
Menu
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi Bull team,
I need to migrate Bull jobs from one Redis instance to another. There are two parts to this problem:
Pause processing of these jobs.
There will be multiple queues, and only some jobs in each queue should be paused, the rest should be processed as usual. The reason being these jobs are associated with a specific user that we want to migrate to a new data center. Is there a way to prevent a job from being processed without pausing the entire queue? I am thinking to identify these jobs with a unique id, e.g.
toMigrate-someUUID
. Maybe when processing, the job could be marked as permanently failed?Add the job to an existing Bull queue on another server, connected to a different Redis instance.
Get the job data and queue name for all paused jobs, then run
addJob
on the appropriate queues in the new data center.I'm guessing we can't use a tool like https://redis.io/commands/migrate/ to migrate each job by key as there are other associated Redis data structures that Bull needs to preserve?
Any help would be greatly appreciated! Thanks for all your work ❤️
Beta Was this translation helpful? Give feedback.
All reactions