Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: streams.events.maxLen queue option has no effect #2847

Open
1 task done
f-w opened this issue Oct 20, 2024 · 1 comment
Open
1 task done

[Bug]: streams.events.maxLen queue option has no effect #2847

f-w opened this issue Oct 20, 2024 · 1 comment

Comments

@f-w
Copy link

f-w commented Oct 20, 2024

Version

v5.21.1

Platform

NodeJS

What happened?

I set queue option streams.events.maxLen to 10, confirmed Redis key bull:<queueName>:meta field opts.maxLenEvents is 10. Still bull:<queueName>:events count exceeds 10.

How to reproduce.

  1. get redis running on 127.0.0.1:6379
  2. put following code in a .ts file and run using ts-node.
    import { Queue, Worker } from 'bullmq';
    
    const myQueue = new Queue('foo', {
      connection: {
        host: '127.0.0.1',
        port: 6379,
      },
      defaultJobOptions: {
        removeOnComplete: true,
      },
      streams: {
        events: {
          maxLen: 10,
        },
      },
    });
    
    async function addJobs() {
      for (let i = 0; i < 10; i++) {
        await myQueue.add('myJobName', {
          count: i,
        });
      }
    }
    
    addJobs();
    new Worker(
      'foo',
      async (job) => {
        console.log({ ...job.data, time: new Date() });
      },
      {
        connection: {
          host: '127.0.0.1',
          port: 6379,
        },
      },
    );
  3. Inspect Redis bull.foo.events key. Will exceeds 10 when all jobs are processed.

Code of Conduct

  • I agree to follow this project's Code of Conduct
@f-w f-w added the bug Something isn't working label Oct 20, 2024
@manast
Copy link
Contributor

manast commented Oct 20, 2024

Unfortunately it is not possible in Redis to trim the streams to an exact value as that would be too slow, so we use "nearly exact trimming": https://redis.io/docs/latest/commands/xtrim/

@manast manast added better docs and removed bug Something isn't working labels Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants