Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Printed events processed interval does not correspond to real processing time #429

Closed
ppcad opened this issue Jul 20, 2023 · 1 comment · Fixed by #431
Closed

Printed events processed interval does not correspond to real processing time #429

ppcad opened this issue Jul 20, 2023 · 1 comment · Fixed by #431
Labels
bug Something isn't working

Comments

@ppcad
Copy link
Collaborator

ppcad commented Jul 20, 2023

Logprep can print the number of processed events for a configured interval.
However, the printed interval is smaller than the actual processing time if processing one event takes longer than that interval.
This can happen when Logprep is starting if it has to load a lot of rules and the print interval is smaller than the time it takes to load them.

Expected behavior
The printed interval for processed events corresponds to the real processing time.

Current behavior
The printed interval for processed can be smaller the real processing time.

Steps to reproduce
Set a small "print_processed_period" and load a lot of rules.

Environment

Logprep version: latest
Python version: 3.11

Possible solution
This might be solved in multiple ways:

  1. Print the real time it took instead of the configured interval time. This means the printed interval might increase if it took longer than configured.
  2. Ensure the processed count gets always printed at the interval, even if events are still being processed.
  3. Ensure everything is loaded before the counting starts.
@ppcad ppcad added the bug Something isn't working label Jul 20, 2023
@dtrai2 dtrai2 linked a pull request Jul 21, 2023 that will close this issue
@dtrai2
Copy link
Collaborator

dtrai2 commented Jul 21, 2023

I just opened a pull request, maybe you can check if that solves the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants