[Core] Avoid metrics log noise when idle

[russellb]

1a789ae [Core] Avoid metrics log noise when idle

commit 1a789ae
Author: Russell Bryant [email protected]
Date: Thu Sep 26 20:56:41 2024 +0000

[Core] Avoid metrics log noise when idle

Prior to this change, there was always a log message every 10 seconds,
even when the system is idle. The noise when the metrics are all zeros
doesn't seem very interesting, so this change makes it so the log
message is only emitted while active. It will log a message whenever
metrics are non-zero, or when returning to idle from non-idle.
Repeated messages during idle now go to DEBUG instead of INFO.

Signed-off-by: Russell Bryant <[email protected]>

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[vrdn-23]

@russellb I think this is a great small QoL improvement. Is this still something get worked on?

[russellb]

@russellb I think this is a great small QoL improvement. Is this still something get worked on?

It's ready for review when someone has time.

[stmcginnis]

This would be great to have!

[vrdn-23]

@simon-mo @mgoin Is this something we can merge before the next release?

[mgoin]

I personally like keeping the log as a sort of "heartbeat" to let me know the server is functioning fine even if there is nothing running. So I'm okay with this but would like another committer who feels more strongly about this as an improvement

[vrdn-23]

I personally like keeping the log as a sort of "heartbeat" to let me know the server is functioning fine even if there is nothing running

I think the functionality of /health endpoint should be what indicates this rather than a log message showing idle numbers.

[russellb]

I personally like keeping the log as a sort of "heartbeat" to let me know the server is functioning fine even if there is nothing running

I think the functionality of /health endpoint should be what indicates this rather than a log message showing idle numbers.

A compromise could be to emit them as DEBUG instead of INFO once the system goes idle.

[vrdn-23]

A compromise could be to emit them as DEBUG instead of INFO once the system goes idle.

That sounds like a valid solution to me. Should also enable users who would prefer to have the logs enable it. Thoughts @mgoin ?

[russellb]

A compromise could be to emit them as DEBUG instead of INFO once the system goes idle.

That sounds like a valid solution to me. Should also enable users who would prefer to have the logs enable it. Thoughts @mgoin ?

I updated the PR to do this.

[vrdn-23]

@mgoin can we go ahead with this approach?

[mgoin]

Yeah let's go with this, thanks!

[comaniac]

Just to check, do we have other DEBUG logging? Like if I enabled DEBUG would I see lots of other logs and even slow the performance?