Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to main: feat: adds TRACE logging level and adjust log levels for escrows #933

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kentbull
Copy link
Contributor

This PR adds the logging changes made to 1.1.27+ and 1.2.4+ from here, here, and here.

The result of this PR is:

  • Adds TRACE logging level in keri.help.init module.
  • Moves escrow logs to TRACE logging level.
  • Moves event body logging to DEBUG level.
  • INFO log level contains SAID or prefix of events rather than an event body.
  • Uniformly uses log substitution where it would improve performance.
  • Uniformly uses f-strings where performance is equivalent yet readability is enhanced.
  • serder.pretty() used in favor of serder.ked or json.dumps(serder.ked, indent=1)
  • Added escrow-specific strings to each escrow to clarify in logs which escrow produced which log message.
  • Various spelling corrections and wording cleanups like Kevery process to Kevery

@kentbull kentbull force-pushed the main-add-trace-logging-and-consistent-logging branch from 9204cdd to 9d6c44f Compare January 29, 2025 00:05
Copy link
Member

@pfeairheller pfeairheller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment and request as the other log bugs:

Most log messages are unique enough that a simple search of the code base finds the one you are looking for. And if they aren't, I'd rather update the few that are to be more specific / descriptive and avoid this bespoke cataloging of log messages.

@SmithSamuelM
Copy link
Collaborator

SmithSamuelM commented Jan 29, 2025

@pfeairheller @kentbull

F-strings in log message that use the python std library logging module are never more performant than % format string with parameter list pattern.

The reason is that the % format is never actualized unless the log level is met. Which is happens deep inside of the logging module. The parameter list and format string are just passed along. So whenever the logger does not actually log, the format string and parameters are not evaluated. Whereas if you pass in instead of the format string an f-string the f-string must be evaluated and expanded first before the logging module ever gets to decide if it needs to log at all. So you incur the cost of expanding the f-string every time not just sometimes. The performace difference between % format strings and f-strings is very small. f-strings can be a little be faster than % strings (this is only true in the latest versions of python3) but when you include the overhead of evaluating the f-string every time versus the % string only gets evaluated when it needs to be logged. Replacing % format strings with f-strings is counter productive in general for logging module use cases.

@SmithSamuelM
Copy link
Collaborator

Where it makes sense to replace % format strings with f-strings is in exception raises. The f-string in an exception raise is only expanded when the raise is called so the performance and clarity benefit of an f-string is worthwhile.

Historically, I did not use f-strings instead of $ format strings because when f-strings first came out they were significantly slower than % format strings. But as of python 3.10 or thereabouts they optimized them to be a little faster. So I have started replacing the % format strings in raise messages with f-strings. F-strings also tend to be more compact.

@kentbull kentbull changed the title feat: adds TRACE logging level and adjust log levels for escrows to main: feat: adds TRACE logging level and adjust log levels for escrows Jan 29, 2025
@kentbull
Copy link
Contributor Author

kentbull commented Jan 29, 2025

F-strings in log message that use the python std library logging module are never more performant than % format string with parameter list pattern.

Where it makes sense to replace % format strings with f-strings is in exception raises.

@SmithSamuelM yes, I agree with what you have said. In my PR I made an attempt to always use the % format string on every log message that was not also part of an exception raise. In other words, I only used F-strings when there was an exception raised and otherwise used the % format strings in log messages.

This is why I changed some of the left over F-strings that had something like logger.debug(f"...\n{serder.pretty()}\n") to be logger.debug("...\n%s\n", serder.pretty()).

@kentbull
Copy link
Contributor Author

@pfeairheller

Most log messages are unique enough that a simple search of the code base finds the one you are looking for.

To bring in my comments from the other PRs

It was very useful to me while debugging multisig to be able to read the log output without having to go search. The reason I added the bespoke cataloging of log messages was because the bespoke labels added context that helped me think through the problem without having to constantly search the codebase.

IMO it's a set of tradeoffs. Add a bit more length to log messages, gain the ability to quickly understand what is going on.

As the person who wrote many, if not most of them, them I can see why you would lean in favor of avoiding the bespoke labels because you have memorized the context and can follow the logic often without having to refer back to the codebase.

As a newcomer to the codebase I found it tremendously helpful to have that specific tagging in the label. I imagine such labels will help other newcomers quickly understand the meaning of the log messages.

If there's a different or better way to accomplish this understanding I'm all ears. Adding prefixes to the log messages really helped especially when I wanted to distinguish between things like the KERIA Anchorer and the KERIpy Anchorer and similar cases.

I like being able to follow the log messages without having to refer back to the codebase, though that is personal preference. I'll go remove the prefixes if you'd rather have them be removed.

@kentbull kentbull force-pushed the main-add-trace-logging-and-consistent-logging branch from 461fbeb to 7de6f16 Compare February 5, 2025 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants