Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to embed json fields metadata inside the log payload #46

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rishiyemme
Copy link
Collaborator

Adding a boolean option moveFieldsIntoPayload to inject specified fields into the log JSON payload to allow for dynamic auto JSON parsing in Sumo. Enabling this option would also prevent fields from being sent in the HTTP request to Sumo.

@rishiyemme rishiyemme marked this pull request as ready for review September 26, 2022 20:47
Copy link
Collaborator

@vsinghal13 vsinghal13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
We need to explicitly call out in the release notes that enabling this feature may be a breaking change if people have existing searches/dashboards set up based on json fields.

@rishiyemme
Copy link
Collaborator Author

rishiyemme commented Oct 5, 2022

Pushed some changes while waiting for some customer feedback on the log format when moveFieldsIntoPayload = true.
Per the latest changes, whenever moveFieldsIntoPayload is true, we always format the log payload into a JSON log format looking like

{"timestamp":1664396202215,"log":"{"timestamp":1654016401446,"PID":"3","USER":"root","MEM":"0.0","CPU":"0.0"}","metadata":{"bigintfield":"228930314431312345","boolfield":"true","intfield":"100","stringfield":"stringvalue"}}

essentially ignoring the payloadText field. The metadata field will include all fields.

Otherwise when moveFieldsIntoPayload is false, we revert to the previous behavior.

@vsinghal13
Copy link
Collaborator

Otherwise when moveFieldsIntoPayload is true, we revert to the previous behavior.
@rishiyemme you meant when it is set to false, we revert to previous behavior.

@@ -0,0 +1,24 @@
include "application"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the different between this file and the test-fields-in-payload-payloadText file?

Probably the other one is missing payloadWrapperKey?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants