Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new Grok Pattern to match multiline strings, e.g. Stacktraces #314

Open
stoerr opened this issue Jul 7, 2022 · 0 comments
Open

new Grok Pattern to match multiline strings, e.g. Stacktraces #314

stoerr opened this issue Jul 7, 2022 · 0 comments

Comments

@stoerr
Copy link

stoerr commented Jul 7, 2022

I'd like to suggest introducing a pattern like ((?m).*) that matches a multiline string, such as what you'd get if you apply the multline filter before the grok filter in logstash to capture e.g. stacktraces in a Java logfile.

Background: When testing my update of GrokConstructor, I noticed that there is still no pattern in the Grok pattern library that would match a stacktrace, as in https://grokconstructor.appspot.com/do/construction?example=0 . Thus it's not possible for the incremental construction to suggest something sensible that'd match the full message for something like that:

2013-02-28 09:57:56,668 ERROR SomeCallLogger - ESS10005 Cpc portalservices: Exception caught while writing log messege to MEA Call:  {}
java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist

	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)

Here you are pretty much stuck in the construction process after
\A%{TIMESTAMP_ISO8601}%{SPACE}%{LOGLEVEL}%{SPACE}%{JAVACLASS}%{SPACE}%{JAVALOGMESSAGE}
and has to use a handmade regular expression like ((?m).*) to match the stacktrace, which likely isn't something everybody can type in without some research.

So my suggestion would be to introduce such a pattern ((?m).*) in the grok pattern library. Not sure what to name it: it could be named STACKTRACE, but I've often seen log messages with several lines that do not have a stacktrace, so something like MULTILINE_REST, GREEDY_MULTILINE, MULTILINE_DATA might be more appropriate.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant