Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.16](backport #41244) Read journal entries from all boots #41404

Merged
merged 1 commit into from
Oct 24, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 23, 2024

Proposed commit message

Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments.

This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

## Disruptive User Impact

Author's Checklist

How to test this PR locally

1. Run the tests

cd filebeat/input/journald
go test -run=TestInputCanReadAllBoots

2. Run Filebeat reading filebeat/input/journald/testdata/multiple-boots.journal

There must be 6 entries, you can see the plaintext entries by looking at filebeat/input/journald/testdata/multiple-boots.export or by running:

journalctl --file filebeat/input/journald/testdata/multiple-boots.export

3. Fully manual test

  1. Run the journald input on a machine (or a journal file) that has got messages from more than one boot
  2. Ensure all messages in the journal are correctly ingested.

Related issues

## Use cases
## Screenshots
## Logs


This is an automatic backport of pull request #41244 done by Mergify.

Some versions of journalctl will only return messages from the current
boot when --follow is passed, it will even ignore the cursor or date
arguments.

This commit reads messages from all boots by first calling journalctl
without the --follow flag, reading all entries and once it
successfully exits, then we restart journalctl with the cursor and the
--follow flag.

The parser test is upsted to use ndjson parser instead of
multiline because the multiline parser can have issues when journald
input is reading from files. There is a corner case where the
journalctl exits successfully and the reader goroutine gets an error,
this makes Next to return early, making the multiline to also return
early.

TestJournaldInput assumed journalctl was run only once, however that
has changed recently. The test is updated to accommodate for that and rename
to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which
better reflects what it is actually testing.

---------

Co-authored-by: Pierre HILBERT <[email protected]>
(cherry picked from commit d21ed32)
@mergify mergify bot requested a review from a team as a code owner October 23, 2024 19:22
@mergify mergify bot added the backport label Oct 23, 2024
@mergify mergify bot requested review from belimawr and khushijain21 and removed request for a team October 23, 2024 19:22
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 23, 2024
@botelastic
Copy link

botelastic bot commented Oct 23, 2024

This pull request doesn't have a Team:<team> label.

@belimawr
Copy link
Contributor

The build seems to be failing because of:

2024-10-23 20:42:19 UTC	helper/socket/ptable_linux.go:23:2: unrecognized import path "kernel.org/pub/linux/libs/security/libcap/cap": https fetch: Get "https://kernel.org/pub/linux/libs/security/libcap/cap?go-get=1": EOF
2024-10-23 20:42:19 UTC	
2024-10-23 20:42:19 UTC	=== Errors
2024-10-23 20:42:19 UTC	helper/socket/ptable_linux.go:23:2: unrecognized import path "kernel.org/pub/linux/libs/security/libcap/cap": https fetch: Get "https://kernel.org/pub/linux/libs/security/libcap/cap?go-get=1": EOF

On my machine I can compile/run the tests without problems... I'll just re-run the tests

@pierrehilbert pierrehilbert merged commit 3fcfd9a into 8.16 Oct 24, 2024
140 of 142 checks passed
@pierrehilbert pierrehilbert deleted the mergify/bp/8.16/pr-41244 branch October 24, 2024 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants