Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch container logs input to /var/log/pod/* path #12500

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Switch container logs input to /var/log/pod/* path #12500

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
656b08b
Switch container logs input to /var/log/pod/* path
pchila Jan 28, 2025
b8d679d
Add changelong and bump patch version
pchila Jan 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/kubernetes/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: 1.81.0
changes:
- description: Switch k8s input paths to /var/log/pods/* to ingest rotated container logs
type: bugfix
link: https://github.com/elastic/integrations/pull/12500
- version: 1.80.0
changes:
- description: Add support for Kibana `9.0.0`
Expand Down
13 changes: 1 addition & 12 deletions packages/kubernetes/data_stream/container_logs/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,4 @@
{{!
Because we use `${kubernetes.container.id}` in the ID, an instance
of this input will be generated for every container, so `paths` must
always be unique per container otherwise there will be data
duplication, at the extreme this will overload Filebeat and cause
data ingestion issues.

This ID is also mentioned in the `README.md, so if it is changed, it
needs to be updated there as well.
}}

id: kubernetes-container-logs-${kubernetes.pod.name}-${kubernetes.container.id}
id: kubernetes-container-logs-${kubernetes.namespace}-${kubernetes.pod.name}-${kubernetes.container.id}
paths:
{{#each paths}}
- {{this}}
Expand Down
9 changes: 1 addition & 8 deletions packages/kubernetes/data_stream/container_logs/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,7 @@ streams:
title: Kubernetes container log path
multi: true
default:
- /var/log/containers/*${kubernetes.container.id}.log
description: >-
For every container the Elastic-Agent can see (usually every
container on the node) an instance of the input will be
created harvesting all paths defined here, even if
the paths contain no variable! Refer to the [integration
documentation](https://www.elastic.co/guide/en/integrations/current/kubernetes.html)
for more details.
- /var/log/pods/${kubernetes.namespace}_${kubernetes.pod.name}_${kubernetes.pod.uid}/${kubernetes.container.name}/*.log
- name: symlinks
type: bool
title: Use Symlinks
Expand Down
2 changes: 1 addition & 1 deletion packages/kubernetes/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.1.2
name: kubernetes
title: Kubernetes
version: 1.80.0
version: 1.81.0
description: Collect logs and metrics from Kubernetes clusters with Elastic Agent.
type: integration
categories:
Expand Down