Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not use [data_stream_template_use_index_patterns_wildcard] parameter for Elasticsearch Fluentd output #1941

Open
Killroad opened this issue Feb 5, 2025 · 0 comments
Labels
bug Something isn't working
Milestone

Comments

@Killroad
Copy link

Killroad commented Feb 5, 2025

Describe the bug:
When attempting to use the parameter data_stream_template_use_index_patterns_wildcard in the configuration for Elasticsearch (within ClusterOutput), the fluentd configuration inside the pod does not reflect it. The parameter is not added to the configuration despite the latest version of fluent-plugin-elasticsearch (5.4.3) being used in the logging-operator, which supports this parameter.

I suspect that this parameter is missing from the implementation here in both 4.8.0 (my current version) and the latest v5.1.1.

Expected behaviour:
The parameter data_stream_template_use_index_patterns_wildcard should be applied in the fluentd configuration inside the pod when set in Elasticsearch section of ClusterOutput

Steps to reproduce the bug:

  1. Use a ClusterOutput definition that includes the parameter data_stream_template_use_index_patterns_wildcard.
  2. Apply the ClusterOutput configuration to a Kubernetes cluster.
  3. Inspect the fluentd configuration inside the pod.
  4. Observe that the parameter is not added to the fluentd configuration.

Environment details:

  • Kubernetes version (e.g. v1.15.2): 1.29.4
  • Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): AKS
  • logging-operator version (e.g. 2.1.1): 4.8.0
  • Install method (e.g. helm or static manifests): logging-operator installed with helm and ClusterOutput/ClusterFlow with raw helm charts
- apiVersion: logging.banzaicloud.io/v1beta1
  kind: ClusterOutput
  metadata:
    name: es-clusteroutput-app
    namespace: logging
  spec:
    elasticsearch:
      host: elasticsearch-proxy-host.example.com
      port: 443
      scheme: https
      ssl_verify: true
      ssl_version: TLSv1_2
      user: elastic
      password:
        valueFrom:
          secretKeyRef:
            name: es-password
            key: password
      data_stream_template_use_index_patterns_wildcard: false
      data_stream_enable: true
      data_stream_name: test

/kind bug

@Killroad Killroad added the bug Something isn't working label Feb 5, 2025
@csatib02 csatib02 added this to the 5.x milestone Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants