Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ESF] Add ability to disable document enrichment #157

Open
tommyers-elastic opened this issue Sep 26, 2022 · 3 comments
Open

[ESF] Add ability to disable document enrichment #157

tommyers-elastic opened this issue Sep 26, 2022 · 3 comments
Assignees

Comments

@tommyers-elastic
Copy link

This is an ask from the Elastic Cloud Monitoring team, in order to enable Logstash output for our Elastic Serverless Forwarder product. elastic/elastic-serverless-forwarder#112.

Based on discussions we have had already with the Logstash team, we have identified the HTTP input to be the most suitable way to send data from our AWS serverless collector to Logstash. The issue is that the plugin enriches input data with additional fields such as host.ip and http.version etc. We would like a way to disable this behaviour, leaving the input documents untouched.

A similar issue has been raised on the elastic agent plugin. https://github.com/logstash-plugins/logstash-input-elastic_agent/issues/3.

@girodav
Copy link

girodav commented Sep 28, 2022

Working directly on this plugin, we can avoid adding additional metadata introducing a new config value, and use it as flag to add / not add the header medatata, together with event.original

We can define this flag as:

  1. An additional parameter to be written in logstash configs
  2. An entry that is added by ESF to the event, before sending it to Logstash

I personally prefer option 1 as I think it's closer to what an user would expect, and users would not need to add extra dummy data to their logs just for this purpose.

I am currently working on a PR to add this functionality #158

@girodav
Copy link

girodav commented Sep 28, 2022

cc @jsvd

@roaksoax roaksoax changed the title Add ability to disable document enrichment [ESF] Add ability to disable document enrichment Oct 25, 2022
@girodav
Copy link

girodav commented Feb 16, 2023

I think we can close this as not needed anymore :) (cc @yaauie @roaksoax @tommyers-elastic )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants