How to remove _id, _index, _type, num_matches, num_hits and @timestamp from elastialert rules

[MohdRashid01]

Hi All,

Below is the content which im getting in my gmail inbox. I want to

remove _id, _index, _type, num_matches, num_hits and @timestamp

all this only i want message to be include in my email triggering everytime.

Below is getting into my gmail inbox

@timestamp: 2021-05-24T11:16:06Z
_id: 2S0WnnkBz7SOxaiw1TZk
_index: logstash-2021.05.24
_type: _doc
message: <30>May 24 11:16:06 fx-prod-1 prod_fx-control-plane.1.knel5yam 2021-05-24 11:16:06.926 INFO 1 --- [nio-8080-exec-9] com.fxlabs.fxt.rest.run.RunController : Find Latest by job id [8a8089ba777311370177734530902ec8] org [8a8081066e02d6a2016e04eacd2005c7] principal [8a808155647d283a01647d7c5e0d07ba]
num_hits: 8
num_matches: 1

and

Here is the rules.yaml file which i used to trigger email for issue

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
# es_host: elasticsearch.example.com

es_host: elasticsearch.test.com
# (Optional)
# Elasticsearch port
#es_port: 14900
es_port: 9200

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword
es_username: testelastic
es_password: xxxx


# (Required)
# Rule name, must be unique
name: Exception Alert

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
#type: blacklist
type: any
include: ["message"]

# (Required)
# Index to search, wildcard supported
#index: logstash*
index: filebeat*

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
#num_events: 1

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
#  hours: 1
#  minutes: 1
  seconds: 1

#compare_key: "message"

#blacklist:

#- "error"

realert:
  minutes: 5

 # This is send out all matches in one email
# aggregation:
#  minutes: 1

#  seconds: 0
#filter:
#- term:
#    message: "[error]"

#filter:
# - match:
 #   message: "job"

filter:
 - query:
      query_string:
 #       query: "message: exception AND  message: control"
        query: "message: job"
#filter:
#- query:
#   query_string:
#    query: "message: error"

alert:

- "slack"
- "email"


#- slack
slack_webhook_url: "https://hooks.slack.com/services/xxxxxxxx"
#slack_username_override: "ElastAlert"
slack_username_override: "Mohd Rashid"

#- email
email: ["[email protected]"]
smtp_host: "smtp.gmail.com"
smtp_port: "587"
from_addr: "[email protected]"

How to do it,

[nsano-rururu]

look
https://elastalert2.readthedocs.io/en/latest/ruletypes.html#alert-subject

[jertel]

@MohdRashid01 Good, now you can start to customize the alert_text, using variable substitution.

Ex:

alert_text_type: alert_text_only
alert_subject: "This is my alert subject"
alert_text: "Alert! {0}"
alert_text_args:
- message

@nsano-rururu Not sure I'm following. The way I'm reading the logic it looks correct. If the alert type is neither text-only nor text-jinja then include the rule text, top-counts, and match-items. That seems correct to me.

[nsano-rururu]

@jertel

my misunderstanding. I'm sorry.
There was no problem in operation.

[w-HA]

to help our config ( change fields regarding yours )

alert_subject: "{0} {1} {2}"
alert_subject_args: ["service","resource","text"]

alert_text: |
Begin problem at {0}
Value : {1}
Tags : {2}
alert_text_args: ["createTime","value","tags"]
alert_text_type: alert_text_only

w-HA

[MohdRashid01]

Hi Thanks
i have try to add but its not working

[jertel]

If you'd like more support you will need to provide further details such as screenshots of the emails after your latest attempts, elastalert logs, configurations, and anything else that you think will give us what we need to assist you.

[MohdRashid01]

Here is the email screenshot...yellow highlighted should not show in email....ONLY MESSAGES should show in email

Here is the rules.yml file store in kibana of elasteralert

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
# es_host: elasticsearch.example.com

es_host: elasticsearch.test.com
# (Optional)
# Elasticsearch port
#es_port: 14900
es_port: 9200

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword
es_username: testelastic
es_password: xxxx


# (Required)
# Rule name, must be unique
name: Exception Alert

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
#type: blacklist
type: any
include: ["message"]

 

# (Required)
# Index to search, wildcard supported
#index: logstash*
index: filebeat*

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
#num_events: 1

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
#  hours: 1
#  minutes: 1
  seconds: 1

#compare_key: "message"

#blacklist:

#- "error"

realert:
  minutes: 5

 # This is send out all matches in one email
# aggregation:
#  minutes: 1

#  seconds: 0
#filter:
#- term:
#    message: "[error]"

#filter:
# - match:
 #   message: "job"

filter:
 - query:
      query_string:
 #       query: "message: exception AND  message: control"
        query: "message: job"
#filter:
#- query:
#   query_string:
#    query: "message: error"

alert:

- "slack"
- "email"


#- slack
slack_webhook_url: "https://hooks.slack.com/services/xxxxxxx"
#slack_username_override: "ElastAlert"
slack_username_override: "Mohd Rashid"

#- email
email: ["[email protected]"]
smtp_host: "smtp.gmail.com"
smtp_port: "587"
from_addr: "[email protected]"

If anything goes wrong do let know i have add include: ["message"] in my rules after that it showing above one screenshot high listed in yellow

[jertel]

Your rule does not have the required fields that we have been telling you to add.

[MohdRashid01]

Where to add and How to add it

I want to remove this all fields (remove _id, _index, _type, num_matches, num_hits and @timestamp ) all this should not shown in email triggering

[jertel]

Follow these instructions: #189 (reply in thread)

[MohdRashid01]

What instructions where are the instructions plse again u text me

[nsano-rururu]

I've answered, so I don't think I need any further support.