kafka

Kafka Monitoring Template with Telegraf and Jolokia

Visualized insights

Requirements

• Set environment variables on host/s from which Telegraf will be sending metrics. These are:
  • INFLUX_ORG, INFLUX_TOKEN, INFLUX_HOST, and INFLUX_BUCKET
• Jolokia JVM Agent
• Apache Kafka
• Apache Zookeeper

Note: Kafka and Zookeper can be easily obtained and managed through the open source Confluent Platform.

Pre-work

• This template does not require -- and therefore support -- TLS configurations. If you have TLS enabled for Jolokia/Zookeeper, you will want to provide that information to the Telegraf configuration. Simply follow this for Jolokia and this for Zookeeper. Comprehensive TLS docs as it pertains to Telegraf can be found here.

Telegraf inputs

• Jolokia
• Zookeeper

Schema

Note: Fields are verbose so the byte size of this dataset is larger than average.

• Telegraf metrics (lines of Line Protocol): 143
• Mean tags per line': 3.50
• Mean fields per line': 9.90
• Tags mode': 3
• Fields mode': 4
• Tags median': 3
• Fields median': 4
• Measurements: 8

Measurements

• kafka_broker (still have to add Type=DelayedOperationPurgatory)
  • Tags
    • host
    • jolokia_agent_url
    • topic
  • Fields
    • BytesInPerSec_FifteenMinuteRate (come back)
• kafka_controller
• kafka_replica_controller
• kafka_network
• zk_client (Kafka perspective)

add ZK metrics: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zookeeper#metrics