Skip to content

Latest commit

 

History

History
263 lines (214 loc) · 16.3 KB

README.md

File metadata and controls

263 lines (214 loc) · 16.3 KB

Cardano Validator Watcher

Cardano Validator Watcher is a Prometheus exporter designed to help you monitor your Cardano validator nodes. It periodically collects essential metrics, providing valuable insights into the performance and health of your infrastructure. These exported metrics can be used to build monitoring dashboards (e.g with Grafana) and configure alerting rules to stay notified when something goes wrong.

banner

Dependencies

This project use the following dependencies:

  • BlockFrost. You need to have a account and a subscription.
  • cncli to calculate the slot leaders.
  • cardano-cli to query additional data from a RPC node.
  • A valid RPC node.
  • Download the Genesis configuration files and provide the VRF signing key for each monitored pool.

Usage

To start the watcher, you need to open a socket connection with your Cardano node. This can be done using the following commands:

With Kubernetes

kubectl port-forward pod/<POD_NAME> 3002 &
socat UNIX-LISTEN:/tmp/cardano.socket,fork,reuseaddr,unlink-early, TCP:127.0.0.1:3002

Without Kubernetes

socat UNIX-LISTEN:/tmp/cardano.socket,fork,reuseaddr,unlink-early, TCP:<IP>:<PORT>

Ensure that you have downloaded the Genesis configuration files. You also need to provide the VRF signing key for each monitored pool.

Then, to start the watcher, execute the following command:

./cardano-validator-watcher [flags]

Flags

Flag Description Default Value Required
--config Path to the configuration file config.yml No
--log-level Log Level info No
--http-server-host Host on which the HTTP server should listen 127.0.0.1 No
--http-server-port Port on which the HTTP server should listen 8080 No
--network Cardano network ID preprod Yes
--database-path Path to the local database mainly used by the Cardano client watcher.db No
--cardano-config-dir Path to the directory where Cardano configuration files are stored /config No
--cardano-timezone Timezone to use with cardano-cli UTC No
--cardano-socket-path Path of the socket to communicate with a Cardano node /var/run/cardano.socket No
--blockfrost-project-id Blockfrost project ID Yes
--blockfrost-endpoint Blockfrost API endpoint Yes
--blockfrost-max-routines Number of routines used by Blockfrost to perform concurrent actions 10 No
--blockfrost-timeout Timeout for requests to the Blockfrost API (in seconds) 60 No
--block-watcher-enabled Enable block watcher True No
--block-watcher-refresh-interval Interval at which the block watcher collects and processes slots (in seconds) 60 No
--pool-watcher-enabled Enable pool watcher True No
--pool-watcher-refresh-interval Interval at which the pool watcher collects data on monitored pools (in seconds) 60 No
--network-watcher-enabled Enable network watcher True No
--network-watcher-refresh-interval Interval at which the network watcher collects data related to network (in seconds) 60 No
--status-watcher-refresh-interval Interval at which the status watcher collects data related to health (in seconds) 15 No

Configuration

The watcher uses a config.yaml file in the current working directory by default. Use the --config flag to specify a different configuration file.

Full Example

pools:
  - instance: "cardano-producer-pool-0"
    id: "pool1abcd1234efgh5678ijklmnopqrstuvwx"
    name: "pool-0"
    key: "config/pool-0.vrf.skey"
  - instance: "cardano-producer-pool-1"
    id: "pool2abcd1234efgh5678ijklmnopqrstuvwx"
    name: "pool-1"
    key: "config/pool-1.vrf.skey"
    exclude: true
    allow-empty-slots: true
network: "mainnet"
block-watcher:
  enabled: true
  refresh-interval: 30
pool-watcher:
  enabled: true
  refresh-interval: 30
network-watcher:
  enabled: true
  refresh-interval: 30
status-watcher:
  enabled: true
  refresh-interval: 15
database:
  path: "watcher.db"
blockfrost:
  project-id: "thisissecret"
  endpoint: "https://cardano-mainnet.blockfrost.io/api/v0"
  max-routines: 10
  timeout: 60
http:
  host: "0.0.0.0"
  port: 8080
cardano:
  config-dir: "config"
  socket-path: "/tmp/cardano.socket"
  timezone: "UTC"

Pools Settings

Field Description Example
instance Name of the instance "cardano-producer-pool-0"
id Pool ID "pool1abcd1234efgh5678ijklmnopqrstuvwx"
name Name of the pool "pool-0"
key Path to the key file "config/pool-0.vrf.skey"
exclude Exclude the pool from monitoring true
allow-empty-slots Pools is allowed to not have slot leaders false

Global Settings

Field Description Example
network Name of the network "mainnet" 
network: "mainnet"

Block Watcher Settings

Field Description Example
enabled Enable block watcher True
refresh-interval Time, in seconds, between two consecutive collections of block data 60 
block-watcher:
  enabled: true
  refresh-interval: 30

Pool Watcher Settings

Field Description Example
enabled Enable pool watcher True
refresh-interval Time, in seconds, between two consecutive collections of pool data 60 
pool-watcher:
  enabled: true
  refresh-interval: 30

Status Watcher Settings

Field Description Example
refresh-interval Time, in seconds, between two consecutive collections of status data 60 
status-watcher:
  refresh-interval: 30

Network Watcher Settings

Field Description Example
enabled Enable network watcher True
refresh-interval Time, in seconds, between two consecutive collections of network data 60 
network-watcher:
  enabled: true
  refresh-interval: 30

Database Settings

Field Description Example
path Path to the database file "watcher.db" 
database:
  path: "watcher.db"

Blockfrost Settings

Field Description Example
project-id Blockfrost project ID "thisissecret"
endpoint Blockfrost API endpoint "https://cardano-mainnet.blockfrost.io/api/v0"
max-routines Number of routines used by Blockfrost to perform concurrent actions 10
timeout Timeout for requests to the Blockfrost API (in seconds) 60 
blockfrost:
  project-id: "thisissecret"
  endpoint: "https://cardano-mainnet.blockfrost.io/api/v0"
  max-routines: 10
  timeout: 60

HTTP Server Settings

Field Description Example
port Port on which the HTTP server should listen 8080 
http:
  host: "0.0.0.0"
  port: 8080

Cardano Settings

Field Description Example
config-dir Path to the directory where Cardano configuration files are stored "config"
socket-path Path of the socket to communicate with a Cardano node "/tmp/cardano.socket"
timezone Timezone to use with cardano-cli "UTC" 
cardano:
  config-dir: "config"
  socket-path: "/tmp/cardano.socket"
  timezone: "UTC"

Metrics

Metric Name Description Type Labels
cardano_validator_watcher_pool_relay_count Number of relays associated with each pool GaugeVec pool_name, pool_id, pool_instance
cardano_validator_watcher_pool_pledge_met Indicates whether the pool has met its pledge requirements or not (0 or 1) GaugeVec pool_name, pool_id, pool_instance
cardano_validator_watcher_pool_saturation_level The current saturation level of the pool in percent GaugeVec pool_name, pool_id, pool_instance
cardano_validator_watcher_next_epoch_start_time Start time of the next epoch in seconds Gauge -
cardano_validator_watcher_monitored_validators_count Number of validators monitored by the watcher Gauge -
cardano_validator_watcher_missed_blocks Number of missed blocks in the current epoch CounterVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_consecutive_missed_blocks Number of blocks missed consecutively. GaugeVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_validated_blocks Number of validated blocks in the current epoch CounterVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_orphaned_blocks Number of orphaned blocks in the current epoch CounterVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_expected_blocks Number of expected blocks in the current epoch CounterVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_latest_slot_processed_by_block_watcher Latest slot processed by block watcher Gauge -
cardano_validator_watcher_next_slot_leader Next slot leader for each monitored pool GaugeVec pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_epoch_duration Duration of an epoch in days Gauge -
cardano_validator_watcher_network_epoch Current epoch number Gauge -
cardano_validator_watcher_network_block_height Latest known block height Gauge -
cardano_validator_watcher_network_slot Latest known slot Gauge -
cardano_validator_watcher_network_epoch_slot Latest known epoch slot Gauge -
cardano_validator_watcher_network_network_pools_total Total number of pools in the network Gauge -
cardano_validator_watcher_network_blocks_proposed_current_epoch Number of blocks proposed in the current epoch by the network Gauge -
cardano_validator_watcher_network_active_stake Total active stake in the network Gauge -
cardano_validator_watcher_chain_id ID of the chain Gauge -
cardano_validator_watcher_health_status Health status of the Cardano validator watcher: 1 = healthy, 0 = unhealthy Gauge -