You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This means that if no events arrive on a subscription for a long time (millions of blocks) we never increase the blockHWM, and write a new checkpoint. So on restart (of ethconnect or the node), we drive a high load on the node as we recreate the filter at a very old block number.
Instead we need a system to safely move the blockHWM forwards, at regular intervals, so it's never more than a few hundred blocks behind the head. This will mean periodically in the polling cycle, asking the node for the current block height ahead of the call to eth_getFilterChanges and if no results are found, then setting the blockHWM to a safe number (such as 100 blocks behind the head, if that's higher than zero, and higher than the previous blockHWM).
Currently we only increase the block high-watermark on a subscription to an event stream, when we have successfully processed a batch:
https://github.com/kaleido-io/ethconnect/blob/ad40de61370ec2b99a8cfc79cdf72e5dcac3d914/internal/kldevents/logprocessor.go#L83
This means that if no events arrive on a subscription for a long time (millions of blocks) we never increase the blockHWM, and write a new checkpoint. So on restart (of ethconnect or the node), we drive a high load on the node as we recreate the filter at a very old block number.
Instead we need a system to safely move the blockHWM forwards, at regular intervals, so it's never more than a few hundred blocks behind the head. This will mean periodically in the polling cycle, asking the node for the current block height ahead of the call to
eth_getFilterChanges
and if no results are found, then setting the blockHWM to a safe number (such as 100 blocks behind the head, if that's higher than zero, and higher than the previous blockHWM).fyi @vdamle @panghalamit
The text was updated successfully, but these errors were encountered: