Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow reverting to preload if fallen behind #345

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Sekhmet
Copy link
Contributor

@Sekhmet Sekhmet commented Mar 7, 2025

This adds ability to revert to preload if sequential indexing ends up falling behind. We check latest block every 50 blocks and if if it's more than BLOCK_PRELOAD_OFFSET * 2 ahead of us preload will be activated again.

Test plan

  1. Run with SX API.
  2. Change ARB1 start block to 312004031.
  3. Run it as ENABLED_NETWORKS=arb1 yarn dev.
  4. You will see it starts with preload, then it switches to sequential indexing.
  5. After a while latest block check fires and you get info how behind we are.
  6. If we are more than 100 blocks behind you will see message we are reverting to preload.
  7. Preloading happens.
  8. Now we are going sequential as well with less than 100 blocks behind.

This adds ability to revert to preload if sequential indexing ends up falling behind.
We check latest block every 50 blocks and if if it's more than BLOCK_PRELOAD_OFFSET * 2
ahead of us preload will be activated again.
@Sekhmet Sekhmet requested review from bonustrack and Copilot March 7, 2025 12:03

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR implements a fallback mechanism that reverts the indexing mode to preload when the sequential indexing falls significantly behind.

  • Introduces a constant (CHECK_LATEST_BLOCK_INTERVAL) to trigger periodic latest block checks.
  • Adds logic to compare the current block number with the latest block number and update the preload range when lag exceeds the threshold.
  • Includes minor logging adjustments in the preload section.

Reviewed Changes

File Description
src/container.ts Adds logic to trigger a revert to preload indexing when too far behind

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

src/container.ts:250

  • Ensure that the variables 'checkpointBlock' and 'preloadedBlock' are correctly defined and updated prior to this condition. It is critical to verify that the intended fallback behavior is triggered only when appropriate values are available.
if (!checkpointBlock && !preloadedBlock) {

src/container.ts:262

  • Double-check that resetting 'this.preloadEndBlock' with 'latestBlock - BLOCK_PRELOAD_OFFSET' aligns with the overall block handling strategy without inadvertently skipping blocks or disrupting the intended preload range.
this.preloadEndBlock = latestBlock - BLOCK_PRELOAD_OFFSET;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant