Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(dot/sync): Implement warp sync strategy #4275

Open
wants to merge 43 commits into
base: development
Choose a base branch
from
Open

feat(dot/sync): Implement warp sync strategy #4275

wants to merge 43 commits into from

Conversation

dimartiro
Copy link
Contributor

@dimartiro dimartiro commented Oct 22, 2024 •
edited
Loading

Changes

Implementation of the warp sync strategy.
This strategy is composed of 2 phases:

  1. Request warp sync proofs until get one completed proof (last finalized block)
  2. Retrieve that block to be used later for the state sync strategy

This goal of this strategy is to reach the latest finalized block of the canonical chain following authority set changes and validating justifications.
After this block is reached, we have to retrieve its state using the state_sync, that will allow us to build blocks on top of that one while we fill the gap between genesis and the latest finalized block retrieved by the warp sync.

Summary of changes

  • Add new warp_sync strategy
    • Add required warp sync p2p messages
    • Add new reputation changes constants
  • Refactor peer set view to share the same reference between the strategies
  • Add CLI config to use warp sync instead of full sync when node is started
  • Tests

Tests

make test

Issues

closes: #3183, #4277

@dimartiro dimartiro self-assigned this Oct 22, 2024
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from 20937f7 to 6a553fb Compare October 24, 2024 13:29
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch 2 times, most recently from da62fed to 7ad8347 Compare October 31, 2024 13:12
@dimartiro dimartiro marked this pull request as ready for review October 31, 2024 22:10
EclesioMeloJunior
EclesioMeloJunior approved these changes Nov 4, 2024
View reviewed changes
Copy link
Member

@EclesioMeloJunior EclesioMeloJunior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

EclesioMeloJunior
EclesioMeloJunior reviewed Nov 4, 2024
View reviewed changes
dot/sync/service.go Show resolved Hide resolved
dot/sync/warp_sync.go Show resolved Hide resolved
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from c2789d6 to 93108f7 Compare November 5, 2024 13:34
haikoschol
haikoschol approved these changes Nov 5, 2024
View reviewed changes
Copy link
Contributor

@haikoschol haikoschol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few nits/questions but LGTM!

dot/sync/service.go Show resolved Hide resolved
dot/sync/warp_sync.go Outdated Show resolved Hide resolved
dot/sync/warp_sync.go Outdated Show resolved Hide resolved
dot/sync/warp_sync.go Outdated Show resolved Hide resolved
dot/sync/warp_sync.go Outdated Show resolved Hide resolved
lib/common/hash.go Outdated Show resolved Hide resolved
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from 014395d to 42faaf7 Compare November 10, 2024 08:27
timwu20
timwu20 reviewed Nov 14, 2024
View reviewed changes
dot/network/messages/warp_sync_test.go Outdated Show resolved Hide resolved
dot/network/messages/warp_sync.go Outdated Show resolved Hide resolved
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from 42faaf7 to 89dcb62 Compare November 15, 2024 08:22
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch 3 times, most recently from 967a77e to 7294e7d Compare December 5, 2024 09:47
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from 7294e7d to a3ee839 Compare December 9, 2024 15:03
@P1sar P1sar added the A-in-QA-or-regression pull request or issues is currently in QA or regression testing phase. label Dec 12, 2024
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch 4 times, most recently from 6004faf to a1b63c3 Compare December 19, 2024 13:27
@dimartiro
Use right procotol id for sync
5fac7f6
@dimartiro
Fix comment
cd41c6c
@dimartiro
Add warp sync message decode test
38d5e94
@dimartiro
Add missing license
7b96c05
@dimartiro
Remove import
3d3b881
@dimartiro
Use right justification number for generic
5572777
@dimartiro
Disable linting lll
e4c1415
@dimartiro
Fix types
8739dff
@dimartiro
Small fixes
4928029
@dimartiro
Create warp sync provider interface
4805aa0
@dimartiro
Remove unnecesary logs
c9f1f47
@dimartiro
Clean up, remove logs and local tests
638019b
@dimartiro
Lint
bee9c1e
@dimartiro
Lint
46049c5
@dimartiro
Fix Test_PeerSupportsProtocol test
9fdf102
@dimartiro
Remove TODO
adc74da
@dimartiro
Fix findScheduledChange
07bd04b
@dimartiro
Improve logs and remove tests
d5ab068
@dimartiro
Improve logs
95fce09
@dimartiro
Fix get target block
5b2bf1c
@dimartiro
Update comment
f1da852
@dimartiro
Remove comment
fcea5a3
@dimartiro
Refactor
1a8e583
@dimartiro
Update comment
c78d77f
@dimartiro
Add error message in panic
f0fa4d5
@dimartiro
Fix get target block
9e0750d
@dimartiro
Add warp sync proof fixture
568c084
@dimartiro
Move types to warpsync package
960e7c7
@dimartiro
Remove unnused linter comment
36b57e0
@dimartiro dimartiro force-pushed the diego/warpsync/strategy branch from a1b63c3 to 36b57e0 Compare December 19, 2024 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@EclesioMeloJunior EclesioMeloJunior EclesioMeloJunior approved these changes

@timwu20 timwu20 timwu20 left review comments

@haikoschol haikoschol haikoschol approved these changes

@jimjbrettj jimjbrettj Awaiting requested review from jimjbrettj jimjbrettj is a code owner

@kishansagathiya kishansagathiya Awaiting requested review from kishansagathiya

@P1sar P1sar Awaiting requested review from P1sar P1sar is a code owner

Assignees

@dimartiro dimartiro

Labels
A-in-QA-or-regression pull request or issues is currently in QA or regression testing phase.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement warp sync strategy
5 participants
@dimartiro @haikoschol @timwu20 @EclesioMeloJunior @P1sar