Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Distributed POST verification #5390

Closed
wants to merge 41 commits into from

Conversation

poszu
Copy link
Contributor

@poszu poszu commented Dec 22, 2023

Part of #5185, closes #5376

  • Local randomness seed to select K3 indices using p2p peer ID
  • Verifying K3 indices of POST
  • Verifying ALL indices for initial POST
  • Publishing POST malfeasance proofs
  • Verifying only the invalid index in POST malfeasance proofs
  • Verify the candidate for a positioning ATX and its chain
  • Verify the candidate for a commitment ATX and its chain
  • Wait "some time" for malfeasance proofs before creating an active set
    ☝️ this is covered by ATX grading
  • configurable duration after which ATXs (and their chains) are considered valid (in terms of POST labels)
  • system tests

Copy link

codecov bot commented Dec 22, 2023

Codecov Report

Attention: 62 lines in your changes are missing coverage. Please review.

Comparison is base (ff567da) 79.5% compared to head (73fec9e) 79.5%.
Report is 1 commits behind head on develop.

Files Patch % Lines
common/types/malfeasance.go 18.5% 21 Missing and 1 partial ⚠️
activation/validation.go 80.5% 13 Missing and 7 partials ⚠️
sql/atxs/atxs.go 86.9% 4 Missing and 2 partials ⚠️
malfeasance/handler.go 87.5% 4 Missing and 1 partial ⚠️
activation/handler.go 84.0% 2 Missing and 2 partials ⚠️
activation/activation.go 94.8% 2 Missing ⚠️
events/events.go 0.0% 2 Missing ⚠️
datastore/store.go 80.0% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #5390    +/-   ##
========================================
  Coverage     79.5%   79.5%            
========================================
  Files          268     268            
  Lines        26591   26851   +260     
========================================
+ Hits         21146   21361   +215     
- Misses        3916    3954    +38     
- Partials      1529    1536     +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dshulyak
Copy link
Contributor

dshulyak commented Dec 22, 2023

Wait "some time" for malfeasance proofs before creating an active set

what does it mean? active set is created at the start of the epoch. it is usually created from atxs that are graded as "good", such that were received 2 hours prior epoch start, and no malfeasance proof was received before epoch start. so effectively honest node selects atxs that could have been verified by someone during 2 hours. in cases when it is not created from grades, it will be created as a superset of activesets in the first block.

the actual problem here is that we don't have reliable sync for malfeasence proofs, so there is no guarantee that node will receive them within 2 hours from being created. this will either need to be improved before releasing distributed verification, or just be aware of this risk

@poszu poszu changed the title [WIP] Distributed POST verification Distributed POST verification Jan 11, 2024
@poszu poszu marked this pull request as ready for review January 11, 2024 11:39
@poszu
Copy link
Contributor Author

poszu commented Jan 11, 2024

bors try

spacemesh-bors bot added a commit that referenced this pull request Jan 11, 2024
@spacemesh-bors
Copy link

try

Build succeeded:

activation/handler_test.go Show resolved Hide resolved
activation/activation_test.go Show resolved Hide resolved
systest/Dockerfile Outdated Show resolved Hide resolved
systest/tests/distributed_post_verification_test.go Outdated Show resolved Hide resolved
systest/tests/distributed_post_verification_test.go Outdated Show resolved Hide resolved
systest/tests/distributed_post_verification_test.go Outdated Show resolved Hide resolved
@poszu
Copy link
Contributor Author

poszu commented Feb 1, 2024

bors merge

spacemesh-bors bot pushed a commit that referenced this pull request Feb 1, 2024
Part of #5185, closes #5376

- [x] Local randomness seed to select K3 indices using p2p peer ID
- [x] Verifying K3 indices of POST
- [x] Verifying ALL indices for initial POST
- [x] Publishing POST malfeasance proofs
- [x] Verifying only the invalid index in POST malfeasance proofs
- [x] Verify the candidate for a positioning ATX and its chain
- [x] Verify the candidate for a commitment ATX and its chain
- [x] Wait "some time" for malfeasance proofs before creating an active set
        ☝️ this is covered by ATX grading
- [x] configurable duration after which ATXs (and their chains) are considered valid (in terms of POST labels)
- [x] system tests
@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 1, 2024

Build failed (retrying...):

@poszu
Copy link
Contributor Author

poszu commented Feb 1, 2024

bors cancel

@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 1, 2024

Canceled.

@poszu
Copy link
Contributor Author

poszu commented Feb 1, 2024

bors merge

spacemesh-bors bot pushed a commit that referenced this pull request Feb 1, 2024
Part of #5185, closes #5376

- [x] Local randomness seed to select K3 indices using p2p peer ID
- [x] Verifying K3 indices of POST
- [x] Verifying ALL indices for initial POST
- [x] Publishing POST malfeasance proofs
- [x] Verifying only the invalid index in POST malfeasance proofs
- [x] Verify the candidate for a positioning ATX and its chain
- [x] Verify the candidate for a commitment ATX and its chain
- [x] Wait "some time" for malfeasance proofs before creating an active set
        ☝️ this is covered by ATX grading
- [x] configurable duration after which ATXs (and their chains) are considered valid (in terms of POST labels)
- [x] system tests
@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 1, 2024

Build failed (retrying...):

  • systest-status

@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 1, 2024

Canceled.

@poszu
Copy link
Contributor Author

poszu commented Feb 1, 2024

bors try

spacemesh-bors bot added a commit that referenced this pull request Feb 1, 2024
@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 1, 2024

try

Build failed:

@poszu
Copy link
Contributor Author

poszu commented Feb 2, 2024

bors merge

spacemesh-bors bot pushed a commit that referenced this pull request Feb 2, 2024
Part of #5185, closes #5376

- [x] Local randomness seed to select K3 indices using p2p peer ID
- [x] Verifying K3 indices of POST
- [x] Verifying ALL indices for initial POST
- [x] Publishing POST malfeasance proofs
- [x] Verifying only the invalid index in POST malfeasance proofs
- [x] Verify the candidate for a positioning ATX and its chain
- [x] Verify the candidate for a commitment ATX and its chain
- [x] Wait "some time" for malfeasance proofs before creating an active set
        ☝️ this is covered by ATX grading
- [x] configurable duration after which ATXs (and their chains) are considered valid (in terms of POST labels)
- [x] system tests
@spacemesh-bors
Copy link

spacemesh-bors bot commented Feb 2, 2024

Pull request successfully merged into develop.

Build succeeded:

@spacemesh-bors spacemesh-bors bot changed the title Distributed POST verification [Merged by Bors] - Distributed POST verification Feb 2, 2024
@spacemesh-bors spacemesh-bors bot closed this Feb 2, 2024
@spacemesh-bors spacemesh-bors bot deleted the distributed-post-verification branch February 2, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

verify selected indices of POST and create malfeasance proofs
3 participants