Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIP-1056 Block File Reader #10072

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Conversation

xin-hedera
Copy link
Collaborator

Description:
This PR adds block file reader to read block stream files as defined by HIP-1056

  • Add ProtoBlockFileReader
  • Add BlockRootHashDigest to calculate block file's root hash

Related issue(s):

Fixes #9894

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

@xin-hedera xin-hedera self-assigned this Jan 9, 2025
@xin-hedera xin-hedera added enhancement Type: New feature importer Area: Importer labels Jan 9, 2025
Signed-off-by: Xin Li <[email protected]>
Signed-off-by: Xin Li <[email protected]>
@@ -37,7 +37,8 @@ package com.hedera.mirror.common.domain.transaction;
public record BlockItem(Transaction transaction,
TransactionResult transactionResult,
List<TransactionOutput> transactionOutput, // Note: List may be empty
Optional<StateChanges> stateChanges) implements StreamItem {}
List<StateChanges> stateChanges // Note: List may be empty
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two reasons to make the change

  • it's unclear from the doc if there should be strictly at most 1 statechanges for a transaction or not
  • there are always non-transactional state changes before block proof, in case the last event in the last round has an event transaction, there will be more than 1 statechanges; it's unnecessary for the reader to sort out which statechanges belong to a transaction, and which belong to the event transaction.

@xin-hedera xin-hedera marked this pull request as ready for review January 9, 2025 18:39
@xin-hedera xin-hedera requested a review from a team as a code owner January 9, 2025 18:39
@xin-hedera xin-hedera added this to the 0.122.0 milestone Jan 9, 2025
Signed-off-by: Xin Li <[email protected]>
Copy link

codecov bot commented Jan 9, 2025

Codecov Report

Attention: Patch coverage is 93.82716% with 10 lines in your changes missing coverage. Please review.

Project coverage is 92.22%. Comparing base (c1fa7bc) to head (5bca60c).

Files with missing lines Patch % Lines
...or/importer/reader/block/ProtoBlockFileReader.java 92.72% 3 Missing and 5 partials ⚠️
...ror/importer/reader/block/BlockRootHashDigest.java 95.65% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10072      +/-   ##
============================================
+ Coverage     92.21%   92.22%   +0.01%     
- Complexity     7858     7896      +38     
============================================
  Files           956      958       +2     
  Lines         32938    33099     +161     
  Branches       4151     4170      +19     
============================================
+ Hits          30373    30525     +152     
- Misses         1588     1593       +5     
- Partials        977      981       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -43,9 +44,6 @@ public class BlockFile implements StreamFile<BlockItem> {
// Used to generate block hash
private BlockProof blockProof;

// Contained within the last StateChange of the block, contains hashes needed to generate the block hash
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have the following two hash leaves

  • previous hash from BlockHeader
  • start of block state hash from BlockProof

The other two are calculated from input block items merkle tree and output block item merkle tree, so we don't need BlockStreamInfo

@@ -60,6 +58,8 @@ public class BlockFile implements StreamFile<BlockItem> {
@ToString.Exclude
private String hash;

private Long index;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add back the index field so we have a setter which makes testing easier. Not really benefit test cases in this PR, however it makes the block stream verifier PR tests easier so as to just set the index, instead of crafting a proper BlockHeader with the correct block number.

Copy link
Contributor

@edwin-greene edwin-greene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type: New feature importer Area: Importer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HIP-1056 Read block files
2 participants