Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RollupParams refactor + STR 478 functional test for full node #371

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

voidash
Copy link
Contributor

@voidash voidash commented Oct 7, 2024

Description

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor

Checklist

  • I have performed a self-review of my code.
  • I have commented my code where necessary.
  • I have updated the documentation if needed.
  • My changes do not introduce new warnings.
  • I have added tests that prove my changes are effective or that my feature works.
  • New and existing tests pass with my changes.

Related Issues

@prajwolrg prajwolrg self-requested a review October 7, 2024 11:16
Copy link
Contributor

@prajwolrg prajwolrg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@delbonis I'm moving part of the conversation from issue #363 here, as it is also relevant to this discussion.

Currently, the RollupParams are passed as input to the ZKVM. The hash of these parameters is used as a public parameter, anchoring all proofs to the RollupParams. This structure ensures that all logic operates with reference to these parameters.

There are three key areas in the provable path where RollupParams are used:

In the Blockspace Proof: Generating tx_filter_rules and validating the committed batch information according to the CredRule

In the CL-STF: Validating block credentials according to the CredRule

In the Checkpoint Proof: Validating the previous checkpoint proof using the verifying key (VK) found in RollupParams

Considerations:

Parameter Size & Serialization:
Currently, the size of the RollupParams is manageable, so the cost of serialization and deserialization isn't excessive. However, it might be beneficial to introduce a constant like RollupParams::DEVNET (similar to bitcoin::params::MAINNET) and use it directly in the guest code, reducing the overhead.
The challenge here is that RollupParams includes the verifying key (VK), which creates a cyclic dependency. One possible solution is to remove the VK from RollupParams and create a separate RollupParamsWithVk struct. Alternatively, the VK could be set to a placeholder (e.g., 0x00) in the guest code and updated with the correct VK later on.

Prover Efficiency:
The tx_filter_rules generation is quite resource-intensive for the prover, consuming ~9 million cycles. In devnet, where L1 blocks are produced every 30 seconds, this becomes more impactful.
Given that tx_filter_rules change infrequently, it might make sense to cache them somewhere, perhaps in the ChainState.
This approach could help optimize prover performance.

Flamegraph Profile Info of processing mainnet block 40321.
image

Copy link

codecov bot commented Oct 7, 2024

Codecov Report

Attention: Patch coverage is 69.64286% with 34 lines in your changes missing coverage. Please review.

Project coverage is 57.26%. Comparing base (2486f5c) to head (1a683f8).

Files with missing lines Patch % Lines
crates/test-utils/src/bitcoin.rs 0.00% 11 Missing ⚠️
bin/datatool/src/main.rs 0.00% 9 Missing ⚠️
bin/strata-client/src/network.rs 0.00% 7 Missing ⚠️
crates/primitives/src/params.rs 0.00% 3 Missing ⚠️
bin/strata-client/src/rpc_client.rs 0.00% 2 Missing ⚠️
bin/strata-client/src/main.rs 0.00% 1 Missing ⚠️
crates/tx-parser/src/utils.rs 90.90% 1 Missing ⚠️
@@            Coverage Diff             @@
##             main     #371      +/-   ##
==========================================
- Coverage   57.35%   57.26%   -0.10%     
==========================================
  Files         255      255              
  Lines       26953    26965      +12     
==========================================
- Hits        15459    15441      -18     
- Misses      11494    11524      +30     
Files with missing lines Coverage Δ
crates/test-utils/src/l2.rs 99.13% <100.00%> (+0.03%) ⬆️
crates/tx-parser/src/deposit/common.rs 100.00% <100.00%> (ø)
crates/tx-parser/src/deposit/deposit_request.rs 98.84% <100.00%> (ø)
crates/tx-parser/src/deposit/deposit_tx.rs 93.50% <100.00%> (ø)
crates/tx-parser/src/deposit/test_utils.rs 100.00% <100.00%> (ø)
crates/tx-parser/src/filter.rs 94.93% <100.00%> (+0.08%) ⬆️
bin/strata-client/src/main.rs 0.00% <0.00%> (ø)
crates/tx-parser/src/utils.rs 93.10% <90.90%> (-1.54%) ⬇️
bin/strata-client/src/rpc_client.rs 0.00% <0.00%> (ø)
crates/primitives/src/params.rs 9.21% <0.00%> (-7.46%) ⬇️
... and 3 more

... and 7 files with indirect coverage changes

Copy link
Collaborator

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to update names of locals/args derived from the type names.

bin/datatool/src/main.rs Outdated Show resolved Hide resolved
crates/primitives/src/params.rs Outdated Show resolved Hide resolved
functional-tests/constants.py Show resolved Hide resolved
functional-tests/fn_node_full_sync.py Outdated Show resolved Hide resolved
functional-tests/constants.py Outdated Show resolved Hide resolved
functional-tests/constants.py Outdated Show resolved Hide resolved
crates/primitives/src/params.rs Outdated Show resolved Hide resolved
crates/tx-parser/src/deposit/test_utils.rs Outdated Show resolved Hide resolved
crates/tx-parser/src/filter.rs Outdated Show resolved Hide resolved
crates/tx-parser/src/utils.rs Show resolved Hide resolved
@delbonis
Copy link
Collaborator

delbonis commented Oct 7, 2024

@prajwolrg Would definitely want to cache it in the chainstate somehow. We also would be able to reuse a bunch of state across an epoch, and that would be used across a range of L1 blocks since we will only do the L1 checkin at the end of the epoch.

remove unwanted params

func-test: fix rollup params config

func-test: add full node sync

datatool, tx-parser: remove addr from DepositTxConfig and add it to DepositTxParams

strata-client: fullnode bug fix, ws to http

strata-client: rename DepositTxParams to Config

resolve comments
.build(url)
.await
.expect("Failed to connect to the RPC server")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@voidash Can you make another PR with just changes related to this fix without the other param changes ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Copy link
Collaborator

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should clean up the tests a bit and make them more granular so we have better coverage.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale file?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be named something like fn_sync_flaky_peers. The point of the test isn't that we're testing full node sync, we're trying to that we can keep syncing as connections are interrupted. The first component of the test name is supposed to be some term describing what function it pertains to. It's also good to add a comment at the start of the file that describes what the test does.

It might actually make sense to split it into 2 or 3 tests that test the different parties restarting.

assert (
net_balance == deposit_amount
), f"deposit processed multiple times, extra: {balance - original_balance - deposit_amount}"
assert balance == deposit_amt * SATS_TO_WEI, f"deposit processed multiple times: {balance}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kinda seems like it's duplicating the above checks? Should specify that that was the amount accepted and what the expected amount was.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants