-
Notifications
You must be signed in to change notification settings - Fork 648
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chain: don't skip early transaction validity checks in state witnesses #12627
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -67,6 +67,18 @@ impl MainTransition { | |
pub struct PreValidationOutput { | ||
pub main_transition_params: MainTransition, | ||
pub implicit_transition_params: Vec<ImplicitTransitionParams>, | ||
/// List of the transactions that are valid and should be processed by e.g. | ||
/// `validate_chunk_state_witness`. | ||
/// | ||
/// This list is exactly the length of the corresponding `ChunkStateWitness::transactions` | ||
/// field. Element at the index N in this array corresponds to an element at index N in the | ||
/// transactions list. | ||
/// | ||
/// Transactions for which a `false` is stored here ought to be ignored/dropped/skipped. | ||
/// | ||
/// All elements will be true for protocol versions where `RelaxedChunkValidation` is not | ||
/// enabled. | ||
pub transaction_validity_check_passed: Vec<bool>, | ||
} | ||
|
||
#[derive(Clone)] | ||
|
@@ -369,11 +381,20 @@ pub fn pre_validate_chunk_state_witness( | |
|
||
let current_protocol_version = | ||
epoch_manager.get_epoch_protocol_version(&state_witness.epoch_id)?; | ||
if !checked_feature!( | ||
let transaction_validity_check_passed = if checked_feature!( | ||
"protocol_feature_relaxed_chunk_validation", | ||
RelaxedChunkValidation, | ||
current_protocol_version | ||
) { | ||
if last_chunk_block.header().is_genesis() { | ||
vec![true; state_witness.transactions.len()] | ||
} else { | ||
let prev_block_header = | ||
store.get_block_header(last_chunk_block.header().prev_hash())?; | ||
let mut check = chain.transaction_validity_check(prev_block_header); | ||
state_witness.transactions.iter().map(|t| check(t)).collect::<Vec<_>>() | ||
} | ||
} else { | ||
let new_transactions = &state_witness.new_transactions; | ||
let (new_tx_root_from_state_witness, _) = merklize(&new_transactions); | ||
let chunk_tx_root = state_witness.chunk_header.tx_root(); | ||
|
@@ -420,7 +441,8 @@ pub fn pre_validate_chunk_state_witness( | |
} | ||
}; | ||
} | ||
} | ||
vec![true; state_witness.transactions.len()] | ||
}; | ||
|
||
let main_transition_params = if last_chunk_block.header().is_genesis() { | ||
let epoch_id = last_chunk_block.header().epoch_id(); | ||
|
@@ -462,7 +484,11 @@ pub fn pre_validate_chunk_state_witness( | |
}) | ||
}; | ||
|
||
Ok(PreValidationOutput { main_transition_params, implicit_transition_params }) | ||
Ok(PreValidationOutput { | ||
main_transition_params, | ||
implicit_transition_params, | ||
transaction_validity_check_passed, | ||
}) | ||
} | ||
|
||
/// Validate that receipt proofs contain the receipts that should be applied during the | ||
|
@@ -611,7 +637,21 @@ pub fn validate_chunk_state_witness( | |
let (mut chunk_extra, mut outgoing_receipts) = | ||
match (pre_validation_output.main_transition_params, cache_result) { | ||
(MainTransition::Genesis { chunk_extra, .. }, _) => (chunk_extra, vec![]), | ||
(MainTransition::NewChunk(new_chunk_data), None) => { | ||
(MainTransition::NewChunk(mut new_chunk_data), None) => { | ||
let mut validity_iterator = | ||
pre_validation_output.transaction_validity_check_passed.iter(); | ||
new_chunk_data.transactions.retain(|t| { | ||
let valid = *validity_iterator.next().unwrap(); | ||
if !valid { | ||
tracing::debug!( | ||
target: "chain", | ||
message="discarding invalid transaction", | ||
tx=%t.get_hash() | ||
); | ||
} | ||
valid | ||
}); | ||
|
||
let chunk_header = new_chunk_data.chunk_header.clone(); | ||
let NewChunkResult { apply_result: mut main_apply_result, .. } = apply_new_chunk( | ||
ApplyChunkReason::ValidateChunkStateWitness, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have one more concern - this modifies the code for applying a chunk during state witness validation, but a chunk can also be applied in other ways. Nodes that track this shard will fetch the chunk and apply it independently, but I'm not sure if they'll filter the transactions in the same way. Applying a chunk with a different set of transactions could produce different results and the consensus could diverge. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was trying to implement exactly that at first, but this code is very much removed from the Do you think it would be better to thread chain all the way to the bowels of the transaction runtime? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
It'd be more foolproof, but I don't know how hard that is to do. We already have There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, I experimented with this over the past week -- it is pretty darn difficult to implement something similar to From there a lot of our code is written assuming a) cheap access to I have a half-done prototype here: master...nagisa:nearcore:chain-provider-style-tx-validation with the main missing pieces being: a) updates to Now, I largely just steamrolled through the change here -- perhaps there is some refactor to chain that would be broadly beneficial and enable this change as well. But I wasn't able to spot anything of the sort with my limited knowledge of the chain code. Maybe you see a reason why it would be better to go with something like this still? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have updated the PR above to There are five separate major call-points in regular code flow where transaction runtime has forced to thread
Out of these the catchup code smells most prone to problems here. Other than that everything else is in test code, utilities and tools. |
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to add this to make test that start at the genesis to pass. There is no "prev_block_header" in this instance. The code further below that relies on the previous block has a similar special case for the genesis.
This vector currently ends up not being used, but if it did, it seemed to me that considering all transactions "valid" is reasonable enough: the transactions can't be much older than any useful
transaction_validity_period
. On the other hand if there are additional checks added totransaction_validity_check
in the future, this might no longer hold… So this is not a particularly resilient piece of code in my eyes.