Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: send partial witnesses faster #12640

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -249,14 +249,14 @@ impl PartialWitnessActor {
}

// Function to generate the parts of the state witness and return them as a tuple of chunk_validator and part.
fn generate_state_witness_parts(
fn generate_and_send_state_witness_parts(
&mut self,
epoch_id: EpochId,
chunk_header: &ShardChunkHeader,
witness_bytes: EncodedChunkStateWitness,
chunk_validators: &[AccountId],
signer: &ValidatorSigner,
) -> Result<Vec<(AccountId, PartialEncodedStateWitness)>, Error> {
) -> Result<(), Error> {
Copy link
Contributor

@shreyan-gupta shreyan-gupta Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might have to wait for Akhi's PR to land

tracing::debug!(
target: "client",
chunk_hash=?chunk_header.chunk_hash(),
Expand All @@ -268,11 +268,8 @@ impl PartialWitnessActor {
let encoder = self.witness_encoders.entry(chunk_validators.len());
let (parts, encoded_length) = encoder.encode(&witness_bytes);

Ok(chunk_validators
.iter()
.zip_eq(parts)
.enumerate()
.map(|(part_ord, (chunk_validator, part))| {
chunk_validators.iter().zip_eq(parts).enumerate().for_each(
stedfn marked this conversation as resolved.
Show resolved Hide resolved
|(part_ord, (chunk_validator, part))| {
// It's fine to unwrap part here as we just constructed the parts above and we expect
// all of them to be present.
let partial_witness = PartialEncodedStateWitness::new(
Expand All @@ -283,9 +280,16 @@ impl PartialWitnessActor {
encoded_length,
signer,
);
(chunk_validator.clone(), partial_witness)
})
.collect_vec())
let validator_witness_tuple = (chunk_validator.clone(), partial_witness);

// Send the parts to the corresponding chunk validator owners.
self.network_adapter.send(PeerManagerMessageRequest::NetworkRequests(
NetworkRequests::PartialEncodedStateWitness(validator_witness_tuple),
));
},
);

Ok(())
}

fn generate_contract_deploys_parts(
Expand Down Expand Up @@ -343,7 +347,7 @@ impl PartialWitnessActor {
let encode_timer = metrics::PARTIAL_WITNESS_ENCODE_TIME
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PARTIAL_WITNESS_ENCODE_TIME metric no longer represents time to just encode. We can perhaps rename to PARTIAL_WITNESS_ENCODE_AND_SEND_TIME. Please quickly ping @pugachAG and let him know about this change in behavior.

Copy link
Contributor Author

@stedfn stedfn Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth specifying it? I guess it just adds a message to the channel on every iteration which should not be expensive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it can block if the channel is bounded and full but I am struggling to understand the macro to be sure

.with_label_values(&[shard_id_label.as_str()])
.start_timer();
let validator_witness_tuple = self.generate_state_witness_parts(
self.generate_and_send_state_witness_parts(
epoch_id,
chunk_header,
witness_bytes,
Expand All @@ -357,13 +361,9 @@ impl PartialWitnessActor {
self.state_witness_tracker.record_witness_sent(
chunk_hash,
witness_size_in_bytes,
validator_witness_tuple.len(),
chunk_validators.len(),
);

// Send the parts to the corresponding chunk validator owners.
self.network_adapter.send(PeerManagerMessageRequest::NetworkRequests(
NetworkRequests::PartialEncodedStateWitness(validator_witness_tuple),
));
Ok(())
}

Expand Down Expand Up @@ -598,7 +598,7 @@ impl PartialWitnessActor {

/// Sends the contract accesses to the same chunk validators
/// (except for the chunk producers that track the same shard),
/// which will receive the state witness for the new chunk.
/// which will receive the state witness for the new chunk.
fn send_contract_accesses_to_chunk_validators(
&self,
key: ChunkProductionKey,
Expand Down
14 changes: 6 additions & 8 deletions chain/client/src/test_utils/setup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -756,14 +756,12 @@ fn process_peer_manager_message_default(
}
}
}
NetworkRequests::PartialEncodedStateWitness(partial_witnesses) => {
for (account, partial_witness) in partial_witnesses {
for (i, name) in validators.iter().enumerate() {
if name == account {
connectors[i]
.partial_witness_sender
.send(PartialEncodedStateWitnessMessage(partial_witness.clone()));
}
NetworkRequests::PartialEncodedStateWitness((account, partial_witness)) => {
for (i, name) in validators.iter().enumerate() {
if name == account {
connectors[i]
.partial_witness_sender
.send(PartialEncodedStateWitnessMessage(partial_witness.clone()));
}
}
}
Expand Down
14 changes: 6 additions & 8 deletions chain/network/src/peer_manager/peer_manager_actor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1069,14 +1069,12 @@ impl PeerManagerActor {
);
NetworkResponses::NoResponse
}
NetworkRequests::PartialEncodedStateWitness(validator_witness_tuple) => {
for (chunk_validator, partial_witness) in validator_witness_tuple {
self.state.send_message_to_account(
&self.clock,
&chunk_validator,
RoutedMessageBody::PartialEncodedStateWitness(partial_witness),
);
}
NetworkRequests::PartialEncodedStateWitness((chunk_validator, partial_witness)) => {
self.state.send_message_to_account(
&self.clock,
&chunk_validator,
RoutedMessageBody::PartialEncodedStateWitness(partial_witness),
);
NetworkResponses::NoResponse
}
NetworkRequests::PartialEncodedStateWitnessForward(
Expand Down
12 changes: 5 additions & 7 deletions chain/network/src/test_loop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -348,13 +348,11 @@ fn network_message_to_partial_witness_handler(
None
}

NetworkRequests::PartialEncodedStateWitness(validator_witness_tuple) => {
for (target, partial_witness) in validator_witness_tuple.into_iter() {
shared_state
.senders_for_account(&target)
.partial_witness_sender
.send(PartialEncodedStateWitnessMessage(partial_witness));
}
NetworkRequests::PartialEncodedStateWitness((target, partial_witness)) => {
shared_state
.senders_for_account(&target)
.partial_witness_sender
.send(PartialEncodedStateWitnessMessage(partial_witness));
None
}
NetworkRequests::PartialEncodedStateWitnessForward(chunk_validators, partial_witness) => {
Expand Down
2 changes: 1 addition & 1 deletion chain/network/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ pub enum NetworkRequests {
/// Message for a chunk endorsement, sent by a chunk validator to the block producer.
ChunkEndorsement(AccountId, ChunkEndorsement),
/// Message from chunk producer to set of chunk validators to send state witness part.
PartialEncodedStateWitness(Vec<(AccountId, PartialEncodedStateWitness)>),
PartialEncodedStateWitness((AccountId, PartialEncodedStateWitness)),
stedfn marked this conversation as resolved.
Show resolved Hide resolved
/// Message from chunk validator to all other chunk validators to forward state witness part.
PartialEncodedStateWitnessForward(Vec<AccountId>, PartialEncodedStateWitness),
/// Requests an epoch sync
Expand Down
Loading