[exec-pool] OnChainConsensusConfig V4 #15862

hariria · 2025-01-31T23:50:45Z

Description

A new OnChainConsensusConfig variant that includes a window_size for execution pool.

Note: window_size is be Option<u64> in the OnChainConfig, but will be converted to Option<usize> once it's deserialized and handled in memory

trunk-io · 2025-01-31T23:50:48Z

⏱️ 1h 1m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
check-dynamic-deps	17m	🟩 🟩 🟩 🟩 🟩 (+3 more)
rust-cargo-deny	11m	🟩 ⬜ 🟩 ⬜ 🟩 (+3 more)
execution-performance / single-node-performance	6m	🟥
test-target-determinator	6m	🟩
rust-doc-tests	5m	🟩
execution-performance / test-target-determinator	5m	🟩
general-lints	3m	🟩 🟩 🟩 ⬜ 🟩 (+3 more)
semgrep/ci	3m	🟩 🟩 🟩 🟩 🟩 (+3 more)
fetch-last-released-docker-image-tag	2m	🟩
file_change_determinator	2m	🟩 🟩 🟩 🟩 🟩 (+3 more)
permission-check	22s	🟩 🟩 🟩 🟩 🟩 (+3 more)
permission-check	16s	🟩 🟩 🟩 🟩 🟩 (+3 more)
file_change_determinator	10s	🟩
permission-check	3s	🟩
determine-docker-build-metadata	3s	🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
test-target-determinator	6m	5m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

JoshLind

Looks reasonable to me 😄

JoshLind · 2025-02-03T19:42:22Z

types/src/on_chain_config/consensus_config.rs

@@ -8,6 +8,9 @@ use move_core_types::account_address::AccountAddress;
 use serde::{Deserialize, Serialize};
 use std::collections::HashMap;

+/// Default Window Size for Execution Pool
+pub const DEFAULT_WINDOW_SIZE: Option<u64> = Some(1u64);


Hmm.. we need to decide if it's Some(1) or None (to disable by default). If we use Some(1), I'll need to update this PR: #15869

JoshLind · 2025-02-03T19:42:44Z

types/src/on_chain_config/mod.rs

@@ -46,7 +46,7 @@ pub use self::{
    consensus_config::{
        AnchorElectionMode, ConsensusAlgorithmConfig, ConsensusConfigV1, DagConsensusConfigV1,
        LeaderReputationType, OnChainConsensusConfig, ProposerAndVoterConfig, ProposerElectionType,
-        ValidatorTxnConfig,
+        ValidatorTxnConfig, DEFAULT_WINDOW_SIZE,


Unused import?

Nah this is a pub use, see above

pub use self::{ approved_execution_hashes::ApprovedExecutionHashes, aptos_features::*, aptos_version::{ AptosVersion, APTOS_MAX_KNOWN_VERSION, APTOS_VERSION_2, APTOS_VERSION_3, APTOS_VERSION_4, }, commit_history::CommitHistoryResource, consensus_config::{ AnchorElectionMode, ConsensusAlgorithmConfig, ConsensusConfigV1, DagConsensusConfigV1, LeaderReputationType, OnChainConsensusConfig, ProposerAndVoterConfig, ProposerElectionType, ValidatorTxnConfig, ValidatorTxnConfig, DEFAULT_WINDOW_SIZE,

bchocho

We should have a fn window_size right?

bchocho · 2025-02-03T20:56:50Z

types/src/on_chain_config/consensus_config.rs

@@ -319,10 +347,23 @@ impl OnChainConsensusConfig {
                alg,
                vtxn: ValidatorTxnConfig::default_enabled(),
            },
+            OnChainConsensusConfig::V4 {
+                vtxn: ValidatorTxnConfig::V0,


nit: can we use the order in the struct?

github-actions · 2025-02-03T23:32:17Z

✅ Forge suite `compat` success on `60f7ca8827f5d64a148c3b163dc4126b0879279b` ==> `f66fff42b55a0948e021c5d2011dc82600b352af`

Compatibility test results for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af (PR)
1. Check liveness of validators at old version: 60f7ca8827f5d64a148c3b163dc4126b0879279b
compatibility::simple-validator-upgrade::liveness-check : committed: 12431.90 txn/s, latency: 2546.44 ms, (p50: 2600 ms, p70: 2700, p90: 3000 ms, p99: 3900 ms), latency samples: 406140
2. Upgrading first Validator to new version: f66fff42b55a0948e021c5d2011dc82600b352af
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 3354.98 txn/s, latency: 8844.81 ms, (p50: 9400 ms, p70: 10600, p90: 11300 ms, p99: 11500 ms), latency samples: 74500
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3323.32 txn/s, latency: 10391.45 ms, (p50: 11400 ms, p70: 11700, p90: 11800 ms, p99: 11900 ms), latency samples: 127600
3. Upgrading rest of first batch to new version: f66fff42b55a0948e021c5d2011dc82600b352af
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 3193.85 txn/s, latency: 9140.89 ms, (p50: 9800 ms, p70: 11500, p90: 12100 ms, p99: 12300 ms), latency samples: 68800
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3073.67 txn/s, latency: 11049.53 ms, (p50: 12000 ms, p70: 12300, p90: 12300 ms, p99: 12600 ms), latency samples: 123680
4. upgrading second batch to new version: f66fff42b55a0948e021c5d2011dc82600b352af
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 6216.68 txn/s, latency: 4906.16 ms, (p50: 5700 ms, p70: 6000, p90: 6300 ms, p99: 6300 ms), latency samples: 116560
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6218.30 txn/s, latency: 5486.65 ms, (p50: 6000 ms, p70: 6100, p90: 6300 ms, p99: 6600 ms), latency samples: 214060
5. check swarm health
Compatibility test for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af passed
Test Ok

github-actions · 2025-02-03T23:32:35Z

✅ Forge suite `realistic_env_max_load` success on `f66fff42b55a0948e021c5d2011dc82600b352af`

two traffics test: inner traffic : committed: 13015.50 txn/s, latency: 3051.90 ms, (p50: 2700 ms, p70: 3300, p90: 4200 ms, p99: 5700 ms), latency samples: 4948780
two traffics test : committed: 99.97 txn/s, latency: 2989.66 ms, (p50: 2500 ms, p70: 3300, p90: 4800 ms, p99: 9100 ms), latency samples: 1800
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 0.613, avg: 0.333", "ConsensusProposalToOrdered: max: 0.312, avg: 0.302", "ConsensusOrderedToCommit: max: 0.588, avg: 0.526", "ConsensusProposalToCommit: max: 0.900, avg: 0.828"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.62s no progress at version 32819 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.86s no progress at version 2073511 (avg 1.86s) [limit 16].
Test Ok

github-actions · 2025-02-03T23:33:34Z

✅ Forge suite `framework_upgrade` success on `60f7ca8827f5d64a148c3b163dc4126b0879279b` ==> `f66fff42b55a0948e021c5d2011dc82600b352af`

Compatibility test results for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af (PR)
Upgrade the nodes to version: f66fff42b55a0948e021c5d2011dc82600b352af
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1857.94 txn/s, submitted: 1862.80 txn/s, failed submission: 4.86 txn/s, expired: 4.86 txn/s, latency: 1538.43 ms, (p50: 1300 ms, p70: 1600, p90: 2400 ms, p99: 3400 ms), latency samples: 168241
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1542.45 txn/s, submitted: 1548.65 txn/s, failed submission: 6.20 txn/s, expired: 6.20 txn/s, latency: 1891.91 ms, (p50: 1800 ms, p70: 2100, p90: 2800 ms, p99: 4500 ms), latency samples: 139321
5. check swarm health
Compatibility test for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af passed
Upgrade the remaining nodes to version: f66fff42b55a0948e021c5d2011dc82600b352af
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1389.14 txn/s, submitted: 1394.42 txn/s, failed submission: 5.28 txn/s, expired: 5.28 txn/s, latency: 2063.84 ms, (p50: 2000 ms, p70: 2300, p90: 3000 ms, p99: 4000 ms), latency samples: 126180
Test Ok

hariria force-pushed the andrew/exec-pool-consensus-config branch from 265024d to e381cf9 Compare February 1, 2025 00:32

hariria requested review from bchocho and JoshLind February 1, 2025 00:34

hariria force-pushed the andrew/exec-pool-consensus-config branch from e381cf9 to caaf57b Compare February 1, 2025 00:39

hariria marked this pull request as ready for review February 1, 2025 00:40

hariria requested review from zekun000, sasha8 and ibalajiarun as code owners February 1, 2025 00:40

JoshLind approved these changes Feb 3, 2025

View reviewed changes

hariria force-pushed the andrew/exec-pool-consensus-config branch from caaf57b to 9183183 Compare February 3, 2025 20:15

bchocho reviewed Feb 3, 2025

View reviewed changes

hariria force-pushed the andrew/exec-pool-consensus-config branch 2 times, most recently from f4de6d8 to 375a095 Compare February 3, 2025 22:35

bchocho approved these changes Feb 3, 2025

View reviewed changes

[exec-pool] OnChainConsensusConfig V4

f66fff4

hariria force-pushed the andrew/exec-pool-consensus-config branch from 375a095 to f66fff4 Compare February 3, 2025 23:04

hariria enabled auto-merge (squash) February 3, 2025 23:05

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[exec-pool] OnChainConsensusConfig V4 #15862

[exec-pool] OnChainConsensusConfig V4 #15862

hariria commented Jan 31, 2025 •

edited

Loading

trunk-io bot commented Jan 31, 2025 •

edited

Loading

JoshLind left a comment

JoshLind Feb 3, 2025

JoshLind Feb 3, 2025

hariria Feb 3, 2025

bchocho left a comment

bchocho Feb 3, 2025

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Feb 3, 2025

github-actions bot commented Feb 3, 2025

github-actions bot commented Feb 3, 2025

[exec-pool] OnChainConsensusConfig V4 #15862

Are you sure you want to change the base?

[exec-pool] OnChainConsensusConfig V4 #15862

Conversation

hariria commented Jan 31, 2025 • edited Loading

Description

trunk-io bot commented Jan 31, 2025 • edited Loading

JoshLind left a comment

Choose a reason for hiding this comment

JoshLind Feb 3, 2025

Choose a reason for hiding this comment

JoshLind Feb 3, 2025

Choose a reason for hiding this comment

hariria Feb 3, 2025

Choose a reason for hiding this comment

bchocho left a comment

Choose a reason for hiding this comment

bchocho Feb 3, 2025

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Feb 3, 2025

✅ Forge suite compat success on 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af

github-actions bot commented Feb 3, 2025

✅ Forge suite realistic_env_max_load success on f66fff42b55a0948e021c5d2011dc82600b352af

github-actions bot commented Feb 3, 2025

✅ Forge suite framework_upgrade success on 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> f66fff42b55a0948e021c5d2011dc82600b352af

hariria commented Jan 31, 2025 •

edited

Loading

trunk-io bot commented Jan 31, 2025 •

edited

Loading

✅ Forge suite `compat` success on `60f7ca8827f5d64a148c3b163dc4126b0879279b` ==> `f66fff42b55a0948e021c5d2011dc82600b352af`

✅ Forge suite `realistic_env_max_load` success on `f66fff42b55a0948e021c5d2011dc82600b352af`

✅ Forge suite `framework_upgrade` success on `60f7ca8827f5d64a148c3b163dc4126b0879279b` ==> `f66fff42b55a0948e021c5d2011dc82600b352af`