Skip to content
This repository has been archived by the owner on Aug 21, 2024. It is now read-only.

feat(concurrency): implement Default for ConcurrencyConfig #1952

Conversation

barak-b-starkware
Copy link
Collaborator

@barak-b-starkware barak-b-starkware commented Jun 4, 2024

This change is Reviewable

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/4_get_last_committed_index branch from 13eb747 to 4642dca Compare June 4, 2024 09:54
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch 2 times, most recently from 669e636 to 89932c8 Compare June 4, 2024 10:17
@codecov-commenter
Copy link

codecov-commenter commented Jun 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.17%. Comparing base (702f401) to head (cf5a729).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1952   +/-   ##
=======================================
  Coverage   78.17%   78.17%           
=======================================
  Files          62       62           
  Lines        8842     8842           
  Branches     8842     8842           
=======================================
  Hits         6912     6912           
  Misses       1492     1492           
  Partials      438      438           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 89932c8 to 53ff368 Compare June 4, 2024 10:29
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/4_get_last_committed_index branch from 4642dca to 544be6f Compare June 4, 2024 10:43
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 53ff368 to 3ea071a Compare June 4, 2024 10:44
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/4_get_last_committed_index branch from 544be6f to 1415441 Compare June 4, 2024 10:50
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch 3 times, most recently from 83a8c9d to ab6459d Compare June 4, 2024 11:13
Copy link
Collaborator

@avi-starkware avi-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @barak-b-starkware, @MohammadNassar1, @noaov1, and @Yoni-Starkware)


crates/blockifier/src/blockifier/config.rs line 15 at r1 (raw file):

impl Default for ConcurrencyConfig {
    fn default() -> Self {
        Self { enabled: false, n_workers: 64, chunk_size: 4 }

n_workers and chunk_size here have swapped values from the default consts in utils.rs.

Why don't you use the constants here?

Suggestion:

        Self { enabled: false, n_workers: 4, chunk_size: 64 }

crates/blockifier/src/concurrency/utils.rs line 15 at r1 (raw file):

pub const DEFAULT_CHUNK_SIZE: usize = 64;
pub const DEFAULT_N_WORKERS: usize = 4;

Ittay Dror said we should determine the number of workers automatically based on the actual hardware we are running on. Otherwise, we would have to reconfigure the batcher each time we change the machine it is running on.

This is probably not in the scope of this PR, but maybe add a TODO somewhere?

Nonblocking.

Code quote:

pub const DEFAULT_N_WORKERS: usize = 4;

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/4_get_last_committed_index branch from 1415441 to 7f99a32 Compare June 4, 2024 13:54
Copy link
Collaborator Author

@barak-b-starkware barak-b-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @avi-starkware, @MohammadNassar1, @noaov1, @OriStarkware, and @Yoni-Starkware)


crates/blockifier/src/blockifier/config.rs line 15 at r1 (raw file):

Previously, avi-starkware wrote…

n_workers and chunk_size here have swapped values from the default consts in utils.rs.

Why don't you use the constants here?

Done.
To use constants from the concurrency module, we need to find some workaround since the concurrency module can be seen only when adding --feartures=concurrency.
I think we can remove the #[cfg(feature = concurrency)].


crates/blockifier/src/concurrency/utils.rs line 15 at r1 (raw file):

Previously, avi-starkware wrote…

Ittay Dror said we should determine the number of workers automatically based on the actual hardware we are running on. Otherwise, we would have to reconfigure the batcher each time we change the machine it is running on.

This is probably not in the scope of this PR, but maybe add a TODO somewhere?

Nonblocking.

Removed from this PR.

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from ab6459d to 0ff98da Compare June 4, 2024 14:01
avi-starkware
avi-starkware previously approved these changes Jun 4, 2024
Copy link
Collaborator

@avi-starkware avi-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @MohammadNassar1, @noaov1, @OriStarkware, and @Yoni-Starkware)

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/4_get_last_committed_index branch from 7f99a32 to 17e6209 Compare June 4, 2024 14:50
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 0ff98da to 5f969f5 Compare June 4, 2024 14:50
Yoni-Starkware
Yoni-Starkware previously approved these changes Jun 5, 2024
Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @MohammadNassar1, @noaov1, and @OriStarkware)

@barak-b-starkware barak-b-starkware changed the base branch from barak/execute_chunk/4_get_last_committed_index to barak/execute_chunk/2_commit_with_apply_writes June 5, 2024 12:08
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 5f969f5 to e54a9a4 Compare June 5, 2024 12:09
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from f0df83a to 43bdffe Compare June 5, 2024 12:38
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from e54a9a4 to c85364c Compare June 5, 2024 12:39
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch 2 times, most recently from 6140939 to 41307ad Compare June 5, 2024 12:47
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from c85364c to eacfd9b Compare June 5, 2024 12:48
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch 2 times, most recently from 2d02615 to 81163fa Compare June 5, 2024 13:39
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from eacfd9b to 1117635 Compare June 5, 2024 13:41
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from 81163fa to 0308a90 Compare June 5, 2024 14:48
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 1117635 to 268fcab Compare June 5, 2024 14:50
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from 0308a90 to 1ffb398 Compare June 5, 2024 14:55
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 268fcab to 038132b Compare June 5, 2024 14:56
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from 1ffb398 to 4868ea8 Compare June 5, 2024 15:04
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 038132b to f992363 Compare June 5, 2024 15:04
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from 4868ea8 to 3719ff4 Compare June 5, 2024 21:52
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch 2 times, most recently from 8b18f10 to 12572b9 Compare June 6, 2024 08:59
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/2_commit_with_apply_writes branch from 3719ff4 to e41aa81 Compare June 6, 2024 12:00
@barak-b-starkware barak-b-starkware changed the base branch from barak/execute_chunk/2_commit_with_apply_writes to main June 6, 2024 12:03
@barak-b-starkware barak-b-starkware dismissed stale reviews from Yoni-Starkware and avi-starkware June 6, 2024 12:03

The base branch was changed.

@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 12572b9 to 1f84a68 Compare June 6, 2024 12:05
@barak-b-starkware barak-b-starkware force-pushed the barak/execute_chunk/5_default_concurrency_config branch from 1f84a68 to cf5a729 Compare June 6, 2024 12:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants