Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster piece getting via RPC #2854

Merged
merged 2 commits into from
Jun 17, 2024
Merged

Faster piece getting via RPC #2854

merged 2 commits into from
Jun 17, 2024

Conversation

nazar-pc
Copy link
Member

JSON decoding is apparently quite costly for pieces, so new archived segment causes head-of-line blocking on RPC connection and apparently this results in some subscriptions to silently break somewhere in jsonrpsee (see paritytech/jsonrpsee#1409). Not sure what breaks, where or why, but this change makes a lot of sense either way.

Before:

2024-06-15T19:08:06.603299Z  INFO subspace_farmer::farmer_cache: Synchronizing piece cache
2024-06-15T19:08:08.105973Z  INFO subspace_farmer::farmer_cache: Finished piece cache synchronization

After:

2024-06-15T19:04:43.315030Z  INFO subspace_farmer::farmer_cache: Synchronizing piece cache
2024-06-15T19:04:43.793678Z  INFO subspace_farmer::farmer_cache: Finished piece cache synchronization

So even with fresh farmer in dev network initial piece cache sync is measurably faster. We'll see if this is sufficient, didn't want to use tokio::task::spawn_blocking here (though it is an option if we want to offload decoding compute out of async code completely).

Code contributor checklist:

@nazar-pc
Copy link
Member Author

Restored 10 pieces at a time limit because subscriptions still break with higher decoding concurrency, even though less frequently.

@nazar-pc nazar-pc force-pushed the faster-piece-getting-via-rpc branch from 2f5fa47 to 744d52b Compare June 16, 2024 00:27
Base automatically changed from sector-write-async-wrapper to main June 17, 2024 07:52
@nazar-pc nazar-pc dismissed shamil-gadelshin’s stale review June 17, 2024 07:52

The base branch was changed.

@nazar-pc nazar-pc merged commit 1f2d60e into main Jun 17, 2024
9 checks passed
@nazar-pc nazar-pc deleted the faster-piece-getting-via-rpc branch June 17, 2024 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants