Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Autosharding API for req-resp protocols #1500

Closed
Tracked by #65
chair28980 opened this issue Aug 23, 2023 · 5 comments
Closed
Tracked by #65

feat: Autosharding API for req-resp protocols #1500

chair28980 opened this issue Aug 23, 2023 · 5 comments
Assignees
Labels
E:1.2: Autosharding for autoscaling See https://github.com/waku-org/pm/issues/65 for details

Comments

@chair28980
Copy link
Contributor

This tracks the work necessary in each client to provide API(s) to applications using req-resp protocols (store, lightpush, filter, etc.) with optional pubsub topic arguments. Each client should then use autosharding to determine the associated shards on behalf of applications. Note that underlying protocols should not be affected and each client implementation should locally populate pubsub topic fields with the shards hashed from content topics received via the API.

Priority: Critical for launch

@chair28980
Copy link
Contributor Author

chair28980 commented Aug 23, 2023

waku-org/research#3

@danisharora099 @weboko

Per the research issue, can y'all verify if this is required for JSWaku and assign ownership as necessary. Thank you!

@danisharora099 danisharora099 moved this to Triage in Waku Sep 7, 2023
@fryorcraken fryorcraken added E:1.2: Autosharding for autoscaling See https://github.com/waku-org/pm/issues/65 for details and removed E:2023-1mil-users labels Sep 8, 2023
@weboko weboko self-assigned this Sep 21, 2023
@weboko weboko moved this from Triage to Priority in Waku Oct 31, 2023
@chair28980 chair28980 moved this from Priority to In Progress in Waku Nov 7, 2023
@adklempner
Copy link
Member

Weekly Update

  • achieved: derive pubsub topic from content topic in encoders/decoders when autosharding is specified
  • next: node config should specify static sharding or autosharding. implement autosharded topics in all req-resp protocols

@fryorcraken
Copy link
Collaborator

Do note that for this epic, it is fine to expect a js-waku node to only belong to one shard and to have the developer pass this "shard" (ie, application and version part of the content topic) at node creation time.

@adklempner
Copy link
Member

Weekly Update

  • achieved: config node for static/autosharding, test all protocols against autosharding RPC endpoints on nwaku
  • next: config application and version on node creation, only discover nodes of same shard

@adklempner adklempner moved this from In Progress to Code Review / QA in Waku Dec 4, 2023
@adklempner
Copy link
Member

Weekly Update

  • achieved: all protocols can be configured to use autosharding for determining pubsub topics
  • next: make autosharding the default behavior when running js-waku

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E:1.2: Autosharding for autoscaling See https://github.com/waku-org/pm/issues/65 for details
Projects
Archived in project
Development

No branches or pull requests

4 participants