BlobSub API

[distractedm1nd]

A subscription to new blocks is to be implemented as per #2737 . @moldis has started an implementation in #3098, which sparked discussions about how we should design the API.

The first suggestion was to return a <-chan BlobsubResponse, where

Subscribe func(context.Context, []share.Namespace) (<-chan BlobsubResponse, error) `perm:"read"`

type BlobsubResponse struct {
          height uint64
          blobsByNamespace BlobsByNamespace
}

But there have been a few internal questions.

1. Does it make sense to subscribe to multiple namespaces in one request at once? It complicates the return type and is an uncommon usecase. Users would still be able to subscribe to multiple namespaces via multiple subscriptions.

2. Should we be returning the full blob, or just the commitment? The idea here is that most users do not actually need the data returned in a blob, but just the commitment for verification.

What are the communities and team's opinions on the above questions?

[distractedm1nd]

My personal opinion is that blobsub's primary use case will be rollup nodes needing to listen to new updates from their namespace. I think only taking a single namespace makes the most sense, but I am very slightly leaning towards returning the full blob instead of the commitment.

My uncertainty here, is after discussions with the rollkit team sometime last year, rollup light nodes don't need to verify the proofs because they trust the light client connection. To me this means that the only nodes needing to subscribe to blobs are those that want to sync new data over Celestia (as opposed to the rollup's native state sync). But I am not sure here, and will discuss with some internal and external teams and post my findings here.

[nashqueue]

rollup light nodes don't need to verify the proofs because they trust the light client connection

That is true, rollkit nodes only need to make sure that the light node the proof is correct

subscribe to blobs are those that want to sync new data over Celestia (as opposed to the rollup's native state sync)

That is also true. The cause can be malicious or non-malicious

Rollkit Light Nodes might want to sync from header namespace to detect double signatures

[vgonkivs]

Thanks for this discussion @distractedm1nd. Regarding your first question, I agree that the rollups will subscribe to their specific namespace, so having a subscription per namespace makes more sense to me. Regarding your second question and the rollkit usecase I think we should ask @nashqueue. But my personal opinion is that we can return height+commitments and the users will decide what else they need to do: getting the blob itself or its proof(or doing nothing bc commitment is enough). Also, let's imagine a usecase when will have a HUGE block filled with a rollup-specific data, would it be efficient to return all this data if rollup does not need it?

[nashqueue]

I see blobsub as an optimization from pull to push. Instead of full nodes asking for it, you will get it pushed. Currently, it is not required, but it is nice to have. cc @Manav-Aggarwal @tuxcanfly