ADR: Block Data Propagation

[liamsi]

Summary

A lot was discussed and said about block propagation (during consensus and in other cases) already. We should capture the decision on how to move forward with the implementation in an ADR.

Expectation / Details

The main purpose of this ADR is to focus on how to get the MVP out of the door (#381). It's OK if the initial design takes some shortcuts to get there faster if this does not introduce (substantial) technical debt.

Additionally, it can and should try to capture as much as possible from the different discussed approaches and tradeoffs. Similarly, it could touch upon future optimizations for iterations right after the MVP (nice-to-have).

If helpful, explorative implementation in parallel to the ADR is encouraged. At this stage, ideally a tracer-bullet and not a prototype. Where necessary a quickly written spike to verify the feasibility of the proposed solution can also be a good idea.

Some topics that could be part of this adr:

• "pull vs push" based (https://docs.google.com/document/d/1mzfU2Ojlt1A5ucb1j_J5TfbIKcblF-9c2QnBRIZg8dk)
• tendermint's p2p package in the context of block gossiping and reactors
• libp2p's pubsusb
• granularity of the gossiped data (shares, rows)?
• (how) does this impact other components, particularly other p2p components, e.g. DAS?
• relation to adr 001 (see below)

Difference to ADR 001

ADR 001 was written under the assumption that validators will also do DAS. Hence, every node-type could have used the ipld API RetrieveBlockData. Now, that we pivoted to removing that node-type (celestiaorg/celestia-specs#177 (review)), block propagation can and should be revisted. Particularly, during consensus, where it remains important that validators receive the block in a timely manner.

References and Context

• a relatively vague and slightly confusing high-level issue about how the implementation might look like exists: add basic description of ABCI Commit.ResponseHeight tendermint/spec#85
• related discussion: Block Data Gossiping #384 (most discussions about this happened in slack / discord though)
• draft PR: Consensus block propagation via IPFS #293
• adr001: https://github.com/lazyledger/lazyledger-core/blob/master/docs/lazy-adr/adr-001-block-propagation.md

[Wondertan]

@liamsi, regarding the granularity of the gossiped data. Some time ago you mentioned that gossiping BlockParts in Tendermint wasn't the best idea. The thing I wonder about is full-block data gossiping vs rows. I think the BlockParts thing is related here, so can you pls share your insights on that.

[liamsi]

Some time ago you mentioned that gossiping BlockParts in Tendermint wasn't the best idea.

Yeah, gossiping block parts, at least in its current form, is not the best idea because we chunk the block differently already (into erasure coded shares). So IMO, we should stick to one approach.

The thing I wonder about is full-block data gossiping vs rows.

My intuition here is that gossiping rows is the best granularity: on the receiving side, you could check if the row you got matches a CID in the DAHeader (otherwise someone is giving you garbage data, e.g. to DoS your node). Remember that blocks could become potentially very large and we don't want to wait for the whole block until making the decision if the received data matches the commitment in the DAHeader or not.

An alternative to gossiping rows could be gossiping shares. This would be more similar to the current BlockParts. But then, we would need to give the inclusion proof with each share to prevent the above's scenario that someone feeds garbage blocks into the network and this would only be detected after downloading all garbage. This would have the major downside that a lot of duplicate data needs to be gossiped (all those inner proof nodes).

[liamsi]

Actually, another idea is to only send half the rows: namely those parts that represent the original data. So the smallest piece or "part" that would be gossiped could be [original part of row] (and maybe a single [proof node] for the other half). From this the receiving part could already compute the other half (this is currently not supported by rsmt2d but we could change that and it plays into your thinking here btw: celestiaorg/rsmt2d#60).

[Wondertan]

Yeah, gossiping block parts, at least in its current form, is not the best idea because we chunk the block differently already (into erasure coded shares).

Ok, but maybe I got your original concern wrongly. I thought the block parts are not a good idea in terms of tendermimt; for our case, it's obviously not a way to go for the reason you mentioned.

[liamsi]

I thought the block parts are not a good idea in terms of tendermimt

There is nothing wrong with blockparts in tendermint. The only questionable thing is that the PartSetHeader is part of BlockID in the header and BlockID is used everywhere. So basically an implementation detail on how to get the block data to other nodes spills everywhere. There are several issues on how to get rid of PartSetHeader in the header; see for instance tendermint/spec#78, and tendermint/tendermint#7922

[liamsi]

Closing as this is no longer relevant now. IMO we should merge the ADR that @Wondertan wrote (after this is briefly addressed).