feat: Adding READMEs throughout the codebase (#248)

* abci readme

* done wit example for abci.go

* SEA

* base lane done

* proposals

* block readme

* fixes

* proposals fix

abci/README.md

@@ -0,0 +1,222 @@
+# Block SDK Proposal Construction & Verification
+
+## Overview
+
+This readme describes how the Block SDK constructs and verifies proposals. To get a high level overview of the Block SDK, please see the [Block SDK Overview](../README.md).
+
+The Block SDK is a set of Cosmos SDK and ABCI++ primitives that allow chains to fully customize blocks for specific use cases. It turns your chain's blocks into a **`highway`** consisting of individual **`lanes`** with their own special functionality. The mempool is no longer a single black box, but rather a set of lanes that can be customized to fit your application's needs. Each lane is meant to deal with specific a set of transactions, and can be configured to fit your application's needs.
+
+Proposal construction / verification is done via the `PrepareProposal` and `ProcessProposal` handlers defined in [`abci.go`](./abci.go), respectively. Each block proposal built by the Block SDK enforces that a block is comprised of contiguous sections of transactions that belong to a single lane.
+
+For example, if your application has 3 lanes, `A`, `B`, and `C`, and the order of lanes is `A -> B -> C`, then the proposal will be built as follows:
+
+```golang
+blockProposal := {
+    Tx1, (Lane A)
+    Tx2, (Lane A)
+    Tx3, (Lane A)
+    Tx4, (Lane B)
+    Tx5, (Lane B)
+    Tx6, (Lane B)
+    Tx7, (Lane C)
+    Tx8, (Lane C)
+    Tx9, (Lane C)
+}
+```
+
+## Proposal Construction
+
+The `PrepareProposal` handler is called by the ABCI++ application when a new block is requested by the network for the given proposer. At runtime, the `PrepareProposal` handler will do the following:
+
+1. Determine the order of lanes it wants to construct the proposal from. If the application is configured to utilize the Block SDK module, it will fetch the governance configured order of lanes and use that. Otherwise, it will use the order defined in your [`app.go`](../tests/app/app.go) file (see our test app as an example).
+2. After determining the order, it will chain together all of the lane's `PrepareLane` methods into a single `PrepareProposal` method.
+3. Each lane will select transactions from its mempool, verify them according to its own rules, and return a list of valid transactions to add and a list of transactions to remove i.e. see [`PrepareLane`](../block/base/abci.go).
+4. The transactions will only be added to the current proposal being built _iff_ the transactions are under the block gas and size limit for that lane. If the transactions are over the limit, they will NOT be added to the proposal and the next lane will be called.
+5. A final proposal is returned with all of the valid transactions from each lane.
+
+If any of the lanes fail during `PrepareLane`, the next lane will be called and the proposal will be built from the remaining lanes. This is a fail-safe mechanism to ensure that the proposal is always built, even if one of the lanes fails to prepare. Additionally, state is mutated _iff_ the lane is successful in preparing its portion of the proposal.
+
+To customize how much block-space a given lane consumes, you have to configure the `MaxBlockSpace` variable in your lane configuration object (`LaneConfig`). Please visit [`lanes.go`](../tests/app/lanes.go) for an example. This variable is a map of lane name to the maximum block space that lane can consume. Note that if the Block SDK module is utilized, the `MaxBlockSpace` variable will be overwritten by the governance configured value.
+
+### Proposal Construction Example
+
+> Let's say your application has 3 lanes, `A`, `B`, and `C`, and the order of lanes is `A -> B -> C`. 
+> 
+> * Lane `A` has a `MaxBlockSpace` of 1000 bytes and 500 gas limit.
+> * Lane `B` has a `MaxBlockSpace` of 2000 bytes and 1000 gas limit.
+> * Lane `C` has a `MaxBlockSpace` of 3000 bytes and 1500 gas limit.
+
+Lane `A` currently contains 4 transactions:
+
+* Tx1: 100 bytes, 100 gas
+* Tx2: 800 bytes, 300 gas
+* Tx3: 200 bytes, 100 gas
+* Tx4: 100 bytes, 100 gas
+
+Lane `B` currently contains 4 transactions:
+
+* Tx5: 1000 bytes, 500 gas
+* Tx6: 1200 bytes, 600 gas
+* Tx7: 1500 bytes, 300 gas
+* Tx8: 1000 bytes, 400 gas
+
+Lane `C` currently contains 4 transactions:
+
+* Tx9: 1000 bytes, 500 gas
+* Tx10: 1200 bytes, 600 gas
+* Tx11: 1500 bytes, 300 gas
+* Tx12: 100 bytes, 400 gas
+
+Assuming all transactions are valid according to their respective lanes, the proposal will be built as follows (this is pseudo-code):
+
+```golang
+// Somewhere in abci.go a new proposal is created:
+blockProposal := proposals.NewProposal(...)
+...
+// First lane to be called is lane A, this will return the following transactions to add after PrepareLane is called:
+partialProposalFromLaneA := {
+    Tx1, // 100 bytes, 100 gas
+    Tx2, // 800 bytes, 300 gas
+    Tx3, // 200 bytes, 100 gas
+}
+...
+// First Lane A will update the proposal.
+if err := blockProposal.Update(partialProposalFromLaneA); err != nil {
+    return err
+}
+...
+// Next, lane B will be called with the following transactions to add after PrepareLane is called:
+//
+// Note: Here, Tx6 is excluded because it and Tx5's gas would exceed the lane gas limit. Tx7 is similarly excluded because it and Tx5's size would exceed the lane block size limit. Note that Tx5 will always be included first because it is ordered first and it is valid given the lane's constraints.
+partialProposalFromLaneB := {
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+}
+...
+// Next, lane B will update the proposal.
+if err := blockProposal.Update(partialProposalFromLaneB); err != nil {
+    return err
+}
+...
+// Finally, lane C will be called with the following transactions to add after PrepareLane is called:
+partialProposalFromLaneC := {
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+...
+// Finally, lane C will update the proposal.
+if err := blockProposal.Update(partialProposalFromLaneC); err != nil {
+    return err
+}
+...
+// The final proposal will be:
+blockProposal := {
+    Tx1, // 100 bytes, 100 gas
+    Tx2, // 800 bytes, 300 gas
+    Tx3, // 200 bytes, 100 gas
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+```
+
+If any transactions are invalid, they will not be included in the lane's partial proposal.
+
+## Proposal Verification
+
+The `ProcessProposal` handler is called by the ABCI++ application when a new block has been proposed by the proposer and needs to be verified by the network. At runtime, the `ProcessProposal` handler will do the following:
+
+1. Determine the order of lanes it wants to verify the proposal with. If the application is configured to utilize the Block SDK module, it will fetch the governance configured order of lanes and use that. Otherwise, it will use the order defined in your [`app.go`](../tests/app/app.go) file (see our test app as an example).
+2. After determining the order, it will chain together all of the lane's `ProcessLane` methods into a single `ProcessProposal` method.
+3. Given that the proposal contains contiguous sections of transactions from a given lane, each lane will verify the transactions that belong to it and return the remaining transactions to verify for the next lane - see [`ProcessLane`](../block/base/abci.go).
+4. After determining and verifiying the transactions that belong to it, the lane will attempt to update the current proposal - so as to replicate the exact same steps done in `PrepareProposal`. If the lane is unable to update the proposal, it will return an error and the proposal will be rejected.
+5. Once all lanes have been called and no transactions are left to verify, the proposal outputted by `ProcessProposal` should be the same as the proposal outputted by `PrepareProposal`!
+
+If any of the lanes fail during `ProcessLane`, the entire proposal is rejected. There ensures that there is always parity between the proposal built and the proposal verified.
+
+### Proposal Verification Example
+
+Following the example above, let's say we recieve the same proposal from the network:
+
+```golang
+blockProposal := {
+    Tx1, // 100 bytes, 100 gas
+    Tx2, // 800 bytes, 300 gas
+    Tx3, // 200 bytes, 100 gas
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+```
+
+The proposal will be verified as follows (this is pseudo-code):
+
+```golang
+// Somewhere in abci.go a new proposal is created:
+blockProposal := proposals.NewProposal(...)
+...
+// First lane to be called is lane A, this will return the following transactions that it verified and the remaining transactions to verify after calling ProcessLane:
+verifiedTransactionsFromLaneA, remainingTransactions := {
+    Tx1, // 100 bytes, 100 gas
+    Tx2, // 800 bytes, 300 gas
+    Tx3, // 200 bytes, 100 gas
+}, {
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+...
+// First Lane A will update the proposal.
+if err := blockProposal.Update(verifiedTransactionsFromLaneA); err != nil {
+    return err
+}
+...
+// Next, lane B will be called with the following transactions to verify and the remaining transactions to verify after calling ProcessLane:
+verifiedTransactionsFromLaneB, remainingTransactions := {
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+}, {
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+...
+// Next, lane B will update the proposal.
+if err := blockProposal.Update(verifiedTransactionsFromLaneB); err != nil {
+    return err
+}
+...
+// Finally, lane C will be called with the following transactions to verify and the remaining transactions to verify after calling ProcessLane:
+verifiedTransactionsFromLaneC, remainingTransactions := {
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}, {}
+...
+// Finally, lane C will update the proposal.
+if err := blockProposal.Update(verifiedTransactionsFromLaneC); err != nil {
+    return err
+}
+...
+// The final proposal will be:
+blockProposal := {
+    Tx1, // 100 bytes, 100 gas
+    Tx2, // 800 bytes, 300 gas
+    Tx3, // 200 bytes, 100 gas
+    Tx5, // 1000 bytes, 500 gas
+    Tx8, // 1000 bytes, 400 gas
+    Tx9, // 1000 bytes, 500 gas
+    Tx10, // 1200 bytes, 600 gas
+    Tx12, // 100 bytes, 400 gas
+}
+```
+
+As we can see, in the process of verifying a proposal, the proposal is updated to reflect the exact same steps done in `PrepareProposal`.
+

adapters/signer_extraction_adapter/README.md

@@ -0,0 +1,31 @@
+# Signer Extraction Adapter (SEA)
+
+## Overview
+
+The Signer Extraction Adapter is utilized to retrieve the signature information of a given transaction. This is purposefully built to allow application developers to retrieve signer information in the case where the default Cosmos SDK signature information is not applicable. 
+
+## Utilization within the Block SDK
+
+Each lane can configure it's own Signer Extraction Adapter (SEA). However, for almost all cases each lane will have the same SEA. The SEA is utilized to retrieve the address of the signer and nonce of the transaction. It's utilized by each lane's mempool to retrieve signer information as transactions are being inserted and for logging purposes as a proposal is being created / verified.
+
+## Configuration
+
+To extend and implement a new SEA, the following interface must be implemented:
+
+```go
+// Adapter is an interface used to determine how the signers of a transaction should be extracted
+// from the transaction.
+type Adapter interface {
+	GetSigners(sdk.Tx) ([]SignerData, error)
+}
+```
+
+The `GetSigners` method is responsible for extracting the signer information from the transaction. The `SignerData` struct is defined as follows:
+
+```go
+type SignerData struct {
+	Signer   sdk.AccAddress
+	Sequence uint64
+}
+```
+

adapters/signer_extraction_adapter/signer_extraction_adapter.go

@@ -25,7 +25,7 @@ func (s SignerData) String() string {
 	return fmt.Sprintf("SignerData{Signer: %s, Sequence: %d}", s.Signer, s.Sequence)
 }
 
-// SignerExtractionAdapter is an interface used to determine how the signers of a transaction should be extracted
+// Adapter is an interface used to determine how the signers of a transaction should be extracted
 // from the transaction.
 type Adapter interface {
 	GetSigners(sdk.Tx) ([]SignerData, error)

block/README.md

@@ -0,0 +1,103 @@
+# Block SDK Mempool & Lanes
+
+## Overview
+
+> This document describes how the Block SDK mempool and lanes operate at a high level. To learn more about how to construct lanes, please visit the [build my own lane readme](../lanes/build-your-own/README.md) and/or the [base lane documentation](./base/README.md). To read about how proposals are construct, visit the [abci readme](../abci/README.md).
+
+Mempools are traditionally used to temporarily store transactions before they are added to a block. The Block SDK mempool is no different. However, instead of treating each transaction the same, the Block SDK allows for developers to create `Lanes` that permit transactions to be ordered differently based on the properties of the transaction itself.
+
+What was once a single monolithic data structure, is now a collection of sub-mempools that can be configured to order transactions in a way that makes sense for the application.
+
+## Lanes
+
+Lanes are utilized to allow developers to create custom transaction order, validation, and execution logic. Each lane is responsible for maintaining its own mempool - ordering transactions as it desires only for the transactions it wants to accept. For example, a lane may only accept transactions that are staking related, such as the free lane. The free lane may then order the transactions based on the user's on-chain stake.
+
+When proposals are constructed, the transactions from a given lane are selected based on highest to lowest priority, validated according to the lane's verfication logic, and included in the proposal.
+
+Each lane must implement the `Lane` interface, although it is highly recommended that developers extend the [base lane](./base/README.md) to create new lanes.
+
+```go
+// LaneMempool defines the interface a lane's mempool should implement. The basic API
+// is the same as the sdk.Mempool, but it also includes a Compare function that is used
+// to determine the relative priority of two transactions belonging in the same lane.
+type LaneMempool interface {
+	sdkmempool.Mempool
+
+	// Compare determines the relative priority of two transactions belonging in the same lane. Compare
+	// will return -1 if this transaction has a lower priority than the other transaction, 0 if they have
+	// the same priority, and 1 if this transaction has a higher priority than the other transaction.
+	Compare(ctx sdk.Context, this, other sdk.Tx) (int, error)
+
+	// Contains returns true if the transaction is contained in the mempool.
+	Contains(tx sdk.Tx) bool
+
+	// Priority returns the priority of a transaction that belongs to this lane.
+	Priority(ctx sdk.Context, tx sdk.Tx) any
+}
+
+// Lane defines an interface used for matching transactions to lanes, storing transactions,
+// and constructing partial blocks.
+type Lane interface {
+	LaneMempool
+
+	// PrepareLane builds a portion of the block. It inputs the current context, proposal, and a
+	// function to call the next lane in the chain. This handler should update the context as needed
+	// and add transactions to the proposal. Note, the lane should only add transactions up to the
+	// max block space for the lane.
+	PrepareLane(
+		ctx sdk.Context,
+		proposal proposals.Proposal,
+		next PrepareLanesHandler,
+	) (proposals.Proposal, error)
+
+	// ProcessLane verifies this lane's portion of a proposed block. It inputs the current context,
+	// proposal, transactions that belong to this lane, and a function to call the next lane in the
+	// chain. This handler should update the context as needed and add transactions to the proposal.
+	// The entire process lane chain should end up constructing the same proposal as the prepare lane
+	// chain.
+	ProcessLane(
+		ctx sdk.Context,
+		proposal proposals.Proposal,
+		txs []sdk.Tx,
+		next ProcessLanesHandler,
+	) (proposals.Proposal, error)
+
+	// GetMaxBlockSpace returns the max block space for the lane as a relative percentage.
+	GetMaxBlockSpace() math.LegacyDec
+
+	// SetMaxBlockSpace sets the max block space for the lane as a relative percentage.
+	SetMaxBlockSpace(math.LegacyDec)
+
+	// Name returns the name of the lane.
+	Name() string
+
+	// SetAnteHandler sets the lane's antehandler.
+	SetAnteHandler(antehander sdk.AnteHandler)
+
+	// Match determines if a transaction belongs to this lane.
+	Match(ctx sdk.Context, tx sdk.Tx) bool
+
+	// GetTxInfo returns various information about the transaction that
+	// belongs to the lane including its priority, signer's, sequence number,
+	// size and more.
+	GetTxInfo(ctx sdk.Context, tx sdk.Tx) (utils.TxWithInfo, error)
+}
+```
+
+## Lane Priorities
+
+Each lane has a priority that is used to determine the order in which lanes are processed. The higher the priority, the earlier the lane is processed. For example, if we have three lanes - MEV, free, and default - proposals will be constructed in the following order:
+
+1. MEV
+2. Free
+3. Default
+
+Proposals will then be verified in the same order. Please see the [readme above](../abci/README.md) for more information on how proposals are constructed using lanes.
+
+The ordering of lane's priorities is determined based on the order passed into the constructor of the Block SDK mempool i.e. `LanedMempool`.
+
+## Block SDK mempool
+
+The `LanedMempool` is a wrapper on top of the collection of lanes. It is solely responsible for adding transactions to the appropriate lanes. Transactions are always inserted / removed to the first lane that accepts / matches the transactions. **Transactions should only match to one lane.**. **In the case where a transaction can match to multiple lanes, the transaction will be inserted into the lane that has the highest priority.**
+
+To read more about the underlying implementation of the Block SDK mempool, please see the implementation [here](./mempool.go).