Electra: Attestation, Aggregation and Proposal Alignment #511

MatheusFranco99 · 2024-11-16T13:52:44Z

Overview

This PR aligns the construction of attestations, aggregations and proposals to the latest changes by the electra fork.

Proposal

Since a versioned data was already fetched and submitted to the beacon node, it was only necessary to accept the Electra version in the ValidatorConsensusData validation.

Attestation

The changes follow the ethereum electra spec for constructing attestations.

Starts using the spec.VersionedAttestation type as the output of the attestation duty. For DataVersionElectra, it represents a electra.Attestation object. For previous versions, it represents a phase0.Attestation.
For creating the input for consensus, the CommitteeRunner still fetches a phase0.AttestationData type since it remains unchanged and is sufficient for constructing the BeaconVote. However, when being constructed after electra, the Index field that identifies the committee is set to 0 (EIP-7549).

Note

The Ethereum specification states that, under electra, a electra.SingleAttestation object should be returned. However, the CommitteeRunner outputs a spec.VersionedAttestation to the BeaconNode as this is the type specified in the go-eth2-client library. Internally, the library transforms the versioned object into a electra.SingleAttestation later on.

Note

The CommitteeRunner will keep running consensus over a BeaconVote. Even though it doesn't include a DataVersion, operators are assumed to compute the correct DataVersion given the duty's slot.

Aggregation

The changes follow the ethereum electra spec for constructing aggregations.

Starts using the VersionedSignedAggregateAndProof type from the go-eth2-client library. For DataVersionElectra, it represents a electra.SignedAggregateAndProof object. For previous versions, it represents a phase0.SignedAggregateAndProof.

Beacon Node

SubmitAttestation now accepts spec.VersionedAttestation rather than phase0.Attestation.
SubmitSignedAggregateSelectionProof now accepts spec.VersionedSignedAggregateAndProof rather than phase0.SignedAggregateAndProof.
SubmitAggregateSelectionProof now should return either a phase0.SignedAggregateAndProof or a electra.SignedAggregateAndProof, depending on the version.
A new method, DataVersion(epoch phase0.Epoch), is added to return the DataVersion given the epoch. Note that, for this frequent API call, it's recommended for the implementation to use a cache.

Tests

All tests for the Aggregator duty and Committee duty (that encompasses the Attester duty) became versioned with a test for version DataVersionPhase0 and another for DataVersionElectra. Tests for the Proposal duty now includes the Electra version.

Testing Utils

Major refactoring for better separating specific duties utils, and for versioning utils for the aggregator and committee duties.

* remove ssz encoding for CommitteeMember and Share * remove commented code --------- Co-authored-by: Alan Li <[email protected]>

…labs#494) * remove domaintype from committeeMember * remove domaintype from share * add config for ssv containing domainType * add custom unmarshal for baserunner to avoid error in unmarshalling config * move domainType to Network * add testing domain in NewTestingNetwork function * Spec - filter committee duties per slot validator duties (ssvlabs#467) * Filter shares for slot `CommitteeRunner` based on validators that have duty for that slot. * Filter duty and create share map according to owned validators * Add test: start duty with no shares for duty's validators * Add test: happy flow for committee with fraction of duty's validators * Generate JSON tests * Apply suggestions --------- Co-authored-by: MatheusFranco99 <[email protected]> * Spec - check att and sync duties exist before submitting (ssvlabs#468) * Meta - Update to go1.22 (ssvlabs#474) * Update go1.20 to go1.22 * Update go.sum with mod tidy * Meta - Update dependencies (ssvlabs#483) * Update dependencies * Fix lint issue * Generate JSON tests to trigger actions * Update fastssz * Generate JSON tests and align ssz error * Revert go-eth2-client version change * Revert fastssz upgrade * Generate SSZ and JSON tests * Meta - Fix static analysis issues (ssvlabs#480) * Solve potential file inclusion via variable * Fix file permission (0644 to 0600) * Add nosec comment for PRNG (pseudo-random number generator) used for testing * Fix lint issue on nil check in []byte type * Update permission from 0444 to 0600 * Update 0444 to 0400 * Meta - Drop unnecessary nolint comments (ssvlabs#477) * Remove nolint comment and export timeout variables * Drop unnecessary nolint * Add comment * Fix lint issue * Spec - Share length validation (ssvlabs#478) * Add share length validation in runner construction * Align to error handling in runners constructions * Add validation to committee runner * Add runner construction tests * Refactor runner construction in testingutil to deal with creation errors * Generate JSON tests * Fix lint issue * Fix comments * Meta - Drop redundant error (ssvlabs#475) * Spec - Drop redundant validation for decided messages (ssvlabs#476) * Remove redundant validation * Align error string * Spec - Sort signers in decided message (ssvlabs#484) * Sort signers in decided message * Add test for sorted signers in decided msg * Generate JSON tests * Fix lint issue * Spec - Stop processing after decided (ssvlabs#487) * Stop processing consensus messages after instance is decided * Align error in qbft tests * Align errors in ssv tests * Generate JSON tests * Fix lint issue * Spec - Drop leftover error check (ssvlabs#469) * Remove leftover err check * Align argument variable name to type * Spec - Secure key storage (ssvlabs#481) * Implement secure key storage using PKCS8 * Fix lint issue * Switch back PKCS8 to PKCS1 * Meta - Remove residual DKG (ssvlabs#502) * Remove DKG signature type * Remove DKG msg type * Remove DKGOperators field from TestKeySet * Remove unused ecdsaKeys field from TestingKeyStorage * Remove unused "ecdsaSKFromHex" function * Generate JSON tests * Spec - Add GoSec action and fix issues (ssvlabs#505) * Add github action and makefile command * Fix issues in round robin proposer function * Fix bad PutUint32 in GetCommitteeID * Fix issue with HasQuorum and HasPartialQuorum * Add role sanitization in GetRoleType and NewMessageType * Add sanitization to BeaconNetwork methods * Add sanitization in testingutils * Add sanitization to height usage in test files * Fix uint64 conversion in runner/postconsensus/valid_msg test * Sanitize ValidatorIndex conversion * Update action name * Fix tests to use valid RunnerRoles * Generate SSZ * Generate JSON tests * Revert the change on GetCommitteeID * Add nosec G115 to GetCommitteeID * revert the merge because it was merged with origin main by accident --------- Co-authored-by: rehs0y <[email protected]> Co-authored-by: MatheusFranco99 <[email protected]>

* Solve entropy in simple case in which a fixed key can be used * Solve entropy in test type by changing the test spec: removing CypherText and using fixed keys

qbft/round_robin_proposer.go

GalRogozinski

You have long switch case statements that go over all block versions... but we only need two in practice: Deneb and Electra.

so maybe reduce the code?

Also optionally (not a blocker) it will be nice to have all the Ethereum specific logic in another file, separate from committee runner. This is because those are specifications that don't relate to our protocol

ssv/committee_runner.go

MatusKysel · 2025-02-10T13:06:33Z

types/consensus_data.go

@@ -7,6 +7,7 @@ import (
 	"github.com/attestantio/go-eth2-client/spec"
 	"github.com/attestantio/go-eth2-client/spec/altair"


and you need to adjust GetBlindedBlockData and GetBlockData to support electra

GalRogozinski

Good job!
After you resolve conflicts and go over my comments Alan must review. This needs to be approved on Friday.

GalRogozinski · 2025-02-13T20:42:22Z

types/consensus_data.go

+	case spec.DataVersionPhase0:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Phase0: ret}, ret, nil
+
+	case spec.DataVersionAltair:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Altair: ret}, ret, nil
+
+	case spec.DataVersionBellatrix:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Bellatrix: ret}, ret, nil
+
+	case spec.DataVersionCapella:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Capella: ret}, ret, nil


maybe we can delete the old versions?

maybe we could keep it? as on some testnets/devnets you could still use old versions

Yes, I think keeping is better

types/messages_encoding.go

types/consensus_data_encoding.go

GalRogozinski · 2025-02-13T20:55:26Z

ssv/aggregator.go

+	case spec.DataVersionPhase0:
+		ret.Phase0 = &phase0.SignedAggregateAndProof{
+			Message:   aggregateAndProof.Phase0,
+			Signature: signature,
+		}
+	case spec.DataVersionAltair:
+		ret.Altair = &phase0.SignedAggregateAndProof{
+			Message:   aggregateAndProof.Altair,
+			Signature: signature,
+		}
+	case spec.DataVersionBellatrix:
+		ret.Bellatrix = &phase0.SignedAggregateAndProof{
+			Message:   aggregateAndProof.Bellatrix,
+			Signature: signature,
+		}
+	case spec.DataVersionCapella:
+		ret.Capella = &phase0.SignedAggregateAndProof{
+			Message:   aggregateAndProof.Capella,
+			Signature: signature,
+		}


why do you have all the old versions?

For this structure, I'm doing the same pattern as in go-eth2-client.

GalRogozinski · 2025-02-13T20:57:16Z

types/consensus_data.go

@@ -203,6 +205,12 @@ func (ci *ValidatorConsensusData) GetBlockData() (blk *api.VersionedProposal, si
 			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
 		}
 		return &api.VersionedProposal{Deneb: ret, Version: ci.Version}, ret.Block, nil
+	case spec.DataVersionElectra:


you may delete the Capella test

I think it's worth keeping it for testnets, even for simulation/private ones, that could use an old version. Wdyt?

GalRogozinski · 2025-02-13T20:57:57Z

types/consensus_data.go

+	case spec.DataVersionPhase0:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Phase0: ret}, ret, nil
+
+	case spec.DataVersionAltair:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Altair: ret}, ret, nil
+
+	case spec.DataVersionBellatrix:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+
+		return &spec.VersionedAggregateAndProof{Version: ci.Version, Bellatrix: ret}, ret, nil
+
+	case spec.DataVersionCapella:
+		ret := &phase0.AggregateAndProof{}
+		if err := ret.UnmarshalSSZ(ci.DataSSZ); err != nil {
+			return nil, nil, errors.Wrap(err, "could not unmarshal ssz")
+		}
+


no need for the old versions

I see, but, for this structure, I'm doing the same pattern as in go-eth2-client.

types/testingutils/beacon_node_consts.go

GalRogozinski

Good Job!

alan-ssvlabs and others added 7 commits November 12, 2024 13:08

remove ssz encoding for CommitteeMember and Share (ssvlabs#490)

93bc347

* remove ssz encoding for CommitteeMember and Share * remove commented code --------- Co-authored-by: Alan Li <[email protected]>

reset on dev

45cc5f0

Tests - Remove entropy in tests (ssvlabs#507)

f7eeb58

* Solve entropy in simple case in which a fixed key can be used * Solve entropy in test type by changing the test spec: removing CypherText and using fixed keys

stop using CommitteeIndex in AttestationData

d854ddd

stop using CommitteeIndex in testing utils

0787fb1

generate JSON tests

11ef6a0

MatheusFranco99 self-assigned this Nov 16, 2024

MatheusFranco99 changed the base branch from main to dev November 16, 2024 13:53

GalRogozinski force-pushed the dev branch 3 times, most recently from b5bd1d3 to bf0ff16 Compare December 19, 2024 22:36

GalRogozinski deleted the branch ssvlabs:main January 9, 2025 08:21

GalRogozinski closed this Jan 9, 2025

GalRogozinski reopened this Jan 9, 2025

MatheusFranco99 added 2 commits January 20, 2025 12:29

update attestantio/go-eth2-client

2f584c3

set committee index to 0 as prposed in EIP-7549

3a00520

MatheusFranco99 changed the title ~~Drop committee index~~ Electra: Attestation Alignment Jan 20, 2025

MatheusFranco99 added 3 commits January 27, 2025 13:24

use electra.SingleAttestation for attester duty

31e394c

align attestation tests to submit electra.SingleAttestation

e720d55

align to pectra aggregator data type

be3ec81

oleg-ssvlabs reviewed Jan 28, 2025

View reviewed changes

qbft/round_robin_proposer.go Outdated Show resolved Hide resolved

MatheusFranco99 changed the title ~~Electra: Attestation Alignment~~ Electra: Attestation and Aggregation Alignment Jan 28, 2025

MatheusFranco99 added 3 commits January 28, 2025 13:27

use versioned attestation response for attestation

637965c

add comment about data version api call

8d01300

Merge branch 'main' into drop-committee-index

19a25e7

MatheusFranco99 changed the base branch from dev to main January 28, 2025 14:16

MatheusFranco99 added 3 commits January 28, 2025 18:06

refactor beacon node testingutils, separate beacon role by file

022785e

make AttestationData construction versioned

9dc6eb9

revert valuecheck redundant change

d5b3da5

rename variable that overwrites an outer-scope one

5b57e06

MatusKysel approved these changes Jan 31, 2025

View reviewed changes

GalRogozinski reviewed Feb 3, 2025

View reviewed changes

MatusKysel reviewed Feb 5, 2025

View reviewed changes

ssv/committee_runner.go Outdated Show resolved Hide resolved

MatheusFranco99 added 7 commits February 5, 2025 10:29

include validator index in versioned attestation

930e111

test objects sent on the wire (single attestation)

5fc28d6

align testingutils to use single attestation

6c40a08

update go-eth2-client version in generate file

fcb8b30

refactor versioned attestation construction

25ca34b

align to new ssz error msg

ca9c4d2

align to go-eth2-client 0.24.0 release

987e181

MatusKysel reviewed Feb 10, 2025

View reviewed changes

GalRogozinski requested a review from MatusKysel February 10, 2025 13:56

MatheusFranco99 added 3 commits February 13, 2025 07:31

support electra blocks

129ab69

align testing for electra blocks

4ec8d28

generate JSON tests

d3fcba1

MatheusFranco99 changed the title ~~Electra: Attestation and Aggregation Alignment~~ Electra: Attestation, Aggregation and Proposal Alignment Feb 13, 2025

y0sher approved these changes Feb 13, 2025

View reviewed changes

MatusKysel approved these changes Feb 13, 2025

View reviewed changes

fix beacon block blinded test

da25db0

GalRogozinski approved these changes Feb 13, 2025

View reviewed changes

GalRogozinski self-requested a review February 13, 2025 15:44

GalRogozinski reviewed Feb 13, 2025

View reviewed changes

GalRogozinski self-requested a review February 13, 2025 21:26

Merge branch 'main' into drop-committee-index

375ae12

MatheusFranco99 mentioned this pull request Feb 14, 2025

Use test electra.BlockContents from go-eth2-client #521

Open

add issue

5119ac4

GalRogozinski approved these changes Feb 16, 2025

View reviewed changes

GalRogozinski merged commit de6806c into ssvlabs:main Feb 16, 2025
2 checks passed

GalRogozinski deleted the drop-committee-index branch February 16, 2025 10:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Electra: Attestation, Aggregation and Proposal Alignment #511

Electra: Attestation, Aggregation and Proposal Alignment #511

MatheusFranco99 commented Nov 16, 2024 •

edited

Loading

GalRogozinski left a comment

MatusKysel Feb 10, 2025

GalRogozinski left a comment

GalRogozinski Feb 13, 2025

MatusKysel Feb 14, 2025

MatheusFranco99 Feb 14, 2025

GalRogozinski Feb 13, 2025

MatheusFranco99 Feb 14, 2025 •

edited

Loading

GalRogozinski Feb 13, 2025

MatheusFranco99 Feb 14, 2025 •

edited

Loading

GalRogozinski Feb 13, 2025

MatheusFranco99 Feb 14, 2025

GalRogozinski left a comment

		@@ -7,6 +7,7 @@ import (
		"github.com/attestantio/go-eth2-client/spec"
		"github.com/attestantio/go-eth2-client/spec/altair"

Electra: Attestation, Aggregation and Proposal Alignment #511

Electra: Attestation, Aggregation and Proposal Alignment #511

Conversation

MatheusFranco99 commented Nov 16, 2024 • edited Loading

Overview

Proposal

Attestation

Aggregation

Beacon Node

Tests

Testing Utils

GalRogozinski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GalRogozinski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MatheusFranco99 Feb 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MatheusFranco99 Feb 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GalRogozinski left a comment

Choose a reason for hiding this comment

MatheusFranco99 commented Nov 16, 2024 •

edited

Loading

MatheusFranco99 Feb 14, 2025 •

edited

Loading

MatheusFranco99 Feb 14, 2025 •

edited

Loading