QBFT + shanghai support by matthew1001 · Pull Request #6353 · besu-eth/besu

matthew1001 · 2024-01-04T17:42:05Z

PR description

Implement shanghai support for QBFT and IBFT chains.

Main areas of change:

Previously the code made the assumption that BFT block creators and validators could be created ahead of time, on creation of a new QbftRound/IbftRound, since the next block number was known ahead of time. With number-based milestones such as london this is fine, but with timestamp-based milestones the type of EVM to use for a new round needs to be based on the time that the block is being created. As such, in many places a protocolSchedule is now passed down the stack instead of a blockValidator or a blockImporter to allow these to be created using the protocolSchedule when the timestamp is known.
The BftProtocolSchedule was only designed to allow a protocol spec to be requested by block number, so a new method getByBlockNumberAndTimestamp that takes both the block number and timestamp has replaced getByBlockNumber, and determines the fork spec based on whichever milestone is most recent.

I don't think any docs changes are required since we don't currently make any mention of shanghai in the docs today.

Fixed Issue(s)

Fixes #5446

github-actions · 2024-01-04T17:42:23Z

I thought about documentation and added the doc-change-required label to this PR if updates are required.
I thought about the changelog and included a changelog update if required.
If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

consensus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRound.java

...us/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRoundFactory.java

...ft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidatorFactory.java

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

…age validator creation Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Signed-off-by: Matt Whitehead <matthew1001@gmail.com>

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

jframe · 2024-01-30T04:44:17Z

...nsus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/BftProtocolSchedule.java

+   * @return the protocol spec for that block number and timestamp
   */
-  public ProtocolSpec getByBlockNumber(final long number) {
+  public ProtocolSpec getByBlockNumberAndTimestamp(final long number, final long timestamp) {


I think this getting a protocol spec by either the block number or timestamp. Better name would be getByBlockNumberOrTimestamp

Done - updated to getByBlockNumberOrTimestamp

jframe · 2024-01-30T04:53:49Z

consensus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRound.java

    LOG.trace("Importing block with extraData={}", extraData);
+    final BlockImporter blockImporter =
+        ((BftProtocolSchedule) protocolSchedule)
+            .getByBlockNumberAndTimestamp(


I think this could just use protocolSchedule.getByBlockHeader(blockToImport.getHeader()) instead. It will look up protocol schedules by both block timestamps and block numbers.

jframe · 2024-01-30T04:56:04Z

...nsus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidator.java

+
+    final BlockValidator blockValidator =
+        ((BftProtocolSchedule) protocolSchedule)
+            .getByBlockNumberAndTimestamp(


Similar to the IbftRound I think you could use the existing protocolSchedule.getByBlockHeader method here instead

jframe · 2024-01-30T05:00:18Z

consensus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/statemachine/QbftRound.java

    }
    LOG.trace("Importing proposed block with extraData={}", extraData);
+    final BlockImporter blockImporter =
+        ((BftProtocolSchedule) protocolSchedule)


Same in the IbftRound think we get away with using the protocolSchedule.getByBlockHeader method here

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 · 2024-01-30T17:02:16Z

Thanks for the latest comments @jframe. I think the latest commits resolve them - let me know what you think.

Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>

...nsus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidator.java

consensus/qbft/src/integration-test/resources/genesis_validator_contract_shanghai.json

...t/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalPayloadValidator.java

...sus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalValidator.java

...rc/main/java/org/hyperledger/besu/consensus/qbft/validation/RoundChangeMessageValidator.java

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

jframe · 2024-02-02T00:38:14Z

...sus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalValidator.java

+    checkState(
+        protocolSchedule instanceof BftProtocolSchedule,
+        "Wrong class type for protocol schedule, requires BftProtocolSchedule");


nit: don't need this check anymore

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch 11 times, most recently from 7bb70c0 to c1f726f Compare January 8, 2024 17:55

jframe reviewed Jan 9, 2024

View reviewed changes

matthew1001 force-pushed the qbft-shanghai branch 5 times, most recently from 0ba4bda to 3e4e682 Compare January 11, 2024 11:35

Refactored version of QBFT/shanghai

7cd7178

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch from 3e4e682 to 7cd7178 Compare January 11, 2024 11:54

More refactoring

1498f91

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch 5 times, most recently from f73e9e0 to 8195395 Compare January 16, 2024 11:06

Add equivalent IBFT fixes. Update QBFT and IBFT round tests

e08527b

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch from 8195395 to e08527b Compare January 16, 2024 14:31

matthew1001 added 2 commits January 16, 2024 15:57

Update the change log

f284d5f

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Remove old comments

20c7d37

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch from 7766e1a to 8665212 Compare January 17, 2024 17:23

Add Shanghai to the combined protocol schedule test, update IBFT mess…

41ab356

…age validator creation Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch from 8665212 to 41ab356 Compare January 17, 2024 17:34

matthew1001 marked this pull request as ready for review January 17, 2024 17:43

matthew1001 changed the title ~~WIP: QBFT + shanghai support~~ QBFT + shanghai support Jan 17, 2024

matthew1001 added 2 commits January 17, 2024 17:50

Merge branch 'main' into qbft-shanghai

a99e3e1

Signed-off-by: Matt Whitehead <matthew1001@gmail.com>

Fix merge

b9caf13

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

matthew1001 force-pushed the qbft-shanghai branch from 10fe48f to b9caf13 Compare January 18, 2024 15:08

matthew1001 added 2 commits January 18, 2024 16:24

Update tests to use shanghai time > 0

6d9e841

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Merge branch 'main' into qbft-shanghai

4fb8d60

jframe reviewed Jan 30, 2024

View reviewed changes

matthew1001 and others added 5 commits January 30, 2024 13:44

Address PR comments

c76c6c1

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Merge branch 'main' into qbft-shanghai

6f00341

Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>

Refactor unit tests

0537e8a

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Refactor unit tests

e2fb007

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

IbftRoundTests update

c832dc3

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Merge branch 'main' into qbft-shanghai

7d44693

Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>

matthew1001 mentioned this pull request Jan 31, 2024

Problem syncing (withdrawals) in besu 24.1.1 with QBFT y shanghai #6507

Closed

jframe reviewed Jan 31, 2024

View reviewed changes

matthew1001 and others added 2 commits February 1, 2024 10:42

Address PR comments

5707343

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Merge branch 'main' into qbft-shanghai

9be5ce5

jframe approved these changes Feb 2, 2024

View reviewed changes

matthew1001 and others added 2 commits February 2, 2024 08:51

Remove unnecessary class type check

e36fc94

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

Merge branch 'main' into qbft-shanghai

97c5041

matthew1001 enabled auto-merge (squash) February 2, 2024 08:51

matthew1001 merged commit 03dd7f1 into besu-eth:main Feb 2, 2024

matthew1001 mentioned this pull request Mar 19, 2024

QBFT/Shanghai blocks failing withdrawal validation on import #6760

Closed

matthew1001 mentioned this pull request May 16, 2024

BFT ignore withdrawals #7115

Merged

siladu mentioned this pull request Apr 14, 2025

Shanghai support for Clique network #8532

Closed

Conversation

matthew1001 commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description

Fixed Issue(s)

Uh oh!

github-actions bot commented Jan 4, 2024 • edited by matthew1001 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthew1001 commented Jan 30, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matthew1001 commented Jan 4, 2024 •

edited

Loading

github-actions bot commented Jan 4, 2024 •

edited by matthew1001

Loading