Skip to content

Commit a9957ce

Browse files
matthew1001jframemacfarla
authored
Bonsai archive feature (#7475)
* rebase of bonsai archive Signed-off-by: Jason Frame <jason.frame@consensys.net> * Change BonsaiReferenceTestWorldStateStorage getContextSafeCopy() to use constructor that reuses worldStateStorage so that we don't lose values in the EvmToolSpecTests Signed-off-by: Jason Frame <jason.frame@consensys.net> * Force archive mode for testing Signed-off-by: Jason Frame <jason.frame@consensys.net> * Rebase on main Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Create new DiffBasedWorldStateConfig when copying bonsai archive world state, and freeze it Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add BONSAI_ARCHIVE storage format Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * New ACCOUNT_FREEZER_STATE DB segment. New BonsaiArchiveFreezer to listen for blocks and move account state to new DB segment Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add freezer segment for account storage Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Find frozen storage slots correctly, refactor the DB segment names Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Check for blocks to freeze state for on startup. Store the most recent block state has been frozen for Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Make sure genesis world state is created in archive mode Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Fix incorrect logs Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Ensure deleted storage is returned from live DB segment, not old storage from the freezer segment Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Adding tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Refactor DB TX with retries Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Honour code hash or account hash config for storing code. Add tests for DB mode Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Refactor code to move one block's worth of state/storage changes at a time. Add more tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Refactoring, fix issues, ensure only 1 batch of blocks is archived at a time Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Improve archive progress logs Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add archive mode tests to bonsai key/value tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Implement snap-serving from a Bonsai Archive node and add tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Parameterize snap-server tests, fix bug in storage slot ranges Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Remove todo comments Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Make BONSAI_ARCHIVE experimental for the first release. Add metrics. Use the term archive, not freezer Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add changelog entry Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Don't store every block that is waiting to be archived Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Remove pre-startup catchup. Start on first block import instead Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Multiple deletes of the same address in a single block causes Bonsai to fail the block Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Metric should be a gauge not a counter Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Refactor archive tests after code refactor Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Remove commented out sections from tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update the BFT soak test to include a Bonsai archive node Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add archive-specific checks to the BFT soak test Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update plugin API hash Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add javadoc Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Merge fix Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Unit test fix Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * More UT fixing Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Tidy up Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/trie/diffbased/bonsai/worldview/BonsaiArchiver.java Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Matt Whitehead <matthew1001@hotmail.com> * Update plugins/rocksdb/src/main/java/org/hyperledger/besu/plugin/services/storage/rocksdb/configuration/BaseVersionedStorageFormat.java Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Matt Whitehead <matthew1001@hotmail.com> * Update plugins/rocksdb/src/main/java/org/hyperledger/besu/plugin/services/storage/rocksdb/configuration/BaseVersionedStorageFormat.java Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Matt Whitehead <matthew1001@hotmail.com> * Remove debug file Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Various refactoring to manage archive flat DB access more cleanly Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Merge fix Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Merge tests Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Use alternative clearStorage() Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Fix metric name Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Don't override rollMutableStateToBlockHash, create archive-specific function name Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update world state with block number being persisted Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update changelog Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Move context management out of flat DB strategy Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Add more test cases, tidy up code Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update plugin hash Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Address review comments Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Persist current block number only when we have a valid block & have calculated its root hash Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Ensure genesis context is initialised for archive Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update k/v storage test Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Tidy up after merge Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Put/get keys as bytes not hex Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update plugin hash Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Test fix Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Update plugin hash Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> * Apply checks for deleted values to entries from archive table Signed-off-by: Matthew Whitehead <matthew.whitehead@kaleido.io> * API hash Signed-off-by: Matthew Whitehead <matthew.whitehead@kaleido.io> * Throw error if trying to revert archive storage format version Signed-off-by: Matthew Whitehead <matthew.whitehead@kaleido.io> * Update plugin hash Signed-off-by: Matthew Whitehead <matthew.whitehead@kaleido.io> --------- Signed-off-by: Jason Frame <jason.frame@consensys.net> Signed-off-by: Matthew Whitehead <matthew1001@gmail.com> Signed-off-by: Matt Whitehead <matthew1001@hotmail.com> Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io> Signed-off-by: Matthew Whitehead <matthew.whitehead@kaleido.io> Co-authored-by: Jason Frame <jason.frame@consensys.net> Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
1 parent 2c87eac commit a9957ce

File tree

49 files changed

+3388
-170
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+3388
-170
lines changed

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
- EIP-7823 - Modexp upper bounds [#8632](https://github.com/hyperledger/besu/pull/8632)
3131
- EIP-7892 - Max number of blobs per transaction [#8761](https://github.com/hyperledger/besu/pull/8761)
3232
- Implement rewardPercentile cap in eth_feeHistory [#8748](https://github.com/hyperledger/besu/pull/8748)
33+
- Experimental Bonsai Archive support [#7475](https://github.com/hyperledger/besu/pull/7475)
3334

3435
### Bug fixes
3536

@@ -108,6 +109,7 @@
108109
#### Dependencies
109110
- Update discovery library to 25.4.0 [#8635](https://github.com/hyperledger/besu/pull/8635)
110111
- Upgrade Gradle to 8.14 and related plugins [#8638](https://github.com/hyperledger/besu/pull/8638)
112+
- Make gas estimation strict by default [#8629](https://github.com/hyperledger/besu/pull/8629)
111113

112114
### Bug fixes
113115
- Fix `besu -X` unstable options help [#8662](https://github.com/hyperledger/besu/pull/8662)

‎acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/configuration/BesuNodeFactory.java‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -561,7 +561,9 @@ public BesuNode createQbftNode(
561561
.dataStorageConfiguration(
562562
storageFormat == DataStorageFormat.FOREST
563563
? DataStorageConfiguration.DEFAULT_FOREST_CONFIG
564-
: DataStorageConfiguration.DEFAULT_BONSAI_CONFIG)
564+
: storageFormat == DataStorageFormat.BONSAI
565+
? DataStorageConfiguration.DEFAULT_BONSAI_CONFIG
566+
: DataStorageConfiguration.DEFAULT_BONSAI_ARCHIVE_CONFIG)
565567
.genesisConfigProvider(GenesisConfigurationFactory::createQbftGenesisConfig);
566568
if (fixedPort) {
567569
builder.p2pPort(

‎acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/bft/BftAcceptanceTestParameterization.java‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,11 @@ public BesuNode createBonsaiNodeFixedPort(BesuNodeFactory factory, String name)
7474
return creatorFn.create(factory, name, true, DataStorageFormat.BONSAI);
7575
}
7676

77+
public BesuNode createBonsaiArchiveNodeFixedPort(BesuNodeFactory factory, String name)
78+
throws Exception {
79+
return creatorFn.create(factory, name, true, DataStorageFormat.X_BONSAI_ARCHIVE);
80+
}
81+
7782
public BesuNode createForestNodeFixedPort(BesuNodeFactory factory, String name) throws Exception {
7883
return creatorFn.create(factory, name, true, DataStorageFormat.FOREST);
7984
}

‎acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/bftsoak/BftMiningSoakTest.java‎

Lines changed: 61 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,15 +33,23 @@
3333
import org.junit.jupiter.api.AfterEach;
3434
import org.junit.jupiter.params.ParameterizedTest;
3535
import org.junit.jupiter.params.provider.MethodSource;
36+
import org.slf4j.Logger;
37+
import org.slf4j.LoggerFactory;
38+
import org.web3j.protocol.core.DefaultBlockParameter;
39+
import org.web3j.tx.exceptions.ContractCallException;
3640

3741
public class BftMiningSoakTest extends ParameterizedBftTestBase {
3842

43+
private static final Logger LOG = LoggerFactory.getLogger(BftMiningSoakTest.class);
44+
3945
private final int NUM_STEPS = 5;
4046

4147
private final int MIN_TEST_TIME_MINS = 60;
4248

4349
private static final long ONE_MINUTE = Duration.of(1, ChronoUnit.MINUTES).toMillis();
4450

51+
private static final long THREE_MINUTES = Duration.of(3, ChronoUnit.MINUTES).toMillis();
52+
4553
private static final long TEN_SECONDS = Duration.of(10, ChronoUnit.SECONDS).toMillis();
4654

4755
static int getTestDurationMins() {
@@ -61,7 +69,7 @@ public void shouldBeStableDuringLongTest(
6169

6270
// Create a mix of Bonsai and Forest DB nodes
6371
final BesuNode minerNode1 = nodeFactory.createBonsaiNodeFixedPort(besu, "miner1");
64-
final BesuNode minerNode2 = nodeFactory.createForestNodeFixedPort(besu, "miner2");
72+
final BesuNode minerNode2 = nodeFactory.createBonsaiArchiveNodeFixedPort(besu, "miner2");
6573
final BesuNode minerNode3 = nodeFactory.createBonsaiNodeFixedPort(besu, "miner3");
6674
final BesuNode minerNode4 = nodeFactory.createForestNodeFixedPort(besu, "miner4");
6775

@@ -81,6 +89,13 @@ public void shouldBeStableDuringLongTest(
8189
SimpleStorage simpleStorageContract =
8290
minerNode1.execute(contractTransactions.createSmartContract(SimpleStorage.class));
8391

92+
// Create another instance of the contract referencing the same contract address but on the
93+
// archive node. This contract instance should be able to query state from the beginning of
94+
// the test
95+
SimpleStorage simpleStorageArchive =
96+
minerNode2.execute(contractTransactions.createSmartContract(SimpleStorage.class));
97+
simpleStorageArchive.setContractAddress(simpleStorageContract.getContractAddress());
98+
8499
// Check the contract address is as expected for this sender & nonce
85100
contractVerifier
86101
.validTransactionReceipt("0x42699a7612a82f1d9c36148af9c77354759b210b")
@@ -103,6 +118,9 @@ public void shouldBeStableDuringLongTest(
103118
// Set to something new
104119
simpleStorageContract.set(BigInteger.valueOf(101)).send();
105120

121+
// Save this block height to check on the archive node at the end of the test
122+
BigInteger archiveChainHeight = minerNode1.execute(ethTransactions.blockNumber());
123+
106124
// Check the state of the contract has updated correctly. We'll set & get this several times
107125
// during the test
108126
assertThat(simpleStorageContract.get().send()).isEqualTo(BigInteger.valueOf(101));
@@ -131,6 +149,7 @@ public void shouldBeStableDuringLongTest(
131149
// Step 2
132150
// Stop one of the nodes, check that the chain continues mining
133151
// blocks
152+
LOG.info("Stopping node 4 to check the chain continues mining (albeit more slowly)");
134153
stopNode(minerNode4);
135154

136155
nextStepEndTime =
@@ -150,6 +169,7 @@ public void shouldBeStableDuringLongTest(
150169
// Step 3
151170
// Stop another one of the nodes, check that the chain now stops
152171
// mining blocks
172+
LOG.info("Stopping node 3 to check that the chain stalls");
153173
stopNode(minerNode3);
154174

155175
chainHeight = minerNode1.execute(ethTransactions.blockNumber());
@@ -169,8 +189,8 @@ public void shouldBeStableDuringLongTest(
169189
// Step 4
170190
// Restart both of the stopped nodes. Check that the chain resumes
171191
// mining blocks
192+
LOG.info("Starting node 3 and node 4 to ensure the chain resumes mining new blocks");
172193
startNode(minerNode3);
173-
174194
startNode(minerNode4);
175195

176196
previousStepEndTime = Instant.now();
@@ -202,11 +222,16 @@ public void shouldBeStableDuringLongTest(
202222
lastChainHeight = chainHeight;
203223
}
204224

225+
LOG.info("Updating the value in the smart contract from 101 to 201");
205226
// Update our smart contract before upgrading from berlin to london
206227
assertThat(simpleStorageContract.get().send()).isEqualTo(BigInteger.valueOf(101));
207228
simpleStorageContract.set(BigInteger.valueOf(201)).send();
208229
assertThat(simpleStorageContract.get().send()).isEqualTo(BigInteger.valueOf(201));
209230

231+
LOG.info(
232+
"Upgrading the entire chain to the London fork one node at a time. The genesis for each node will be updated to londonBlock = "
233+
+ lastChainHeight.intValue()
234+
+ 120);
210235
// Upgrade the chain from berlin to london in 120 blocks time
211236
int londonBlockNumber = lastChainHeight.intValue() + 120;
212237
upgradeToLondon(minerNode1, minerNode2, minerNode3, minerNode4, londonBlockNumber);
@@ -220,6 +245,9 @@ public void shouldBeStableDuringLongTest(
220245
previousStepEndTime.plus(getTestDurationMins() / NUM_STEPS, ChronoUnit.MINUTES);
221246
lastChainHeight = chainHeight;
222247

248+
// Allow the chain to restart mining blocks
249+
Thread.sleep(THREE_MINUTES);
250+
223251
while (System.currentTimeMillis() < nextStepEndTime.toEpochMilli()) {
224252
Thread.sleep(ONE_MINUTE);
225253
chainHeight = minerNode1.execute(ethTransactions.blockNumber());
@@ -230,6 +258,8 @@ public void shouldBeStableDuringLongTest(
230258
lastChainHeight = chainHeight;
231259
}
232260

261+
LOG.info(
262+
"Chain has successfully upgraded to the London fork. Checking the contract state is correct");
233263
// Check that the state of our smart contract is still correct
234264
assertThat(simpleStorageContract.get().send()).isEqualTo(BigInteger.valueOf(201));
235265

@@ -253,13 +283,32 @@ public void shouldBeStableDuringLongTest(
253283
Thread.sleep(TEN_SECONDS);
254284
}
255285

286+
LOG.info(
287+
"Deploying a smart contract that should only work if the chain is running on the shanghai fork");
256288
SimpleStorageShanghai simpleStorageContractShanghai =
257289
minerNode1.execute(contractTransactions.createSmartContract(SimpleStorageShanghai.class));
258290

259291
// Check the contract address is as expected for this sender & nonce
260292
contractVerifier
261-
.validTransactionReceipt("0x05d91b9031a655d08e654177336d08543ac4b711")
293+
.validTransactionReceipt("0xfeae27388a65ee984f452f86effed42aabd438fd")
262294
.verify(simpleStorageContractShanghai);
295+
296+
// Archive node test. Check the state of the contract when it was first updated in the test
297+
LOG.info(
298+
"Checking that the archive node shows us the original smart contract value if we set a historic block number");
299+
simpleStorageArchive.setDefaultBlockParameter(
300+
DefaultBlockParameter.valueOf(archiveChainHeight));
301+
assertThat(simpleStorageArchive.get().send()).isEqualTo(BigInteger.valueOf(101));
302+
303+
try {
304+
simpleStorageContract.setDefaultBlockParameter(
305+
DefaultBlockParameter.valueOf(archiveChainHeight));
306+
// Should throw ContractCallException because a non-archive not can't satisfy this request
307+
simpleStorageContract.get().send();
308+
Assertions.fail("Request for historic state from non-archive node should have failed");
309+
} catch (ContractCallException e) {
310+
// Ignore
311+
}
263312
}
264313

265314
private static void updateGenesisConfigToLondon(
@@ -295,21 +344,26 @@ private void upgradeToLondon(
295344
final int londonBlockNumber)
296345
throws InterruptedException {
297346
// Node 1
347+
348+
LOG.info("Upgrading node 1 to london fork");
298349
stopNode(minerNode1);
299350
updateGenesisConfigToLondon(minerNode1, true, londonBlockNumber);
300351
startNode(minerNode1);
301352

302353
// Node 2
354+
LOG.info("Upgrading node 2 to london fork");
303355
stopNode(minerNode2);
304356
updateGenesisConfigToLondon(minerNode2, true, londonBlockNumber);
305357
startNode(minerNode2);
306358

307359
// Node 3
360+
LOG.info("Upgrading node 3 to london fork");
308361
stopNode(minerNode3);
309362
updateGenesisConfigToLondon(minerNode3, true, londonBlockNumber);
310363
startNode(minerNode3);
311364

312365
// Node 4
366+
LOG.info("Upgrading node 4 to london fork");
313367
stopNode(minerNode4);
314368
updateGenesisConfigToLondon(minerNode4, true, londonBlockNumber);
315369
startNode(minerNode4);
@@ -323,21 +377,25 @@ private void upgradeToShanghai(
323377
final long shanghaiTime)
324378
throws InterruptedException {
325379
// Node 1
380+
LOG.info("Upgrading node 1 to shanghai fork");
326381
stopNode(minerNode1);
327382
updateGenesisConfigToShanghai(minerNode1, shanghaiTime);
328383
startNode(minerNode1);
329384

330385
// Node 2
386+
LOG.info("Upgrading node 2 to shanghai fork");
331387
stopNode(minerNode2);
332388
updateGenesisConfigToShanghai(minerNode2, shanghaiTime);
333389
startNode(minerNode2);
334390

335391
// Node 3
392+
LOG.info("Upgrading node 3 to shanghai fork");
336393
stopNode(minerNode3);
337394
updateGenesisConfigToShanghai(minerNode3, shanghaiTime);
338395
startNode(minerNode3);
339396

340397
// Node 4
398+
LOG.info("Upgrading node 4 to shanghai fork");
341399
stopNode(minerNode4);
342400
updateGenesisConfigToShanghai(minerNode4, shanghaiTime);
343401
startNode(minerNode4);

‎app/src/main/java/org/hyperledger/besu/cli/subcommands/storage/RevertMetadataSubCommand.java‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,9 @@ public void run() {
118118
switch (dataStorageFormat) {
119119
case FOREST -> 1;
120120
case BONSAI -> 2;
121+
case X_BONSAI_ARCHIVE ->
122+
throw new IllegalArgumentException(
123+
"X_BONSAI_ARCHIVE storage format cannot be reverted to v1");
121124
};
122125

123126
@JsonSerialize

‎app/src/main/java/org/hyperledger/besu/cli/subcommands/storage/TrieLogSubCommand.java‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@
3030
import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
3131
import org.hyperledger.besu.ethereum.worldstate.ImmutableDataStorageConfiguration;
3232
import org.hyperledger.besu.ethereum.worldstate.ImmutablePathBasedExtraStorageConfiguration;
33-
import org.hyperledger.besu.plugin.services.storage.DataStorageFormat;
3433

3534
import java.io.IOException;
3635
import java.io.PrintWriter;
@@ -328,8 +327,8 @@ private static TrieLogContext getTrieLogContext() {
328327
BesuController besuController = createBesuController();
329328
final DataStorageConfiguration config = besuController.getDataStorageConfiguration();
330329
checkArgument(
331-
DataStorageFormat.BONSAI.equals(config.getDataStorageFormat()),
332-
"Subcommand only works with data-storage-format=BONSAI");
330+
config.getDataStorageFormat().isBonsaiFormat(),
331+
"Subcommand only works with data-storage-format=BONSAI or X_BONSAI_ARCHIVE");
333332

334333
final StorageProvider storageProvider = besuController.getStorageProvider();
335334
final BonsaiWorldStateKeyValueStorage rootWorldStateStorage =

‎app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,9 +85,12 @@
8585
import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
8686
import org.hyperledger.besu.ethereum.transaction.TransactionSimulator;
8787
import org.hyperledger.besu.ethereum.trie.forest.ForestWorldStateArchive;
88+
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.BonsaiArchiveWorldStateProvider;
8889
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.BonsaiWorldStateProvider;
8990
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.cache.BonsaiCachedMerkleTrieLoader;
9091
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.storage.BonsaiWorldStateKeyValueStorage;
92+
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.worldview.BonsaiArchiver;
93+
import org.hyperledger.besu.ethereum.trie.pathbased.common.storage.PathBasedWorldStateKeyValueStorage;
9194
import org.hyperledger.besu.ethereum.trie.pathbased.common.trielog.TrieLogManager;
9295
import org.hyperledger.besu.ethereum.trie.pathbased.common.trielog.TrieLogPruner;
9396
import org.hyperledger.besu.ethereum.worldstate.DataStorageConfiguration;
@@ -803,6 +806,19 @@ public BesuController build() {
803806
}
804807
}
805808

809+
if (DataStorageFormat.X_BONSAI_ARCHIVE.equals(
810+
dataStorageConfiguration.getDataStorageFormat())) {
811+
final BonsaiWorldStateKeyValueStorage worldStateKeyValueStorage =
812+
worldStateStorageCoordinator.getStrategy(BonsaiWorldStateKeyValueStorage.class);
813+
final BonsaiArchiver archiver =
814+
createBonsaiArchiver(
815+
worldStateKeyValueStorage,
816+
blockchain,
817+
scheduler,
818+
((BonsaiWorldStateProvider) worldStateArchive).getTrieLogManager());
819+
blockchain.observeBlockAdded(archiver);
820+
}
821+
806822
final List<Closeable> closeables = new ArrayList<>();
807823
closeables.add(protocolContext.getWorldStateArchive());
808824
closeables.add(storageProvider);
@@ -866,6 +882,24 @@ private TrieLogPruner createTrieLogPruner(
866882
return trieLogPruner;
867883
}
868884

885+
private BonsaiArchiver createBonsaiArchiver(
886+
final WorldStateKeyValueStorage worldStateStorage,
887+
final Blockchain blockchain,
888+
final EthScheduler scheduler,
889+
final TrieLogManager trieLogManager) {
890+
final BonsaiArchiver archiver =
891+
new BonsaiArchiver(
892+
(PathBasedWorldStateKeyValueStorage) worldStateStorage,
893+
blockchain,
894+
scheduler::executeServiceTask,
895+
trieLogManager,
896+
metricsSystem);
897+
898+
archiver.initialize();
899+
LOG.info("Bonsai archiver initialised");
900+
return archiver;
901+
}
902+
869903
/**
870904
* Create synchronizer synchronizer.
871905
*
@@ -1155,6 +1189,22 @@ yield new BonsaiWorldStateProvider(
11551189
evmConfiguration,
11561190
worldStateHealerSupplier);
11571191
}
1192+
case X_BONSAI_ARCHIVE -> {
1193+
final BonsaiWorldStateKeyValueStorage worldStateKeyValueStorage =
1194+
worldStateStorageCoordinator.getStrategy(BonsaiWorldStateKeyValueStorage.class);
1195+
1196+
yield new BonsaiArchiveWorldStateProvider(
1197+
worldStateKeyValueStorage,
1198+
blockchain,
1199+
Optional.of(
1200+
dataStorageConfiguration
1201+
.getPathBasedExtraStorageConfiguration()
1202+
.getMaxLayersToLoad()),
1203+
bonsaiCachedMerkleTrieLoader,
1204+
besuComponent.map(BesuComponent::getBesuPluginContext).orElse(null),
1205+
evmConfiguration,
1206+
worldStateHealerSupplier);
1207+
}
11581208
case FOREST -> {
11591209
final WorldStatePreimageStorage preimageStorage =
11601210
storageProvider.createWorldStatePreimageStorage();

0 commit comments

Comments
 (0)