Skip to content

Replace BytesTrieSet with HashSet#9641

Merged
siladu merged 4 commits intobesu-eth:mainfrom
siladu:replace-BytesTrieSet
Jan 27, 2026
Merged

Replace BytesTrieSet with HashSet#9641
siladu merged 4 commits intobesu-eth:mainfrom
siladu:replace-BytesTrieSet

Conversation

@siladu
Copy link
Copy Markdown
Contributor

@siladu siladu commented Jan 15, 2026

Related to #9165
(Replaces #9166)

This PR effectively reverts #6069 by wholesale removing BytesTrieSet and going back to using HashSet as was the case with this PR: #5537

Benchmarks show improvements across CREATE, CREATE2, SELFDESTRUCT and jumpdest analysis and no regressions including *CALL.


No mainnet improvement/regression detected, newPayload response time p50:

replace-BytesTrieSet-mainnet

Performed benchmarking using evm block-test and the eest benchmark compute suite as of ethereum/execution-specs@2c83b84

Ran https://github.com/ethereum/execution-specs/tree/forks/amsterdam/tests/benchmark/compute/instruction
and https://github.com/ethereum/execution-specs/tree/forks/amsterdam/tests/benchmark/compute/scenario
ignoring the precompile suite.

Instruction Benchmark: Control vs replace-BytesTrieSet

  Summary
  ┌────────────────────────────┬────────────────────────────┬────────────────────────────┬────────────────────┐
  │         Comparison         │          Control           │            Test            │     Difference     │
  ├────────────────────────────┼────────────────────────────┼────────────────────────────┼────────────────────┤
  │ All runs (7 ctrl, 3 test)  │ 463,541 ms @ 165.69 MGas/s │ 454,614 ms @ 168.77 MGas/s │ -8,927 ms (-1.93%) │
  ├────────────────────────────┼────────────────────────────┼────────────────────────────┼────────────────────┤
  │ repeat2 only (2 vs 3 runs) │ 461,082 ms @ 166.11 MGas/s │ 454,614 ms @ 168.77 MGas/s │ -6,468 ms (-1.40%) │
  └────────────────────────────┴────────────────────────────┴────────────────────────────┴────────────────────┘
  Top 10 Improvements
  ┌──────┬──────────────────┬─────────────────────────────────────────────────┐
  │ Rank │      Change      │                      Test                       │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 1    │ -47.7% (-219 ms) │ test_create[0 bytes with value-CREATE2]         │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 2    │ -46.3% (-206 ms) │ test_create[0 bytes without value-CREATE2]      │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 3    │ -45.6% (-200 ms) │ test_create[0 bytes with value-CREATE]          │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 4    │ -44.4% (-283 ms) │ test_selfdestruct_initcode[value_bearing_True]  │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 5    │ -44.4% (-191 ms) │ test_create[0 bytes without value-CREATE]       │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 6    │ -44.3% (-288 ms) │ test_selfdestruct_initcode[value_bearing_False] │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 7    │ -38.8% (-2 ms)   │ test_create[max code size-CREATE2]              │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 8    │ -35.8% (-420 ms) │ test_selfdestruct_created[value_bearing_True]   │
  └──────┴──────────────────┴─────────────────────────────────────────────────┘
  Top Regressions (small absolute values)
  ┌──────┬────────────────────┬────────────────────────────────────────────────┐
  │ Rank │       Change       │                      Test                      │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 1    │ +103.2% (+1.54 ms) │ test_log[zeros_topic-1_MiB_zeros_data-log1]    │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 2    │ +99.0% (+1.49 ms)  │ test_log[zeros_topic-1_MiB_non_zero_data-log2] │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 3    │ +84.7% (+1.43 ms)  │ test_log[zeros_topic-1_MiB_zeros_data-log4]    │
  └──────┴────────────────────┴────────────────────────────────────────────────┘
  Note: The regressions are all sub-2ms tests - the high percentages are misleading. The improvements are in the 200-420ms range, far outweighing the tiny regressions.

  Conclusion

  replace-BytesTrieSet is ~1.4-1.9% faster overall, with significant improvements in CREATE, CREATE2, and SELFDESTRUCT operations. The log test "regressions" are noise in sub-2ms tests.

Scenario Benchmark Results: Control vs replace-BytesTrieSet

Methodology: 10 iterations each, using evmtool block-test with execution-focused tests

Summary

Metric Control Test Difference
Total Time 6232.49 ms 5545.00 ms -687.49 ms (-11.0%)
Throughput 364.83 MGas/s 410.60 MGas/s +45.77 MGas/s
Tests 24 24 -
Iterations 10 10 -

Result: Test is 11.0% faster (687 ms saved)

Top Improvements

Test Control Test Change
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 452.48 ms 336.74 ms -25.6%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 469.16 ms 352.35 ms -24.9%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 453.07 ms 346.33 ms -23.6%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 497.59 ms 383.95 ms -22.8%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 477.13 ms 375.30 ms -21.3%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 503.35 ms 404.36 ms -19.7%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 104.38 ms 99.45 ms -4.7%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 354.88 ms 339.69 ms -4.3%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 210.20 ms 202.09 ms -3.9%
test_auth_transaction[benchmark-gas-value_100M-fork_Pragu... 327.28 ms 316.97 ms -3.2%

Potential Regressions

Test Control Test Change
test_block_full_of_ether_transfers[benchmark-gas-value_10... 211.63 ms 222.87 ms +5.3%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 111.42 ms 114.89 ms +3.1%
test_block_full_access_list_and_data[benchmark-gas-value_... 53.18 ms 54.72 ms +2.9%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 113.49 ms 115.54 ms +1.8%
test_block_full_data[benchmark-gas-value_100M-fork_Prague... 51.18 ms 52.02 ms +1.6%

Note: Small regressions (<5%) are within measurement variance (CV typically 5-15%)

All Tests (sorted by improvement)

# Test Control (ms) Test (ms) Diff Ctrl CV Test CV
1 test_jumpdest_analysis[benchmark-gas-value_100M... 452.48 336.74 -25.6% 1.4% 8.7%
2 test_jumpdest_analysis[benchmark-gas-value_100M... 469.16 352.35 -24.9% 0.8% 7.2%
3 test_jumpdest_analysis[benchmark-gas-value_100M... 453.07 346.33 -23.6% 1.6% 7.8%
4 test_jumpdest_analysis[benchmark-gas-value_100M... 497.59 383.95 -22.8% 1.1% 6.2%
5 test_jumpdest_analysis[benchmark-gas-value_100M... 477.13 375.30 -21.3% 1.0% 7.7%
6 test_jumpdest_analysis[benchmark-gas-value_100M... 503.35 404.36 -19.7% 1.0% 6.5%
7 test_block_full_of_ether_transfers[benchmark-ga... 104.38 99.45 -4.7% 16.2% 13.0%
8 test_block_full_of_ether_transfers[benchmark-ga... 354.88 339.69 -4.3% 8.3% 5.1%
9 test_block_full_of_ether_transfers[benchmark-ga... 210.20 202.09 -3.9% 17.5% 18.6%
10 test_auth_transaction[benchmark-gas-value_100M-... 327.28 316.97 -3.2% 3.9% 0.9%
11 test_block_full_of_ether_transfers[benchmark-ga... 111.68 108.33 -3.0% 13.5% 14.3%
12 test_block_full_data[benchmark-gas-value_100M-f... 15.12 14.67 -2.9% 9.7% 6.0%
13 test_block_full_of_ether_transfers[benchmark-ga... 228.92 223.80 -2.2% 14.2% 14.7%
14 test_block_full_of_ether_transfers[benchmark-ga... 338.85 332.40 -1.9% 11.2% 9.2%
15 test_block_full_of_ether_transfers[benchmark-ga... 232.31 228.66 -1.6% 11.8% 11.0%
16 test_empty_block[benchmark-gas-value_100M-fork_... 0.97 0.96 -0.5% 3.3% 3.3%
17 test_auth_transaction[benchmark-gas-value_100M-... 284.67 284.28 -0.1% 3.9% 4.3%
18 test_auth_transaction[benchmark-gas-value_100M-... 294.89 295.53 +0.2% 1.5% 4.4%
19 test_auth_transaction[benchmark-gas-value_100M-... 334.67 339.09 +1.3% 1.9% 2.6%
20 test_block_full_data[benchmark-gas-value_100M-f... 51.18 52.02 +1.6% 3.1% 2.2%
21 test_block_full_of_ether_transfers[benchmark-ga... 113.49 115.54 +1.8% 6.0% 11.3%
22 test_block_full_access_list_and_data[benchmark-... 53.18 54.72 +2.9% 8.0% 11.5%
23 test_block_full_of_ether_transfers[benchmark-ga... 111.42 114.89 +3.1% 7.6% 9.6%
24 test_block_full_of_ether_transfers[benchmark-ga... 211.63 222.87 +5.3% 9.0% 9.2%

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
@siladu siladu marked this pull request as ready for review January 23, 2026 03:58
Copy link
Copy Markdown
Contributor

@lu-pinto lu-pinto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@ahamlat ahamlat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@siladu siladu enabled auto-merge (squash) January 27, 2026 06:02
@siladu siladu merged commit 1a199f1 into besu-eth:main Jan 27, 2026
46 of 61 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Performance Jan 27, 2026
@github-project-automation github-project-automation bot moved this from Open PRs to Done in 26.1.0 Release Jan 27, 2026
@siladu siladu deleted the replace-BytesTrieSet branch January 27, 2026 06:39
macfarla pushed a commit to CPerezz/besu that referenced this pull request Feb 6, 2026
Improves performance of CREATE, CREATE2, SELFDESTRUCT and jumpdest analysis and no visible regressions including *CALL, according to https://github.com/ethereum/execution-specs/tree/c6c380acffa586cfd683384fb2e09348e9a97c03/tests/benchmark/compute benchmark suite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants