Skip to content

Improve Conflict Detection in Parallelization by Considering Slots to Reduce False Positives#7923

Merged
matkt merged 15 commits intobesu-eth:mainfrom
matkt:feature/para-evm-2
Feb 17, 2025
Merged

Improve Conflict Detection in Parallelization by Considering Slots to Reduce False Positives#7923
matkt merged 15 commits intobesu-eth:mainfrom
matkt:feature/para-evm-2

Conversation

@matkt
Copy link
Copy Markdown
Contributor

@matkt matkt commented Nov 26, 2024

PR description

This update improve conflict detection during transaction parallelization by checking storage slot changes. By considering modified slots in addition to addresses, we improve the accuracy of conflict detection.

Key Changes:

  • Added storage slot consideration in conflict detection.
  • Reduced false positives by more accurately identifying conflicts between transactions and block state.

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

  • unit tests: ./gradlew build
  • acceptance tests: ./gradlew acceptanceTest
  • integration tests: ./gradlew integrationTest
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
@ahamlat
Copy link
Copy Markdown
Contributor

ahamlat commented Nov 26, 2024

This new conflict detection improved the number of parallelized transactions from ~40% to ~60%.
The numbers in the charts below are TPS.

Before this PR (24.10.0 running with --Xbonsai-parallel-tx-processing-enabled=true)
image

With this PR (running with --Xbonsai-parallel-tx-processing-enabled=true)
image

With more transactions executed in parallel, we noticed around 20% performance improvement compared to the last parallelization implementation. The numbers below are from instances running both EL and CL and are not using high spec flag.

image

We can notice also almost twice better performance when comparing to besu 24.10.0 with Teku as a CL

image

I don't see any overhead on the resource usage level.

@matkt
Copy link
Copy Markdown
Contributor Author

matkt commented Nov 26, 2024

we need to do stability tests before merging

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
- copy storageKeyHashLookup when cloning accumulator

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Copy link
Copy Markdown
Contributor

@macfarla macfarla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changelog entry?

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
@matkt matkt force-pushed the feature/para-evm-2 branch from 68a5c15 to 85bfd20 Compare December 4, 2024 16:31
@matkt matkt force-pushed the feature/para-evm-2 branch from 788f628 to 9985e73 Compare January 7, 2025 19:37
matkt added 3 commits January 22, 2025 14:45
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Copy link
Copy Markdown
Contributor

@ahamlat ahamlat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, comparing with the first version of the parallelization, this PR has better performances even on 4 Cores, without more overhead. Conflict detection takes even less time than the first version.

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
@matkt matkt enabled auto-merge (squash) February 17, 2025 11:55
@matkt matkt disabled auto-merge February 17, 2025 11:56
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
@matkt matkt enabled auto-merge (squash) February 17, 2025 11:57
@matkt
Copy link
Copy Markdown
Contributor Author

matkt commented Feb 17, 2025

changelog entry?

done

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
@matkt matkt force-pushed the feature/para-evm-2 branch from 0a23a9b to 552bc52 Compare February 17, 2025 13:14
@matkt matkt merged commit 41a0647 into besu-eth:main Feb 17, 2025
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants