Store trielog as blobdb

[usmansaleem]

PR description

Store trielog as blobdb to improve performance. #5475 has shown that blobdb is better choice for static data as it reduced the rocksdb write amplifications during compaction. Manually tested upgrade and downgrade.

Graph showing bytes written during compaction. The yellow bar represents the node with "trielog as blobdb" PR applied (blobdb-2), the green bar represents the node without the blobdb patch (blobdb-3). On average, bytes written during compaction is lower for "trielog as blobdb vs trielog w/o blobdb":

Following graph shows fork choice update time (24 hours). There doesn't seem to be any negative impact of writing trielog as blobdb.

Fixed Issue(s)

See #5866

[github-actions]

• I thought about documentation and added the doc-change-required label to this PR if updates are required.
• I thought about the changelog and included a changelog update if required.
• If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

[siladu]

If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

How was this part tested?

[siladu]

@usmansaleem I'm not really sure what the graph in the description is showing, can you add some explanation please?

Also, have a feeling we discussed it last year, but can't remember what the outcome of the following test was?

Verify if pruning has an impact on performance due to compaction #5866

[siladu]

@usmansaleem I'm not really sure what the graph in the description is showing, can you add some explanation please?

ah, just spotted #5866 (comment)

Have you got a graph for the trie log pruning enabled?

---

Last one...

Should we have a feature flag for this or should it be enabled by default?

I guess the answer to this was enabled by default because it's possible to downgrade to an older version of the db?

[jframe]

Store trielog as blobdb to improve performance. #5475 has shown that blobdb is better choice for static data as it reduced the rocksdb write amplifications during compaction. Manually tested upgrade and downgrade.

Do you have a graph on the impact on the engine API FCU and new payload?

[fab-10]

Since after #6445 there can be deletions to this column family, there is the need to enable, and tune, blob garbage collection for it, since now it is it disabled by default (blockchain data cf is meant to be append only with no deletions*)

• there is an experimental feature for pruning the blockchain data as well, so enabling blob garbage collections is useful also there

[fab-10]

LGTM.
It maybe be required to tune the following two options if the trielog purge is enabled and the space is not reclaimed quicky enough (cc @siladu )

• blob_garbage_collection_age_cutoff: the cutoff that the GC logic uses to determine which blob files should be considered “old.” For example, the default value of 0.25 signals to RocksDB that blobs residing in the oldest 25% of blob files should be relocated by GC. This parameter can be tuned to adjust the trade-off between write amplification and space amplification.

• blob_garbage_collection_force_threshold: if the ratio of garbage in the oldest blob files exceeds this threshold, targeted compactions are scheduled in order to force garbage collecting the blob files in question, assuming they are all eligible based on the value of blob_garbage_collection_age_cutoff above. This can help reduce space amplification in the case of skewed workloads where the affected files would not otherwise be picked up for compaction. This option is currently only supported with leveled compactions.