Non-reproducibility in `Ecal/Trends/TriggerTowerTask Et sum of TPs above threshold` histogram in workflows 9.0 and 25.0

[makortel]

#48162 (comment) shows a comparison difference in workflows 9.0 and 25.0 in Ecal/Trends/TriggerTowerTask Et sum of TPs above threshold histogram. The comparison looks like

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @makortel.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

Also shown in #48493 (comment)

[makortel]

Both PRs had CMSSW_15_1_X_2025-07-07-1100 as the baseline

[makortel]

Also in

• Add SimpleTrackValidation binned in eta and pt #48472 (comment) (CMSSW_15_1_X_2025-07-07-1100)
• TICL Objects in HLT NanoAOD #48500 (comment) (CMSSW_15_1_X_2025-07-08-1100)
• TICL-barrel: run CLUE in the barrel calorimeters and first workflows #47859 (comment) (CMSSW_15_1_X_2025-07-02-1100)
• [SIM] Clean up custom physics sub-library #48488 (comment) (CMSSW_15_1_X_2025-07-07-1100)
• [Phase-2 L1T] fixing out of bound indices in Phase2L1CaloPFClusterEmulator #48478 (comment) (CMSSW_15_1_X_2025-07-03-1100) also in wf 145.014
• HLT Phase2 Alpaka menu in relval_gpu #48463 (comment) (CMSSW_15_1_X_2025-07-02-2300)
• Run3-Sim164X Separate HcalTestNumbering flag for signal and PU which enables signal from FullSim and PU from FastSim #48448 (comment) (CMSSW_15_1_FASTPU_X_2025-07-02-2300)
• Moving 2024 Data Wfs HLT Key to hltKey2024 #48479 (comment) (CMSSW_15_1_X_2025-07-03-2300)
• Improved Event Content for the TkAlHLT* ALCARECO  #48474 (comment) (CMSSW_15_1_X_2025-07-03-1100)
• update a vertex smearing parameters for 2025 pO MC #48480 (comment) (CMSSW_15_1_X_2025-07-04-2300)
• Change fromVertex parameter from False to True for DGL reconstruction #48475 (comment) (CMSSW_15_1_X_2025-07-03-2300)
• Increase Nano relval WF counter buffer size #48459 (comment) (CMSSW_15_1_X_2025-07-02-1100)
• ScoutingCollectionMonitor add muon "Vtx" collections #48453 (comment) (CMSSW_15_1_X_2025-07-01-2300)
• MTD DetId: add static methods to MTDDetId to test whether it is BTL or ETL #48466 (comment) (CMSSW_15_1_X_2025-07-02-2300)
• Fix for n_layers Check in CAHitNtupletAlpaka::globalBeginRun #48477 (comment) (CMSSW_15_1_X_2025-07-03-1100)
• MTD Validation: update vertex multiplicity plots #48464 (comment) (CMSSW_15_1_X_2025-07-02-1100)
• Change of HF threshold of OO era #48483 (comment) (CMSSW_15_1_X_2025-07-04-1100)
• Update getCudaCoresPerSM for Blackwell GPUs #48467 (comment) (CMSSW_15_1_X_2025-07-02-2300)
• remove warnings about UnusedProductsForCanDeleteEarly in the phase-2 HLT menu #48470 (comment) (CMSSW_15_1_X_2025-07-02-2300)
• [NGT] Fix filling of ValueMap for DeepJet scores for NanoAOD Phase2 HLT #48462 (comment) (CMSSW_15_1_X_2025-07-02-2300)
• [TrackingTools/KalmanUpdators] Add using declaration to prevent funct… #48461 (comment) (CMSSW_15_1_X_2025-07-02-1100)
• Avoid FSC Channels when Making HCAL LUT XML Payloads #48446 (comment) (CMSSW_15_1_X_2025-06-30-1100)
• Introduce a stub of LHCInfoPer* Payload Inspector #48451 (comment) (CMSSW_15_1_X_2025-06-30-1100)
• [HGCAL] Update HGCal unpacker to handle flexible fedId assignment #48449 (comment) (CMSSW_15_1_X_2025-06-30-1100)
• apply stage2 L1T customizations to TrackToTrackComparisonHists #48440 (comment) (CMSSW_15_1_X_2025-06-29-2300)
• fix the loop range in mkFit MkFinder hit checks #48454 (comment) (CMSSW_15_1_X_2025-06-30-2300)
• Add protection to skip addition FSC channels. #48444 (comment) (CMSSW_15_1_X_2025-06-30-1100)

Not seen in PR tests done on CMSSW_15_1_X_2025-06-29-0000 or CMSSW_15_1_X_2025-06-27-1100

[makortel]

FYI @cms-sw/ecal-dpg-l2 @cms-sw/orp-l2

(I don't really know to what area to assign this)