Skip to content

CI - zombienet cumulus tests zombienet sdk #8954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 311 commits into
base: master
Choose a base branch
from
Open

CI - zombienet cumulus tests zombienet sdk #8954

wants to merge 311 commits into from
+2,401 −1,509

Conversation

lrubasze
Copy link
Contributor

@lrubasze lrubasze commented Jun 23, 2025 •
edited
Loading

This PR includes the following changes:

  • Migrates Zombienet Cumulus tests to zombienet-sdk
  • Re-enables the tests, with the following exceptions (to be addressed separately):
    • zombienet-cumulus-0009-elastic_scaling_pov_recovery – #8999
    • zombienet-cumulus-0010-elastic_scaling_multiple_block_per_slot – #9018
  • Adds the following tests to CI:
    • zombienet-cumulus-0011-dht-bootnodes
    • zombienet-cumulus-0012-parachain_extrinsic_gets_finalized
    • zombienet-cumulus-0013-elastic_scaling_slot_based_rp_offset

lrubasze added 30 commits May 13, 2025 13:51
@lrubasze
switch to wait_log_line_count_with_timeout_v2
7672810
@lrubasze
full-node-catching-up-test: remove zombienet test files
4682c58
@lrubasze
full-node-catching-up-test: switch to new wait_log_line_count_with_ti…
9beb6bf 
…meout
@lrubasze
pov-recovery-test: switch to new wait_log_line_count_with_timeout
db64784
@lrubasze
rpc-collator-build-blocks-test: initial version
16b45bd
@lrubasze
rpc-collator-build-blocks-test: remove zombienet test files
2706151
@lrubasze
align to zombienet-sdk changes
06cfc3a
@lrubasze
move bootnodes test
83ac44e
@lrubasze
Update zombienet cumulus CI
38ef236
@lrubasze
update Cargo.lock
4c26913
@lrubasze
elastic-scaling-pov-recovery-test: migrated from zombienet
9b5d417
@lrubasze
elastic-scaling-pov-recovery-test: remove zombienet test files
b7fb5b2
@lrubasze
elastic-scaling-pov-recovery-test: update CI
b0ee722
@lrubasze
pov-recovery-test: update
2f99477
@lrubasze
fmt
ef5f645
@lrubasze
add utils module
bff56f1
@lrubasze
Merge branch 'master' into migrate-more-tests-to-zombienet-sdk
5492943
@lrubasze
add missing licence
f3b1fa2
@lrubasze
Merge branch 'master' into migrate-more-tests-to-zombienet-sdk
69ee421
@lrubasze
cleanup
52b7d55
@lrubasze
do not retry 'zombienet-sdk' tests in CI
b32d9af
@lrubasze
fix 'zombienet-sdk' CI
a3f6ce4
@lrubasze
clippy
ca9aaf0
@lrubasze
tweak test params to check parachain registration
c007ab6
@lrubasze
tweak test params to check if node is up
ff395f0
@lrubasze
runtime-upgrade-test: provide WASM binary for CI test
1757fc1
@lrubasze
tweak elastic_scaling multiple blocks per slot params
8e48095
@lrubasze
tweak test params for CI
06d81c8
@lrubasze
zombienet-sdk from git to allow insecure url RPC connections
44a56d0
@lrubasze
add dht bootnodes test to CI
d868888
skunert
skunert approved these changes Jul 4, 2025
View reviewed changes
Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left some last nits.

pepoviola
pepoviola reviewed Jul 4, 2025
View reviewed changes
@lrubasze lrubasze mentioned this pull request Jul 7, 2025
Open
@lrubasze
Copy link
Contributor Author

lrubasze commented Jul 7, 2025 •
edited
Loading

🧪 Test Results from Weekend Run – 2025-07-04 - 2025-07-07

  • Failed / Total runs: 2 / 266 (Failure rate: 0.75%)
  • Skipped (flaky) / Total flaky tests: 2 / 13

❌ Failures

testy

bkchr and others added 2 commits July 7, 2025 14:34
@bkchr
Replace assert_para_throughput with assert_finalized_para_throughput
e08d8ea 
There is no need to have two functions which are essentially doing the same. It is also better to oberserve the finalized blocks, which also simplifies the code. So, this pull request is replacing the
`assert_para_throughput` with `assert_finalized_para_throughput`. It also replaces any usage of `assert_finalized_para_throughput` with `assert_para_throughput`.
@github-actions
Update from github-actions[bot] running command 'fmt'
a6c9fd8
@lrubasze lrubasze mentioned this pull request Jul 7, 2025
Merged
lrubasze added a commit to paritytech/zombienet-sdk that referenced this pull request Jul 7, 2025
@lrubasze
Add wait until is up methods (#372)
33b8c44 
This PR adds `wait_until_is_up()` methods as suggested
[here](paritytech/polkadot-sdk#8954 (comment)).

Additionally it fixes `test_wait_log_count_with_lookahead_regex_fails`
test.
@pepoviola pepoviola self-requested a review July 7, 2025 13:57
@lrubasze lrubasze mentioned this pull request Jul 7, 2025
Closed
@skunert skunert mentioned this pull request Jul 8, 2025
Merged
@lrubasze
Copy link
Contributor Author

lrubasze commented Jul 9, 2025

🧪 Test Results from Nigtly Run – 2025-07-08

  • Failed / Total runs: 8 / 54 (Failure rate: 14.8%)
  • Skipped (flaky) / Total flaky tests: 1 / 13

❌ Failures

Failure rate increased significantly.
Potential root cause: merging changes from #9117
To be investigated.

newplot(2)

@pepoviola
Copy link
Contributor

Maybe we need to review the ranges after merging #9117 . Can you send me the plot (html).
Thx!!

@lrubasze
Copy link
Contributor Author

lrubasze commented Jul 9, 2025 •
edited
Loading

Maybe we need to review the ranges after merging #9117 . Can you send me the plot (html). Thx!!

Here is the plot: chart_ci-zombienet-cumulus-tests-zombienet-sdk.html.zip

Failures:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pepoviola pepoviola pepoviola left review comments

@skunert skunert skunert approved these changes

@alvicsam alvicsam alvicsam approved these changes

Assignees
No one assigned
Labels
R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests. T18-zombienet_tests Trigger zombienet CI tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@lrubasze @pepoviola @skunert @alvicsam @lazam @iulianbarbu @bkchr