pabot ordering file does not work with multiple stages

[MarvKler]

Please see the following question which i have raise in slack - here you can also find the complete conversation with the reason for creating this issue.

Main reason here, as you can read below, i would like to execute test suite B completely independent from all other suite after test suite A has been finished. At the moment test suite G & H are unfortunately waiting until test suite B has been finished. This handling should be fixed / changed imo.

Hi! I have observed a strange behaviour with my current ordering file you can find below. I am using pabot version 4.3.2.
I would expect that TestSuiteB is completely running in parallel to all other suites after TestSuiteA has been finished - TestSuiteA is kind of a pre-condition for all of my suites.
Actually, TestSuiteG and TestSuiteH are starting only when TestSuiteB has been finished, but i have configured a different behaviour right ? Is there any simple issue in my configuration or could it be an issue by pabot itself ?
I am running pabot in the cli with config parameter --processes 6 .

--suite TestSuiteA

--suite TestSuiteB #DEPENDS TestSuiteA

--suite TestSuiteC #DEPENDS TestSuiteA
--suite TestSuiteD #DEPENDS TestSuiteA
--suite TestSuiteE #DEPENDS TestSuiteA
--suite TestSuiteF #DEPENDS TestSuiteA

--suite TestSuiteG #DEPENDS TestSuiteC #DEPENDS TestSuiteD #DEPENDS TestSuiteE #DEPENDS TestSuiteF
--suite TestSuiteH #DEPENDS TestSuiteC #DEPENDS TestSuiteD #DEPENDS TestSuiteE #DEPENDS TestSuiteF

[joonaskuisma]

As I mentioned in the Slack discussion, this would indeed be a useful improvement, but due to the current implementation it’s quite a heavy task to implement. Still, it’s something that should probably be taken up in the future.

At the same time, it could also make sense to enable optional skipping of suites or tests if something goes wrong earlier in their dependency chain. For example, if something fails in TestSuiteA, it could be handy to stop execution there, since the system may already be in a state where dependent suites B–F cannot succeed anyway.

Part of my original Slack reply:

The root cause is that Pabot’s --ordering file execution order is currently (and also in version 4.3.2) static, not dynamic. The order is determined before the run actually starts.

For this to work, it must be ensured that the previous so-called Stage is fully completed. In Pabot’s current implementation, there is effectively a #WAIT between stages. This is probably because #DEPENDS is a newer feature that has been implemented on top of #WAIT.

So, your example looks like this:

Stage 0:
A
Stage 1:
B
C
D
E
F
Stage 2:
G
H

And that is why G and H also have to wait for test B. However, the ordering is still correct, even if the execution time isn’t always optimal. If test B takes longer to execute than C, D, E, or F, this can be suboptimal. But if not, then it doesn’t matter.