Unequal partitioning through partitioning file

[suratdas]

In the image below, we wanted Windows, Mac and Linux cases to run in parallel as defined in the partitioning file. This could not be done with default partitioning which works by splitting the test cases equally. This pull request implements suite based unequal split.

Partitioning file

Page	Partition	Test System	Order
Demo.DemoSuite.Linux	0	slim	0
Demo.DemoSuite.Mac	1	slim	0
Demo.DemoSuite.Windows	2	slim	0

[fhoeben]

I completely understand splitting equally is not always what one wants (in my own project I create a partitioner that does something similar, and generates the split based on previous run times). I'll look at your changes this weekend.

As a side note: for your specific case of running 3 suites in parallel wouldn't it be easier just to start one run per suite? Then there is no need to work based on a partitioning file.

[fhoeben]

Changes look pretty minimal and logical. Could you also add some (unit) tests?

[suratdas]

I presented the case in a simplified way but we sometimes have more than one suite in a partition. We change and decide how many suites to go in which partition depending on priority/test environment/execution time etc.. It was easier to manage via partition file rather than modifying Jenkins parameters for run per suite.

As a side note: for your specific case of running 3 suites in parallel wouldn't it be easier just to start one run per suite? Then there is no need to work based on a partitioning file.

[fhoeben]

Do I read your change correctly that the existing code already allowed for uneven split, your change just makes for simpler configuration by not requiring each page to be mentioned. Instead one can configure suites to be placed in a partition.

Configuration per test page is checked first, then any configuration up the tree is checked and only then does the equal partitioning kick in. Correct?

Seems perfectly fine, but it does seem to warrant some test cases. Will you be adding those?

[suratdas]

Yes, you are absolutely correct.
Right now, I am going through the tests to understand and then will add unit tests.

[suratdas]

Unit tests added and documentation updated.

[fhoeben]

Looks quite good. Almost there.