Skip to content

Ability to specify granularity of the tests to be distributed #7

Closed
@cfclrk

Description

@cfclrk

As discussed in this post: http://stackoverflow.com/questions/4637036/is-there-a-way-to-control-how-pytest-xdist-runs-tests-in-parallel

When using distmode = 'load', it would nice if xdist provided a mechanism to allow users to specify an alternate test collection algorithm. As the StackOverflow post mentions, splitting up by TestSuite or module would be very useful.

Activity

nicoddemus

nicoddemus commented on Oct 6, 2015

@nicoddemus
Member

Hi @Ludachrispeed, thanks for the report.

This is a long requested feature, being duplicated by pytest-dev/pytest#175 and pytest-dev/pytest#738. I'm closing this as a duplicate then.

wkerzendorf

wkerzendorf commented on Nov 22, 2016

@wkerzendorf

@nicoddemus I've tried to follow this feature request among the multiple issues - and they are seemingly all closed. I want to do something similar. The expensive calculation is all within the setup of a test class and the setup is parameterized. So is there a way to do this - even a hacky one?

RonnyPfannschmidt

RonnyPfannschmidt commented on Nov 22, 2016

@RonnyPfannschmidt
Member

@wkerzendorf currently the xdist internals are not in a shape that makes this easy

collection runs only on the workers, sheduling happens only on the controller, however only the only communicated metadata are test ids# not even fixtures and their scopes/ids

wkerzendorf

wkerzendorf commented on Nov 22, 2016

@wkerzendorf

@RonnyPfannschmidt any other suggestion you have (even outside xdist)? One of our tests likely takes days. Is there a way to tell it not to split up the class, etc? thanks

RonnyPfannschmidt

RonnyPfannschmidt commented on Nov 22, 2016

@RonnyPfannschmidt
Member

@wkerzendorf just have a cli option to run exactly that thest and simply dont run it in a normal test run

if you have singluar end2end tests that take days - run them separate - in general - a general parallel test-runner is not thinking in terms of massive tests vs normal tests

wkerzendorf

wkerzendorf commented on Nov 22, 2016

@wkerzendorf

@RonnyPfannschmidt we do this. So we have a parametrized setup of these long running tests (let's say A, B, C, D). Can I just say - run one parameter set of a parametrized test?
Thanks for helping me out here - I also don't want to spam your issue tracker - is there a chat room for pytest?

nicoddemus

nicoddemus commented on Nov 22, 2016

@nicoddemus
Member

I think @RonnyPfannschmidt means something like this:

  1. Mark your slow/integration tests with a mark, say slow.
  2. Run your regular tests using xdist: pytest -n auto -m "not slow"
  3. Run your slow tests without xdist: pytest -m slow.

It should be simple to change your CI system to run pytest twice instead of only once. You can have even more layers of test sessions if you prefer.

nicoddemus

nicoddemus commented on Nov 22, 2016

@nicoddemus
Member

Btw there's a pytest channel at freenode where people answer questions.

wkerzendorf

wkerzendorf commented on Nov 22, 2016

@wkerzendorf

@nicoddemus - we already do this.

wkerzendorf

wkerzendorf commented on Nov 22, 2016

@wkerzendorf

But are there as helpful people as you and @RonnyPfannschmidt on this freenode channel ;-) I'll go there and give it a try.

nicoddemus

nicoddemus commented on Nov 22, 2016

@nicoddemus
Member

(btw I'm perfectly fine in using the issue tracker for this type of conversation. you can easily share code and the solution can be found by others later)

So that I understand, do you have one test which is parametrized, but is only slow for some of those parameters?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @RonnyPfannschmidt@cfclrk@wkerzendorf@nicoddemus

        Issue actions

          Ability to specify granularity of the tests to be distributed · Issue #7 · pytest-dev/pytest-xdist