Pass test result to finalizer for cleaning up debug data

[pytestbot]

Originally reported by: Torsten Landschoff (BitBucket: bluehorn, GitHub: bluehorn)

---

We are currently juggling with big files in tests which we currently store in tmpdir.
This kills our Jenkins slaves because py.test will keep temporary files for 3 runs of each test suite. We are no up at > 10GB of temporary files for some projects per test run.

Keeping the temporary files is indeed a nice feature which we often use during TDD to find out what we were missing. But the temporary files of a passed test are 99.99% worthless for us.

Therefore I'd like to have a tmpdir fixture that cleans up the temporary files if the test was successful or skipped. Or have some options to configure how py.test handles temporary files wrt. what is kept.

As an intermediate step I am currently moving some pytest.fixture functions to clean up the temporary files after use, but I end up removing the files even if the test failed.

What I am also just realizing: We moved the temporary files inside the Jenkins workspace of each project to keep track of disk usage and for easy access via the web interface. If only temporary files for failed tests are kept, then we could archive them as build artifacts so that we can even diagnose tests that failed in the past.

I was thinking about something in the line of

#!python

@pytest.fixture
def database(request):
    # Pull SQLite test database from somewhere
    dbfile = "..."
    def finalizer(testresult):
        if testresult == "PASS":
            os.unlink(dbfile)

    request.addfinalizer(finalizer, parameters=["testresult"])

Alternatively, pytest could use introspection to inject the testresult if the finalizer takes a testresult argument.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/288

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

just quickly an example that works with today's pytest:

# content of conftest.py

import pytest

@pytest.fixture
def testresult():
    class TestResult:
        pass
    return TestResult()

@pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
    if call.when == "call":
        if "testresult" in item.fixturenames:
            rep = __multicall__.execute()
            item.funcargs["testresult"].rep = rep
            return rep

and the test file:

import pytest

@pytest.fixture
def database(request, tmpdir, testresult):
    p = tmpdir.ensure("somefile")
    def finalizer():
        if testresult.rep.passed:
            p.remove()
    request.addfinalizer(finalizer)
    return p

def test_pass(database):
    assert database.check()

def test_fail(database):
    assert 0

We could think about exposing something like "testresult" in standard pytest.

[pfctdayelise]

I closed #230 as duplicate, maybe someone should make this as a plugin.

[RonnyPfannschmidt]

closing this one as there is a supported way to do this