Lots of unit tests need networking

[jakobkukla]

Hi,

I'm currently trying to package Yamtrack for Nix/NixOS. Nix builds packages in a sandbox without networking (for reproducibility). When building in this sandbox 197 (more than half) unit tests fail with HTTP connection errors. Enabling networking inside the sandbox results in all tests passing. I understand that Yamtrack tests its integration with metadata providers, etc... and thus needs to connect to their APIs. But it seems like a lot of the failing tests are not really related to (or specifically testing) these APIs. I would normally just disable tests that need access to thirdparty APIs but in this case we would lose a lot of test coverage for unrelated stuff.

Just wanted to ask if this is a known issue? If this is something that isn't going to change in the near future, I would propose adding pytest markers which would make it easier to filter out tests that rely on internet access. I would also be willing to contribute such a change.

Love the project btw, thanks :)

[FuzzyGrim]

Hi,

Currently, almost all tests require a network connection. The main way I set up my tests is by testing the integration with the API provider and then caching that data for use by the other tests.

[66Bunz]

It could be possible to provide mock provider data and use those in the tests if specified...?
Something like:
coverage run src/manage.py test app users integrations lists events --parallel --mode=offline
Then the manage.py file can load the variable that is then passed to the test settings, and the tests will need to be modified to search for the mode value so they can use provider mock data or request it online

def main():
    """Run administrative tasks."""
    if "test" in sys.argv:
        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.test_settings")
        for arg in sys.argv:
            if arg.startswith("--mode="):
                _, val = arg.split("=", 1)
                os.environ["TEST_MODE"] = val
    else:
        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")

    execute_from_command_line(sys.argv)

Something like this