Skip to content

[Bug]: make integration can't be run repeatedly, integration tests can't be executed concurrently #686

@dkovba

Description

@dkovba

I have done the following

  • I have searched the existing issues
  • If possible, I've reproduced the issue using the 'main' branch of this project

Steps to reproduce

  1. Run make cleancontent.
  2. Run make integration two times.

Alternatively:

  1. Run make cleancontent.
  2. Revert Make test suites run sequentially with respect to other test suites #200 to enable a parallel execution of integration tests.
  3. Run make integration once.

Current behavior

✘ Test testNetworkCreateAndUse() recorded an issue at TestCLINetwork.swift:85:13: Expectation failed: success
↳ Request to http://192.168.65.2:59242 failed after 10 retries
✘ Test testNetworkCreateAndUse() failed after 62.447 seconds with 1 issue.
✘ Suite TestCLINetwork failed after 62.448 seconds with 1 issue.
✘ Test run with 3 tests in 1 suite failed after 62.448 seconds with 1 issue.
✘ Test testForwardTCP() recorded an issue at TestCLIRunOptions.swift:512:13: Expectation failed: success
↳ Request to http://127.0.0.1:50477 failed after 10 retries
✘ Test testForwardTCP() failed after 75.191 seconds with 1 issue.
✘ Suite TestCLIRunCommand failed after 75.191 seconds with 1 issue.
✘ Test run with 20 tests in 1 suite failed after 75.192 seconds with 1 issue.
✘ Test testBuildNetworkAccess() recorded an issue at CLIBuilderTest.swift:168:10: Caught error: executionFailed("build failed: stdout= stderr=\u{1B}[33mWarning!\u{1B}[0m Running debug build. Performance may be degraded.\n#1 [resolver] fetching image...ghcr.io/linuxcontainers/alpine:3.20\n#1 DONE 0.0s\n\n#2 [internal] load .dockerignore\n#2 transferring context: 2B done\n#2 DONE 0.0s\n\n#3 oci-layout://ghcr.io/linuxcontainers/alpine:3.20@sha256:28a33e0390da2b341d05e53803c6766977039baedfc8a82a4910d5acc4755b66\n#3 resolve ghcr.io/linuxcontainers/alpine:3.20@sha256:28a33e0390da2b341d05e53803c6766977039baedfc8a82a4910d5acc4755b66 0.1s done\n#3 sha256:4cbb6abcfc2406820012ef89d697859152a729e7dbde6a4217f99136708f716d 7.55kB / 7.55kB done\n#3 sha256:94e9d8af22013aabf0edcaf42950c88b0a1350c3a9ce076d61b98a535a673dd9 4.09MB / 4.09MB 0.0s done\n#3 extracting sha256:94e9d8af22013aabf0edcaf42950c88b0a1350c3a9ce076d61b98a535a673dd9 0.1s done\n#3 DONE 0.2s\n\n#3 oci-layout://ghcr.io/linuxcontainers/alpine:3.20@sha256:28a33e0390da2b341d05e53803c6766977039baedfc8a82a4910d5acc4755b66\n#3 extracting sha256:4cbb6abcfc2406820012ef89d697859152a729e7dbde6a4217f99136708f716d done\n#3 DONE 0.2s\n\n#4 [linux/arm64 1/2] RUN nc -zv 8.8.8.8 53 || exit 1\n#4 3.135 nc: 8.8.8.8 (8.8.8.8:53): Host is unreachable\n#4 ERROR: process \"/bin/sh -c nc -zv ${ADDRESS%:*} ${ADDRESS##*:} || exit 1\" did not complete successfully: exit code: 1\n------\n > [linux/arm64 1/2] RUN nc -zv 8.8.8.8 53 || exit 1:\n3.135 nc: 8.8.8.8 (8.8.8.8:53): Host is unreachable\n------\nError: unknown (2): failed to solve: process \"/bin/sh -c nc -zv ${ADDRESS%:*} ${ADDRESS##*:} || exit 1\" did not complete successfully: exit code: 1\n")
✘ Test testBuildNetworkAccess() failed after 4.978 seconds with 1 issue.
✘ Suite CLIBuilderTest failed after 48.434 seconds with 1 issue.
✘ Suite TestCLIBuildBase failed after 61.151 seconds with 1 issue.
↳ /* CLIBuildBase is the base class used for creating builder tests. Subtests classes
  // for these tests are nested in extensions of CLIBuildBase so that we can set
  // the serialized parallelization attribute across all builder tests.
  */
✘ Test run with 21 tests in 5 suites failed after 61.151 seconds with 1 issue.

Expected behavior

All tests pass.

Environment

N/A

Relevant log output

N/A

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions