"Can not connect to Ryuk" on macOS with Docker for Mac 2.4.0.0

[sazzer]

I've seen this exact error come and go a fair bit on here, but it's just today started happening for me.

I'm using testcontainers-java 1.12.0, but updating to the latest version has no impact at all.

When running my test, I get the following:

2020-08-28 17:19:18.778,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2020-08-28 17:19:18.833,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Will use 'okhttp' transport
2020-08-28 17:19:19.490,INFO ,[main]:,o.t.d.UnixSocketClientProviderStrategy,,,,,Accessing docker with local Unix socket
2020-08-28 17:19:19.491,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2020-08-28 17:19:19.492,INFO ,[main]:,o.t.DockerClientFactory,,,,,Docker host IP address is localhost
2020-08-28 17:19:19.680,INFO ,[main]:,o.t.DockerClientFactory,,,,,Connected to docker:
  Server Version: 19.03.13-beta2
  API Version: 1.40
  Operating System: Docker Desktop
  Total Memory: 2996 MB
2020-08-28 17:19:19.988,INFO ,[main]:,o.t.u.RegistryAuthLocator,,,,,Credential helper/store (docker-credential-desktop) does not have credentials for quay.io

And then a seemingly infinite amount of the following:

2020-08-28 17:19:20.733,WARN ,[testcontainers-ryuk]:,o.t.u.ResourceReaper,,,,,Can not connect to Ryuk at localhost:32781
java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at java.net.Socket.<init>(Socket.java:434)
	at java.net.Socket.<init>(Socket.java:211)
	at org.testcontainers.utility.ResourceReaper.lambda$start$1(ResourceReaper.java:113)
	at java.lang.Thread.run(Thread.java:748)

This is running with Java 1.8.0_222 from Amazon Corretto, running on macOS 10.15.6.

This definitely worked yesterday. I did have an update to the Docker Engine today, and I'm assuming that's relevant, but everything else I'm doing with Docker is all working fine - including a different project that uses the Rust version of TestContainers. (Though I notice that doesn't use Ryuk, which might be relevant)

Cheers

[mpschaeuble]

@sazzer Have you found a solution in the meantime? I have the same issue after updating Docker Engine today.

Edit: "Downgraded" from Docker Desktop Edge to Docker Desktop Stable and the issue has gone. Looks like a problem in combination with Docker 2.3.6.0.

[gesellix]

For the recent Docker Desktop 2.4.x Release on macOS, see #3159.

[BHSDuncan]

I'm on MacOS 10.14.6 (Mojave).

Both TestContainers v1.12.3 and the latest (v1.14.3) have ceased to work with the latest update of Docker that I received today. The exact same symptoms as the OP happen.

Version 2.4.0.0 (48506) from the "stable" Channel.

Engine is: 19.03.13

Anyone found this and/or have a workaround, short of downgrading/rolling back the version of Docker Desktop (Community)?

[benlei-gfm]

I'm also experiencing the same issue

[karuppiah7890]

Same issue. 🙋

Docker Desktop Community - Version: 2.4.0.0 (48506). Channel: stable.

$ docker version
Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 16:58:31 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

testImplementation "org.testcontainers:testcontainers:1.14.3"

[rnorth]

Docker for Mac 2.4.0.0 was released yesterday, pushing this issue into the stable version.

We released 1.15.0-rc2 which includes a fix (#3159) for compatibility - please upgrade.

[telapo]

Docker for Mac 2.4.0.0 was released yesterday, pushing this issue into the stable version.

We released 1.15.0-rc2 which includes a fix (#3159) for compatibility - please upgrade.

No-one has asked me, but I confirm that 1.15.0-rc2 works ;-)

[tmohme]

Same here: 1.15.0-rc2 solves the problem.
Thanks for the fast reaction 👍

[jonckvanderkogel]

Thanks @rnorth it does indeed fix one issue it seems. However, I'm now facing a different issue that used to work before this.

I'm creating a PostgreSQLContainer based on a postgis image like this:

PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgis/postgis:12-3.0-alpine")
                .withDatabaseName("panoramix_db")
                .withUsername("sa")
                .withPassword("sa");

Before I updated my Docker Desktop this used to work but now I'm getting this error:

Caused by: java.lang.IllegalStateException: Failed to verify that image 'postgis/postgis:12-3.0-alpine' is a compatible substitute for 'postgres'. This generally means that you are trying to use an image that Testcontainers has not been designed to use. If this is deliberate, and if you are confident that the image is compatible, you should declare compatibility in code using the `asCompatibleSubstituteFor` method. For example:
   DockerImageName myImage = DockerImageName.parse("postgis/postgis:12-3.0-alpine").asCompatibleSubstituteFor("postgres");

EDIT
Figured it out. So to fix the above code snippet it would become something like this:

PostgreSQLContainer postgreSQLContainer =
                new PostgreSQLContainer(
                        DockerImageName.parse("postgis/postgis:12-3.0-alpine").asCompatibleSubstituteFor("postgres"))
                        .withDatabaseName("panoramix_db")
                        .withUsername("sa")
                        .withPassword("sa");

[jkroepke]

I have the same issue.

Using 1.15.0-rc2 could be a workaround. It's available in maven central.

https://mvnrepository.com/artifact/org.testcontainers/testcontainers/1.15.0-rc2

[schophil]

1.15.0-rc2 works for me to.

I do have some questions regarding this topic:
When will 1.15.0 be released?
What is the impact of not using ryuk?

[jkroepke]

What is the impact of not using ryuk?

Containers will not terminated.

[tonymurphy]

Having issues launching servers using 1.15.0-rc2 and Docker 2.4.0.0 on MacOS

12:42:35.588 [main] INFO o.t.d.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/usr/bin, /bin, /usr/sbin, /sbin]) 12:42:36.026 [main] ERROR o.t.d.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were: 12:42:36.026 [main] ERROR o.t.d.DockerClientProviderStrategy - UnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (com/fasterxml/jackson/annotation/JsonMerge). Root cause ClassNotFoundException (com.fasterxml.jackson.annotation.JsonMerge) 12:42:36.026 [main] ERROR o.t.d.DockerClientProviderStrategy - As no valid configuration was found, execution cannot continue

Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=hub.docker.com/postgres:11.9-alpine, imagePullPolicy=DefaultPullPolicy()) at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1282) at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:616) at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:321) ... 55 more Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration