httpx.ConnectError: All connection attempts failed

[ganschen]

Dear all, I hope I find you good.

I'm facing a weird issue with the AsyncClient:

Platform

• docker, python:3.9-alpine3.13

version

• httpx==0.21.1

Code

client = httpx.AsyncClient()    
req = client.build_request("GET", url)    
r = await client.send(req, stream=True)    

Traceback

File "/code/./app/learning/api/api_file.py", line 105, in download
    r = await client.send(req, stream=True)
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1600, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1628, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1665, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1702, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 291, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed

Best Regards,

[tomchristie]

So, it couldn't connect. Although the error messaging obviously isn't very helpful for you in determining why.

Take things step by step. First, simplify it right down...

import httpx

httpx.get(url)  # What does this result in?

Try a different library...

import requests

requests.get(url)  # How about now?

Try a different tool entirely...

Try a different library...

$ curl <url>  # This?

Also, is this a public URL that you can share, so others can help you more easily?

[ganschen]

Thanks for replying, this is a private AWS S3 storage call, and error is not always happen, it only happens in one of the hundreds of concurrent calls, so wondering it's a client error or a server error.

[YuvalHad1220]

did you resolve the issue?

[akdev-tech]

The same errors when trying to make 15 concurrent calls from Docker running on AWS EC2 instance

[ganschen]

@MilkaPeeka @akdev-tech , i resolved issue by upgrading anyio from 3.4.0 to 3.5.0

[akdev-tech]

Just made some tests. I meet the same problem when sharing httpx.AsyncClient() instance between coroutines. But when I changed my code to every coroutine using own httpx.AsyncClient() instance the problem has gone.

Before:

client = httpx.AsyncClient()

def coro():
    req = await client.post("<url>", json=data)

After:

def coro():
    async with httpx.AsyncClient() as client:
        req = await client.post("<url>", json=data)

[HMaker]

So httpx is not coroutine-safe?

[zanieb]

httpx is coroutine safe but your client context should span the lifetime of all the coroutines e.g.

do

async with httpx.AsyncClient(...) as client:
    async with anyio.create_task_group() as tg:
        tg.start_soon(client.post, ...)
        tg.start_soon(client.post, ...)

not

async with anyio.create_task_group() as tg:
    async with httpx.AsyncClient(...) as client:
        tg.start_soon(client.post, ...)
    tg.start_soon(client.post, ...)

If you have problems please provide a complete example. I'm not sure what @akdev-tech was doing but you should not need to context manage the client in every coroutine.

[johannesloibl]

I'm doing it like that (apparently correctly):

async def upload_shards_async(directory: Path, client_kwargs: dict, concurrent_uploads: int = 100):
    semaphore = anyio.Semaphore(concurrent_uploads)
    async with AsyncRestClient(**client_kwargs) as client, anyio.create_task_group() as tg:
        for i, (stem, meta_file, payload_file) in enumerate(FileSystemShardExporter.iter_shards(directory)):
            tg.start_soon(upload_shard_async, client, payload_file, meta_file, semaphore, name=f"{i}_{stem}")

The issue occurs every other thousand tasks :(

[Yuhui0620]

Same as @akdev-tech , I meet the same error when getting stream response with one global httpx.AsyncClient instance:

client = httpx.AsyncClient()

async def request():
    response = await self.client.send(req, stream=True)

But the error is not raised offen(about 1~2% per day), so I am not sure whether it's a server-side problem or a client-side problem.

Full traceback below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 122, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
    return await self._backend.connect_tcp(
  File "/usr/local/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: All connection attempts failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/gatewayserver/routers/v1/chat.py", line 55, in forward_chat_request
    streaming_response = await chat_forward_service.forward_request(backend, request, background_tasks)
  File "/usr/local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 47, in __call__
    do = self.iter(retry_state=retry_state)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "/data/gatewayserver/services/forward.py", line 151, in forward_request
    response = await self.client.send(req, stream=True)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 372, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed

[tommyhe6]

We are getting the same issue still when using httpx with openai. It only fails a small percentage of the time in our deployed instance. Did you end up figuring out why it happened?

[Yuhui0620]

It was a server-side memory leak problem in my case, which made the server down at sometime.