Skip to content

AsyncTogether client throws unclosed client session error #296

Open
@venkatasg

Description

@venkatasg

Describe the bug
I'm trying to use the AsyncTogether client to create batched chat completion calls. With the Deepseek-V3 model in particular, I keep running into the following error after only a couple of batch calls (I'm on Build tier 3 and spent 100$).

2025-04-16 16:16:41,918 - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x14fbaea90>

Here is my relevant code as to how I'm using the client for my research:

client = AsyncTogether(
    api_key = api_key
)

async def async_chat_completion(model_name, messages):
    tasks = [client.chat.completions.create(
        model=model_name,
        messages=[{"role": "user", "content": message['prompt']}],
        temperature=1,
        seed=1,
        max_tokens=1024
        )
        for message in messages
    ]
    
    responses = await asyncio.gather(*tasks)
    
    return responses

## This is within another function's for loop that constructs the batch
responses = asyncio.run(async_chat_completion(model_name, batch))

Based on other bug reports, the issue seems to be unclosed client sessions. Is this recommended behavior (not described in the examples/documentation), and if so how would one do this with the async client?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions