Open
Description
Describe the bug
I'm trying to use the AsyncTogether
client to create batched chat completion calls. With the Deepseek-V3
model in particular, I keep running into the following error after only a couple of batch calls (I'm on Build tier 3 and spent 100$).
2025-04-16 16:16:41,918 - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x14fbaea90>
Here is my relevant code as to how I'm using the client for my research:
client = AsyncTogether(
api_key = api_key
)
async def async_chat_completion(model_name, messages):
tasks = [client.chat.completions.create(
model=model_name,
messages=[{"role": "user", "content": message['prompt']}],
temperature=1,
seed=1,
max_tokens=1024
)
for message in messages
]
responses = await asyncio.gather(*tasks)
return responses
## This is within another function's for loop that constructs the batch
responses = asyncio.run(async_chat_completion(model_name, batch))
Based on other bug reports, the issue seems to be unclosed client sessions. Is this recommended behavior (not described in the examples/documentation), and if so how would one do this with the async client?
Metadata
Metadata
Assignees
Labels
No labels