multiple prompts in a batch is not currently supported

[pseudotensor]

Is this planned? Seems like good idea to support full OpenAI behavior and any batching is already handled well by vLLM, so should be relatively easy I would guess?

vllm/vllm/entrypoints/openai/api_server.py

Lines 396 to 401 in acbed3e

	elif isinstance(first_element, (str, list)):
	# TODO: handles multiple prompt case in list[list[int]]
	if len(request.prompt) > 1:
	return create_error_response(
	HTTPStatus.BAD_REQUEST,
	"multiple prompts in a batch is not currently supported")

[simon-mo]

This has been addressed.