Fix v1_chat_generate_request to allow for None content #5701
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Related to #5452.
Allow content field to be None, as per OpenAI types: https://github.com/openai/openai-python/blob/05810dd4088b6dbfc4194d7b0bea03eec236c83a/src/openai/types/chat/chat_completion_assistant_message_param.py#L46C67-L47C1
This fixes issues when running agentic-workflow related use of SGLang, where the assistant's output may be fed back into /v1/chat/completions as is without modification. For example, in the following flow:
LLM attempts tool call i.e. content is None and tool_calls is a valid list
LLM's response is fed back into messages in ChatCompletionRequest without modification
It will currently fail on the request generation step since a non-string typed content is automatically assumed to be a list, even though content can be None, even though the OpenAI spec allows this. This change fixes this issue.
Modifications
Checklist