Update DeepSeek-R1-0528 function call chat template

[Xu-Wenqing]

Motivation

DeepSeek updated default chat template of DeepSeek-R1-0528 model:
https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/commit/4236a6af538feda4548eca9ab308586007567f52#d2h-846292
So we are updating the function call chat template following DeepSeek official changes.

Modifications

Update DeepSeek-R1-0528 function call chat template following the change of official chat template:
https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/commit/4236a6af538feda4548eca9ab308586007567f52#d2h-846292

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[gemini-code-assist]

Hello @Xu-Wenqing, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello! Gemini here, providing a summary of this pull request. This PR aims to update the function call chat template for the DeepSeek-R1-0528 model. The motivation, as stated in the description, is to align the template with recent changes made to the official default chat template for this model by DeepSeek-AI. The changes primarily involve adjusting the formatting and structure of messages within the Jinja template, particularly how user, assistant, and tool messages are handled, and refining the logic for adding special tokens like <｜Assistant｜> and tool call/output markers.

Highlights

• Template Alignment: The core purpose of this PR is to update the tool_chat_template_deepseekr1.jinja file to match the latest official chat template for the DeepSeek-R1-0528 model, ensuring compatibility and correct behavior for function calling.
• Message Formatting Adjustments: Significant changes were made to how user and assistant messages are formatted. The <｜Assistant｜> token is now appended immediately after the user message and removed from the beginning of assistant messages (both with and without tool calls). This aligns with the updated official template structure.
• Tool Call and Output Handling: The logic for rendering tool calls and tool outputs has been refined. This includes adding logic to close a previous tool output block before an assistant message with tool calls, adjusting the formatting of tool call arguments, and adding newlines between subsequent tool outputs.
• State Management and Prompting Logic: A new state variable is_last_user was added to track if the last message processed was from the user. This new state, along with the existing is_tool state, is used to refine the condition for adding the final <｜Assistant｜> generation prompt at the end of the conversation.
• Internal Thought Handling: Logic was added to strip content before the last </think> tag in assistant messages, suggesting the template now accounts for models that might output internal reasoning before the final response or tool call.

Changelog

• examples/chat_template/tool_chat_template_deepseekr1.jinja
  • Added is_last_user variable to namespace for tracking the last message role (line 4).
  • Corrected newline characters from \n\n to `

` when joining system prompts (line 11).

• Cached message content in a content variable for easier access (line 33).
• Modified user message formatting to append <｜Assistant｜> immediately after the user's content (lines 34-39).
• Added logic to strip content before the last </think> tag in assistant messages (lines 40-44).
• Refined assistant message handling with tool calls, including adding tool outputs end if following a tool message, removing redundant <｜Assistant｜> tokens, and adjusting state variables (lines 45-66).
• Refined assistant message handling without tool calls, removing the redundant <｜Assistant｜> token (lines 67-75).
• Adjusted tool message formatting to add a newline before subsequent tool outputs (line 83).
• Updated the condition for adding the final <｜Assistant｜> generation prompt based on is_last_user and is_tool states (lines 90-92).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Template updated,
DeepSeek's new format embraced,
Tokens now align.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively updates the DeepSeek-R1-0528 function call chat template to align with the official changes from Hugging Face. The modifications appear to accurately reflect the upstream commit, which is crucial for correct model behavior.

Key improvements include:

• Correct handling of newlines (e.g., \n to ), which likely fixes potential misinterpretations by the model.
• Introduction of the is_last_user flag and associated logic adjustments for accurate <｜Assistant｜> token placement.
• More robust checks for tool_calls (using is defined and is not none).
• Improved readability by assigning message['content'] to a content variable early in the loop.

The changes are well-contained and directly address the motivation stated in the PR description.

Regarding the checklist in the PR description:

• Unit Tests: While this is a template change, having unit tests that render the template with various message histories (especially edge cases involving tool calls and multiple system messages) would be highly beneficial to ensure correctness and prevent regressions. Could you consider if adding such tests is feasible for this template?
• Documentation: If this chat template's specifics are documented elsewhere, please ensure those documents are updated to reflect these changes.

Overall, this is a good update. Thank you for addressing this!

Summary of Findings

• Correctness of Newline Handling: The PR correctly changes escaped newlines (e.g., \\n) to actual newlines (\n) in the Jinja template. This is crucial for ensuring the model receives correctly formatted input, as the previous version would have inserted literal '\n' strings instead of newline characters.
• Alignment with Official Template: The changes meticulously follow the official DeepSeek-R1-0528 chat template update, including logic for is_last_user and placement of special tokens.
• Testing and Documentation (Checklist): The PR checklist items for unit tests and documentation updates are unchecked. It would be beneficial to consider adding unit tests for chat templates to verify rendering for various scenarios and to update any relevant documentation.

Merge Readiness

The code changes are well-implemented and align with the official upstream updates. The correction of newline handling is particularly noteworthy. While I cannot approve the PR myself, I believe it is in good shape. Addressing the points from the PR checklist, especially regarding unit tests, would further enhance confidence. I recommend that other reviewers verify these changes before merging.

[danielhanchen]

@Xu-Wenqing @zhyncs Sorry to bother you guys, but I was investigating this chat template, but it seems like the original R1 and this one has a malformed ## Tools\n\n### Function string. Ie:

Make sure the JSON is valid.## Tools

### Function

You have the following functions available:```

Is `## Tools\n\n### Function` necessary and correct? @Xu-Wenqing According to your test in https://github.com/vllm-project/vllm/pull/18874 - BFCL gets 93.25%, so it's definitely working - I'm assuming the malformed strings aren't that important and doesn't affect results?