Add DeepSeek-R1-0528 function call chat template

[Xu-Wenqing]

Motivation

DeepSeek-R1-0528 model support function call, add function call chat template.

Modifications

add DeepSeek-R1-0528 chat template for function call.

Usage:

python -m sglang.launch_server ... --tool-call-parser deepseekv3 --chat-template ./examples/chat_template/tool_chat_template_deepseekr1.jinja

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[gemini-code-assist]

Hello @Xu-Wenqing, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team, gemini-code-assist here to provide a summary of this pull request. This PR, currently in draft status, aims to integrate support for the DeepSeekR1-0528 model's function calling capabilities. It achieves this by adding a new Jinja chat template specifically designed to format messages according to the requirements of this model, including handling user, assistant, tool, and system roles, as well as the specific token sequences used for function calls and tool outputs.

Highlights

• DeepSeekR1-0528 Support: Adds a new chat template specifically for the DeepSeekR1-0528 model to enable correct message formatting for function calling.
• Function Calling Template: Implements the required message formatting for DeepSeekR1-0528's function calling mechanism, including handling tool calls and tool outputs using specific tokens and structures.
• New Jinja Template File: Introduces a new Jinja file (examples/chat_template/tool_chat_template_deepseekr1.jinja) containing the chat formatting logic for the new model.

Changelog

• examples/chat_template/tool_chat_template_deepseekr1.jinja
  • Added a new Jinja template file (tool_chat_template_deepseekr1.jinja) to define the chat format for the DeepSeekR1-0528 model.
  • Includes logic to process system messages and prepend them to the prompt (lines 5-14).
  • Appends tool descriptions to the system prompt if tools are provided (lines 17-28).
  • Formats user messages using the <｜User｜> token (lines 33-36).
  • Formats assistant messages containing tool calls using <｜tool calls begin｜>, <｜tool call begin｜>, <｜tool sep｜>, <｜tool call end｜>, and <｜tool calls end｜> tokens, including JSON arguments (lines 37-52).
  • Formats tool messages (outputs) using <｜tool outputs begin｜>, <｜tool output begin｜>, <｜tool output end｜>, and <｜tool outputs end｜> tokens (lines 65-73, 75-77).
  • Handles standard assistant responses, potentially stripping content before a </think> tag (lines 53-64).
  • Adds the <｜Assistant｜> token at the end if add_generation_prompt is true and the last message wasn't a tool output (lines 78-80).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Template takes the chat,
Formats tokens, calls the tools,
Model speaks just right.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new Jinja chat template for DeepSeekR1-0528 with function calling. The template structure is similar to the existing deepseekv3 template, which is a good approach for consistency.

I've identified a couple of high-severity issues related to state management (ns.is_first and ns.is_output_first flags) that could lead to incorrect prompt formatting for conversations involving multiple tool calls or sequences of tool outputs. These need to be addressed.

Additionally, for this template to be usable by the sglang system, it will need to be registered in python/sglang/lang/chat_template.py, and a corresponding matching function should be added or updated if necessary. Unit tests for this new template would also be essential to ensure its correctness across various chat scenarios.

Overall, a good start, but the identified issues should be fixed before this can be merged.

Summary of Findings

• State Management for Tool Calls: The template is missing crucial resets for ns.is_first and ns.is_output_first flags within assistant messages that initiate tool calls. This can lead to incorrect prompt formatting in conversations with multiple tool interactions. (High Severity)
• Missing Newline at End of File: The Jinja template file tool_chat_template_deepseekr1.jinja is missing a newline character at the end of the file. While this is a minor stylistic issue, it's good practice to include one. (Low Severity - not commented directly due to review settings)
• Template Registration and Testing: For the new template to be functional, it needs to be registered in python/sglang/lang/chat_template.py, and unit tests should be added to verify its behavior. (Out of scope for diff review, but important for PR completion)

Merge Readiness

This pull request is currently a draft and has a good foundation for adding the DeepSeekR1-0528 chat template. However, there are high-severity issues related to state management in the template that need to be addressed to ensure correct prompt generation. Additionally, the template needs to be integrated into the broader system (registration and testing) for it to be functional.

I recommend addressing the identified high-severity issues in the template code first. As I am an AI reviewer, I cannot approve pull requests. Please ensure further review and approval from project maintainers after the changes are made.