[Bug] sglang 0.4.5-post3 deepseekv3-0324 function call if the message contains system information, the function behaves abnormally.

[stevewein]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

After deploying deepseekv3-0324 using sglang 0.4.5-post3-cu124, when testing the function call, it was found that if the message contains system information, the function behaves abnormally.

Reproduction

After deploying deepseekv3-0324 using sglang 0.4.5-post3-cu124, when testing the function call, it was found that if the message contains system information, the function behaves abnormally.

Environment

for example

---

curl "http://XXX/v1/chat/completions"
-H "Content-Type: application/json"
-d '{
"messages": [
{
"role": "system",
"content": "Use the tools to answer the questions."
},
{
"role": "user",
"content": "What is the weather in Tokyo"
}
],
"temperature": 0.3,
"max_tokens": 1000,
"model": "mssgpt_v3",
"tool_choice": "auto",
"tools": [
{
"type": "function",
"function": {
"name": "add",
"description": "Add two numbers",
"parameters": {
"properties": {
"a": {
"title": "A",
"type": "integer"
},
"b": {
"title": "B",
"type": "integer"
}
},
"required": [
"a",
"b"
],
"title": "addArguments",
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "get_secret_word",
"description": "",
"parameters": {
"properties": {},
"title": "get_secret_wordArguments",
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "",
"parameters": {
"properties": {
"city": {
"title": "City",
"type": "string"
}
},
"required": [
"city"
],
"title": "get_current_weatherArguments",
"type": "object"
}
}
}
]
}
'
the function call is wrong and "content" is not null, "tool_calls" is null

{
"id": "1d2854e227fe437191ee78b16e1bcfd8",
"object": "chat.completion",
"created": 1745811416,
"model": "mssgpt-v3",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "I'll add the numbers 7 and 22 for you. One moment.???????????????????????????",
"reasoning_content": null,
"tool_calls": null
},
"logprobs": null,
"finish_reason": "stop",
"matched_stop": 1
}
],
"usage": {
"prompt_tokens": 278,
"total_tokens": 349,
"completion_tokens": 71,
"prompt_tokens_details": null
}
}

---

BUT....

---

curl "http://xxxx/v1/chat/completions"
-H "Content-Type: application/json"
-d '{
"messages": [

    {
        "role": "user",
        "content": "What is the weather in Tokyo"
    }
],
"temperature": 0.3,
"max_tokens": 1000,
"model": "mssgpt_v3",
"tool_choice": "auto",
"tools": [
    {
        "type": "function",
        "function": {
            "name": "add",
            "description": "Add two numbers",
            "parameters": {
                "properties": {
                    "a": {
                        "title": "A",
                        "type": "integer"
                    },
                    "b": {
                        "title": "B",
                        "type": "integer"
                    }
                },
                "required": [
                    "a",
                    "b"
                ],
                "title": "addArguments",
                "type": "object"
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_secret_word",
            "description": "",
            "parameters": {
                "properties": {},
                "title": "get_secret_wordArguments",
                "type": "object"
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "",
            "parameters": {
                "properties": {
                    "city": {
                        "title": "City",
                        "type": "string"
                    }
                },
                "required": [
                    "city"
                ],
                "title": "get_current_weatherArguments",
                "type": "object"
            }
        }
    }
]

}
'
function call correct tool_calls reuslt successs

{
"id": "f387b0fe9e5a40e8b40a585d69dcfeb1",
"object": "chat.completion",
"created": 1745811225,
"model": "mssgpt_v3",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"reasoning_content": null,
"tool_calls": [
{
"id": "2",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{"city": "Tokyo"}"
}
}
]
},
"logprobs": null,
"finish_reason": "tool_calls",
"matched_stop": null
}
],
"usage": {
"prompt_tokens": 261,
"total_tokens": 282,
"completion_tokens": 21,
"prompt_tokens_details": null
}
}

[squirrelfish]

I had the same problem

[finger92]

Through my testing of function call of Deepseek V3, I have found that not all prompts can achieve satisfactory results with a high success rate. So I prefered to use an unified and tested prompt.
It seems that there are three possible solutions here.

1. The first is to add the system prompt in the request to the prompt in the code. (it may cause the output unstable)
2. Remove the system prompt from the request in the code.
3. When calling, do not pass a system prompt.
   what do you prefer?

[CatherineSue]

@finger92 will this problem happen when using your adapted chat template? I'm trying to adapt your chat template in the original PR but still downloading the deepseekv3 model

[finger92]

@finger92 will this problem happen when using your adapted chat template? I'm trying to adapt your chat template in the original PR but still downloading the deepseekv3 model

which chat template? I think we need to create a new one to support user's system prompt.

[CatherineSue]

@finger92 will this problem happen when using your adapted chat template? I'm trying to adapt your chat template in the original PR but still downloading the deepseekv3 model

which chat template? I think we need to create a new one to support user's system prompt.

I see. I was referring to this one.

[finger92]

@finger92 will this problem happen when using your adapted chat template? I'm trying to adapt your chat template in the original PR but still downloading the deepseekv3 model

which chat template? I think we need to create a new one to support user's system prompt.

I see. I was referring to this one.

Got it, I think using this template will put user's system prompt ahead of the one written in code. I think it will work but I haven't tested yet

[CatherineSue]

{"id":"ad1bb957bacc4813ad863f0d94592525","object":"chat.completion","created":1745987566,"model":"sgl-model","choices":[{"index":0,"message":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"id":"2","index":null,"type":"function","function":{"name":"get_current_weather","arguments":"{\"city\": \"Tokyo\"}"}}]},"logprobs":null,"finish_reason":"tool_calls","matched_stop":null}],"usage":{"prompt_tokens":274,"total_tokens":295,"completion_tokens":21,"prompt_tokens_details":null}}

I tried the request with system message, the chat template with deepseek-v3 on h200. It seems to work.

[CatherineSue]

{"id":"ad1bb957bacc4813ad863f0d94592525","object":"chat.completion","created":1745987566,"model":"sgl-model","choices":[{"index":0,"message":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"id":"2","index":null,"type":"function","function":{"name":"get_current_weather","arguments":"{\"city\": \"Tokyo\"}"}}]},"logprobs":null,"finish_reason":"tool_calls","matched_stop":null}],"usage":{"prompt_tokens":274,"total_tokens":295,"completion_tokens":21,"prompt_tokens_details":null}}

I tried the request with system message, the chat template with deepseek-v3 on h200. It seems to work.

Update: I realize that I kept the prompt engineering in adapter.py otherwise it doesn't work. The chat template might need some updates still.

In the chat template it adds tools under assistant role, but in adapter.py tools message prompt was added under system role.