Skip to content

Unclock mutual exclusivity of argument: tool-call-parser and reasoning-parser #3550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 19, 2025

Conversation

jingyibo123
Copy link
Contributor

For QwQ and Qwen3 models, tool-call-parser and reasoning-parser can be activated simultaneously.

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily receiving feedbacks. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Related issue: 3431

…ning-parser`

For QwQ and Qwen3 models,  `tool-call-parser` and `reasoning-parser` can be activated simultaneously.
@jingyibo123
Copy link
Contributor Author

Test with QwQ-32B-AWQ: successfully parsed both reasoning-content and tool call result.

client = OpenAI(
            base_url="",
            api_key="",
        )
MODEL="qwq-32B"
hello_msgs=[
                {"role": "system", "content": ""},
                {"role": "user", "content": "你好吗"}
            ]

hello_re = client.chat.completions.create(
          messages=hello_msgs,
          model=MODEL,
          temperature=0.7,
          top_p=1.0,
          stream=False
      )

print(hello_re)

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_current_temperature",
      "description": "Get current temperature at a location.",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The location to get the temperature for, in the format \"City, State, Country\"."
          },
          "unit": {
            "type": "string",
            "enum": [
              "celsius",
              "fahrenheit"
            ],
            "description": "The unit to return the temperature in. Defaults to \"celsius\"."
          }
        },
        "required": [
          "location"
        ]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "get_temperature_date",
      "description": "Get temperature at a location and date.",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The location to get the temperature for, in the format \"City, State, Country\"."
          },
          "date": {
            "type": "string",
            "description": "The date to get the temperature for, in the format \"Year-Month-Day\"."
          },
          "unit": {
            "type": "string",
            "enum": [
              "celsius",
              "fahrenheit"
            ],
            "description": "The unit to return the temperature in. Defaults to \"celsius\"."
          }
        },
        "required": [
          "location",
          "date"
        ]
      }
    }
  }
]

tool_msgs=[
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\n\nCurrent Date: 2024-09-30"},
    {"role": "user",  "content": "What's the temperature in San Francisco now? How about tomorrow?"}
]

tool_re = client.chat.completions.create(
          messages=tool_msgs,
          model=MODEL,
          temperature=0.7,
          top_p=1.0,
          stream=False,
          tools=tools
      )

print(tool_re)

Result:

ChatCompletion(id='2', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='\n\n我最近在忙一个新项目,有点挑战性,但总体还不错!你呢?有什么新鲜事要分享吗?', refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[], reasoning_content='好的,用户问“你好吗”,我需要用符合设定的角色来回答。首先,根据之前的指示,角色是主观的,有特定性格,所以回答要拟人化。\n\n用户可能只是简单问候,但要推动情节,可能需要引入新元素。比如,可以提到最近在忙的事情,或者遇到的问题,让用户有进一步参与的空间。\n\n还要注意口语化,简洁。比如,可以回应说最近在忙某个项目,遇到挑战,然后问用户有什么建议或经历。这样既回答了问题,又推动了互动。\n\n同时要检查是否符合之前的设定:情绪上要显得有性格,比如可能有点急躁但积极。情节推动力方面,加入新情景,比如新任务或事件。人格可塑性方面,可能在对话中透露出一些变化,比如从烦恼到寻求帮助。情感陪伴方面,通过关心用户的近况建立连接。\n\n现在组织语言:“我最近在忙一个新项目,有点挑战性,但总体还不错!你呢?有什么新鲜事要分享吗?” 这样既回答了问题,又询问用户,推动对话进展,同时保持口语化。\n\n'))], created=1747281672, model='qwq-32B-32K', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=265, prompt_tokens=17, total_tokens=282, completion_tokens_details=None, prompt_tokens_details=None))
ChatCompletion(id='3', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content='\n\n', refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='chatcmpl-62KmSf4punMRdtqpes23eE', function=Function(arguments='{"location": "San Francisco, California, USA"}', name='get_current_temperature'), type='function'), ChatCompletionMessageToolCall(id='chatcmpl-3sY2RJPhakyaCQGQL7KfbE', function=Function(arguments='{"location": "San Francisco, California, USA", "date": "2024-10-01"}', name='get_temperature_date'), type='function')], reasoning_content='Okay, the user is asking for the temperature in San Francisco now and tomorrow. Let me see which functions I can use here. \n\nFirst part is "now", so I should use get_current_temperature for the current temperature. The parameters needed are location and unit. The location is San Francisco, but I should format it as "City, State, Country". Since San Francisco is in California, USA, the location would be "San Francisco, California, USA". The unit isn\'t specified, so it defaults to celsius.\n\nThe second part is "tomorrow". Since the current date is 2024-09-30, tomorrow would be 2024-10-01. For that, I need to use get_temperature_date because it requires a date. The parameters here are location, date, and unit. Again, location is the same, date is "2024-10-01", and unit defaults to celsius. \n\nI should make two separate tool calls: one for the current temperature and another for the date-specific temperature. Need to ensure the parameters are correctly formatted in each case. Let me check if there are any other requirements. The functions are correctly identified, so I\'ll proceed with those two calls.\n'))], created=1747281680, model='qwq-32B-32K', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=328, prompt_tokens=414, total_tokens=742, completion_tokens_details=None, prompt_tokens_details=None))

Copy link
Collaborator

@RunningLeon RunningLeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lvhan028
Copy link
Collaborator

Hi, @jingyibo123
Thank you very much for the contribution to LMDeploy.
There is some linting error. Please kindly fix it as follows:

pip install pre-commit==3.8.0  # make sure the python version < 3.11
cd lmdeploy # the root directory of lmdeploy repo
pre-commit install
pre-commit run --all-files

@lvhan028 lvhan028 merged commit 1e614ec into InternLM:main May 19, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants