Skip to content

Commit e4e9734

Browse files
jmhoadobrzyn
authored andcommitted
[Doc] Updated Llama section in tool calling docs to have llama 3.2 config info (vllm-project#16857)
Signed-off-by: jmho <[email protected]> Signed-off-by: Agata Dobrzyniewicz <[email protected]>
1 parent bafc9ac commit e4e9734

File tree

1 file changed

+13
-8
lines changed

1 file changed

+13
-8
lines changed

docs/source/features/tool_calling.md

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -152,12 +152,13 @@ Recommended flags: `--tool-call-parser mistral --chat-template examples/tool_cha
152152

153153
Supported models:
154154

155-
* `meta-llama/Meta-Llama-3.1-8B-Instruct`
156-
* `meta-llama/Meta-Llama-3.1-70B-Instruct`
157-
* `meta-llama/Meta-Llama-3.1-405B-Instruct`
158-
* `meta-llama/Meta-Llama-3.1-405B-Instruct-FP8`
155+
All Llama 3.1 and 3.2 models should be supported.
156+
157+
* `meta-llama/Llama-3.1-*`
158+
* `meta-llama/Llama-3.2-*`
159+
160+
The tool calling that is supported is the [JSON based tool calling](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/#json-based-tool-calling). For [pythonic tool calling](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/text_prompt_format.md#zero-shot-function-calling) introduced by the Llama-3.2 models, see the `pythonic` tool parser below.
159161

160-
The tool calling that is supported is the [JSON based tool calling](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/#json-based-tool-calling). For [pythonic tool calling](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/text_prompt_format.md#zero-shot-function-calling) in Llama-3.2 models, see the `pythonic` tool parser below.
161162
Other tool calling formats like the built in python tool calling or custom tool calling are not supported.
162163

163164
Known issues:
@@ -166,10 +167,14 @@ Known issues:
166167
2. The model can generate parameters with a wrong format, such as generating
167168
an array serialized as string instead of an array.
168169

169-
The `tool_chat_template_llama3_json.jinja` file contains the "official" Llama chat template, but tweaked so that
170-
it works better with vLLM.
170+
VLLM provides two JSON based chat templates for Llama 3.1 and 3.2:
171+
172+
* `examples/tool_chat_template_llama3.1_json.jinja` - this is the "official" chat template for the Llama 3.1
173+
models, but tweaked so that it works better with vLLM.
174+
* `examples/tool_chat_template_llama3.2_json.jinja` - this extends upon the Llama 3.1 chat template by adding support for
175+
images.
171176

172-
Recommended flags: `--tool-call-parser llama3_json --chat-template examples/tool_chat_template_llama3_json.jinja`
177+
Recommended flags: `--tool-call-parser llama3_json --chat-template {see_above}`
173178

174179
#### IBM Granite
175180

0 commit comments

Comments
 (0)