feat: Allow passing tokenizer template values to HuggingFace chat models #31489

Aristote-code · 2025-06-04T09:44:34Z

This commit introduces two ways for you to customize chat templating when using ChatHuggingFace:

Custom Chat Template String: A new chat_template parameter has been added to the ChatHuggingFace constructor. This allows you to provide a custom Jinja template string, which will be assigned to tokenizer.chat_template after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures.
Dynamic Template Variables via **kwargs: The _to_chat_prompt method in ChatHuggingFace (which is responsible for formatting messages using tokenizer.apply_chat_template) has been modified to accept arbitrary keyword arguments (**kwargs). These kwargs are then passed directly to tokenizer.apply_chat_template. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to invoke, stream, generate, etc.

Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to apply_chat_template. Documentation has been updated in the ChatHuggingFace class docstring and in the HuggingFace integration notebook (docs/docs/integrations/chat/huggingface.ipynb) to explain these new features with examples.

This change addresses issue #31470 by providing a flexible way for you to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

Thank you for contributing to LangChain!

PR title: "package: description"
- Where "package" is whichever of langchain, core, etc. is being modified. Use "docs: ..." for purely docs changes, "infra: ..." for CI changes.
- Example: "core: add foobar LLM"
PR message: Delete this entire checklist and replace with
- Description: a description of the change
- Issue: the issue # it fixes, if applicable
- Dependencies: any dependencies required for this change
- Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out!
Add tests and docs: If you're adding a new integration, please include
1. a test for the integration, preferably unit tests that do not rely on network access,
2. an example notebook showing its use. It lives in docs/docs/integrations directory.
Lint and test: Run make format, make lint and make test from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:

Make sure optional dependencies are imported within a function.
Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
Most PRs should not touch more than one package.
Changes should be backwards compatible.

If no one reviews your PR within a few days, please @-mention one of baskaryan, eyurtsev, ccurme, vbarda, hwchase17.

This commit introduces two ways for you to customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. This change addresses issue langchain-ai#31470 by providing a flexible way for you to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

vercel · 2025-06-04T09:44:39Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 7, 2025 2:00pm

codspeed-hq · 2025-06-04T09:46:12Z

CodSpeed Walltime Performance Report

Merging #31489 will not alter performance

_{Comparing Aristote-code:feat-huggingface-chat-template-values (eb262b1) with master (ece9e31)}

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched benchmarks

codspeed-hq · 2025-06-04T09:52:21Z

CodSpeed Instrumentation Performance Report

Merging #31489 will not alter performance

_{Comparing Aristote-code:feat-huggingface-chat-template-values (eb262b1) with master (ece9e31)}

Summary

✅ 13 untouched benchmarks

This commit introduces two ways for you to customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. Linting errors (E501 Line too long) found in a previous CI run have been fixed. This change addresses issue langchain-ai#31470 by providing a flexible way for you to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

This commit introduces two ways for you to customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. Linting errors (E501 Line too long) found in a previous CI run have been fixed by reformatting dictionary comprehensions and other long lines. This change addresses issue langchain-ai#31470 by providing a flexible way for you to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

This commit introduces two ways for you to customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. Further fixes for E501 (Line too long) linting errors based on CI feedback. This change addresses issue langchain-ai#31470 by providing a flexible way to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

This commit introduces two ways you can customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. This commit also includes comprehensive linting and formatting fixes for the `libs/partners/huggingface` directory to align with project standards, including `pyupgrade`, `ruff`, `black`, and `isort` changes. This change addresses issue langchain-ai#31470 by providing a flexible way to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

This commit introduces two ways to customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. This commit also includes comprehensive linting and formatting fixes for the `libs/partners/huggingface` directory to align with project standards, including `pyupgrade`, `ruff --fix`, `black`, `isort`, and `ruff format` changes. This change addresses issue langchain-ai#31470 by providing a flexible way to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

This commit introduces two ways you can customize chat templating when using `ChatHuggingFace`: 1. **Custom Chat Template String**: A new `chat_template` parameter has been added to the `ChatHuggingFace` constructor. This allows you to provide a custom Jinja template string, which will be assigned to `tokenizer.chat_template` after the tokenizer is loaded. This gives you full control over the chat prompt formatting if the default template associated with a model is not suitable or if you want to experiment with different prompt structures. 2. **Dynamic Template Variables via `**kwargs`**: The `_to_chat_prompt` method in `ChatHuggingFace` (which is responsible for formatting messages using `tokenizer.apply_chat_template`) has been modified to accept arbitrary keyword arguments (`**kwargs`). These `kwargs` are then passed directly to `tokenizer.apply_chat_template`. This allows you to define variables in your Jinja chat templates (either the default one or a custom one) and provide values for these variables dynamically during calls to `invoke`, `stream`, `generate`, etc. Unit tests have been added to verify these new functionalities, including setting custom templates and passing keyword arguments to `apply_chat_template`. Documentation has been updated in the `ChatHuggingFace` class docstring and in the HuggingFace integration notebook (`docs/docs/integrations/chat/huggingface.ipynb`) to explain these new features with examples. This commit also includes comprehensive linting and formatting fixes for the `libs/partners/huggingface` directory to align with project standards. This includes: - Fixing E501 (line too long) errors. - Correcting an invalid `type: ignore[import]` comment in `llms/huggingface_endpoint.py`. - Resolving an `AttributeError` in mypy for `tests/unit_tests/test_chat_models.py`. - Ensuring `pyupgrade`, `ruff`, `black`, and `isort` checks pass. This change addresses issue langchain-ai#31470 by providing a flexible way to pass tokenizer template values, interpreted as HuggingFace chat template strings and variables for those Jinja templates.

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. langchain Related to the langchain package labels Jun 4, 2025

vercel bot deployed to Preview June 4, 2025 09:56 View deployment

vercel bot deployed to Preview June 4, 2025 10:27 View deployment

vercel bot deployed to Preview June 4, 2025 11:20 View deployment

vercel bot deployed to Preview June 4, 2025 14:31 View deployment

vercel bot deployed to Preview June 4, 2025 18:25 View deployment

vercel bot deployed to Preview June 4, 2025 18:41 View deployment

vercel bot deployed to Preview June 7, 2025 14:00 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Allow passing tokenizer template values to HuggingFace chat models #31489

feat: Allow passing tokenizer template values to HuggingFace chat models #31489

Uh oh!

Aristote-code commented Jun 4, 2025

Uh oh!

vercel bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

feat: Allow passing tokenizer template values to HuggingFace chat models #31489

Are you sure you want to change the base?

feat: Allow passing tokenizer template values to HuggingFace chat models #31489

Uh oh!

Conversation

Aristote-code commented Jun 4, 2025

Uh oh!

vercel bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Walltime Performance Report

Merging #31489 will not alter performance

Summary

Uh oh!

codspeed-hq bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Instrumentation Performance Report

Merging #31489 will not alter performance

Summary

Uh oh!

Uh oh!

vercel bot commented Jun 4, 2025 •

edited

Loading

codspeed-hq bot commented Jun 4, 2025 •

edited

Loading

codspeed-hq bot commented Jun 4, 2025 •

edited

Loading