implement packed quantize row / dequantize row API #3915
Closed
+625
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D72121939 |
✅ Deploy Preview for pytorch-fbgemm-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
SamGinzburg
force-pushed
the
export-D72121939
branch
from
19330ae to
bafd0fd
Compare
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
bafd0fd to
a4146f3
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72121939 |
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 Pull Request resolved: pytorch#3915 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
a4146f3 to
99d101a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72121939 |
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 Pull Request resolved: pytorch#3915 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
99d101a to
b932338
Compare
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
b932338 to
051e6de
Compare
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
051e6de to
9f3672a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D72121939 |
SamGinzburg
added a commit
to SamGinzburg/FBGEMM
that referenced
this pull request
Apr 2, 2025
Summary: X-link: facebookresearch/FBGEMM#1004 Pull Request resolved: pytorch#3915 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
SamGinzburg
force-pushed
the
export-D72121939
branch
from
9f3672a to
75cccc8
Compare
Summary: X-link: facebookresearch/FBGEMM#1004 Pull Request resolved: pytorch#3915 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939
|
This pull request was exported from Phabricator. Differential Revision: D72121939 |
SamGinzburg
force-pushed
the
export-D72121939
branch
from
75cccc8 to
41cb838
Compare
|
This pull request has been merged in def7bbe. |
q10
pushed a commit
to q10/FBGEMM
that referenced
this pull request
Apr 10, 2025
Summary: Pull Request resolved: facebookresearch/FBGEMM#1004 X-link: pytorch#3915 API for a packed version of quantize/dequantize row. This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor. Example usage: ``` a = torch.randn(shape, dtype=torch.bfloat16, device="cuda") packed_values = quantize_fp8_packed_row_raw( a, use_triton=True, ) # Undo scaling. a_bf16 = dequantize_fp8_packed_row(packed_values) torch.testing.assert_close(a_bf16, a, atol=2e-1, rtol=1e-1) ``` A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing). Reviewed By: jiawenliu64 Differential Revision: D72121939 fbshipit-source-id: c88f99977e21e64f6f6f7dbb3c7430c2c2b56d0e
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
API for a packed version of quantize/dequantize row.
This version will return a single, contiguous tensor in memory instead of returning two tensors, and operates on the contiguous tensor.
Example usage:
A third API: "quantize_fp8_packed_row" mimics the API of quantize_fp8_row (mainly for testing).
Differential Revision: D72121939