-
Notifications
You must be signed in to change notification settings - Fork 365
Closed
Labels
feature requestNew feature or requestNew feature or requestpriority: highrelease: v1.3Tagged to be included in v1.3Tagged to be included in v1.3
Description
Is your feature request related to a problem? Please describe.
In this graph:
INFO: [Torch-TensorRT - Debug Build] - Partitioned Graph: [Segment Block @0:
Target: TensorRT
Graph: graph(%index.1 : Tensor,
%data.1 : Tensor):
%2 : int = prim::Constant[value=4]() # test_int64.py:28:0
%3 : bool = prim::Constant[value=0]() # test_int64.py:28:0
%4 : NoneType = prim::Constant()
%index : Tensor = aten::to(%index.1, %2, %3, %3, %4) # test_int64.py:28:0
%data.3 : Tensor = aten::mul(%data.1, %data.1) # test_int64.py:29:0
return (%index, %data.3)
Segment Block @1:
Target: Torch
Graph: graph(%data.3 : Tensor,
%index : Tensor):
%2 : int = prim::Constant[value=1]() # test_int64.py:30:0
%0 : Tensor = aten::scatter(%data.3, %2, %index, %2) # test_int64.py:30:0
return (%0)
%index
is converted to int32, but in block 1, scatter
function needs data type int64 but got int32.
This is because TensorRT doesn't support int64, so Torch-TensorRT will cast all int64=>int32 to run them in TensorRT. However, when partitioning is enabled, some functions in Torch they still need type int64 to run.
Describe the solution you'd like
This could be supported if every aten::to operation is recorded and then cast the types between torch and tensorrt.
Metadata
Metadata
Labels
feature requestNew feature or requestNew feature or requestpriority: highrelease: v1.3Tagged to be included in v1.3Tagged to be included in v1.3
Type
Projects
Milestone
Relationships
Development
Select code repository
Activity
ncomly-nvidia commentedon Sep 12, 2022
@inocsin for viz.
inocsin commentedon Sep 13, 2022
We should also record which value has been truncated in converson process
Christina-Young-NVIDIA commentedon Dec 20, 2022
Duplicate of TensorRT #1546. Is this already support in the current codebase? Bo needs to confirm that we can close this issue.
peri044 commentedon Jan 4, 2023
@bowang007 Is this support already in the master ? https://github.com/pytorch/TensorRT/blob/master/core/partitioning/shape_analysis.cpp#L231-L259
Christina-Young-NVIDIA commentedon Jan 10, 2023
This one is indeed already supported in the master. @bowang007 to confirm and close.
bowang007 commentedon Jan 12, 2023
supported in #1407 , closing