❓ [Question] Is the module compiled by TRTorch thread safe?

[uni19]

Hi
If the native torchscript module is thread safe when its forward function is called from multithread, would the module compiled by TRTorch be thread safe?

[narendasan]

That is a good question. We haven't tested for thread safety. If I had to say I would say no, the logger for one is probably not thread safe and the actual execution probably needs to have a lock for inference since TensorRT ExecutionContext is not thread safe in the single module multiple thread case. Even in the multiple module case I think the logger will still cause problems. Can you file a feature issue to add thread-safety to the runtime?

[github-actions]

This issue has not seen activity for 30 days, Remove stale label or comment or this will be closed in 5 days

[github-actions]

This issue has not seen activity for 30 days, Remove stale label or comment or this will be closed in 5 days

[github-actions]

This issue has not seen activity for 30 days, Remove stale label or comment or this will be closed in 5 days

[github-actions]

This issue has not seen activity for 30 days, Remove stale label or comment or this will be closed in 5 days