Infer time after conversion and ram usage

[romil611]

Nvidia's TensorRT is a handy tool improve inference time. However on converting DPT to TensorRT, the inference actually went up by almost 750%. The onnx itself took a lot of time during inference. For making the onnx file I had changed all the unflatten function to view.
If you have any leads on how to improve the inference time or improve the conversion process to the onnx then please share. Also, the ram usage is quite high. If you have suggestions on alternate functions to improve ram usage then do suggest.

[shivin101]

Hi can you let me know which TensorRT version and ONNX opset you used for the conversion to TensorRT

[romil611]

NVIDIA Jetson Xavier NX (Developer Kit Version)
L4T 32.6.1 [ JetPack 4.6 ]
Ubuntu 18.04.6 LTS
Kernel Version: 4.9.253-tegra
CUDA 10.2.300
CUDA Architecture: 7.2
OpenCV version: 4.4.0
OpenCV Cuda: YES
CUDNN: 8.2.1.32
TensorRT: 8.0.1.6
OPSet Version 11
Vision Works: 1.6.0.501
VPI: ii libnvvpi1 1.1.12 arm64 NVIDIA Vision Programming Interface library
Vulcan: 1.2.70