windows上使用anconda安装tensorrt环境

windows上使用anconda安装tensorrt环境

    • 1 安装tensorrt
      • 1.1 下载最新的稳定的tensorrt 8.6.1(tensorrt对应的cuda、cudnn等版本是参考链接4)
      • 1.2 将tensorrt添加到环境变量
      • 1.3 安装tensorrt依赖
      • 1.4 安装Pycuda
      • 1.5 安装pytorch
    • 2 测试
      • 2.1 测试TensorRT 样例(这个测试主要来源于参考链接1)
      • 2.2 测试trtexec是否可以使用(这个测试主要来源于参考链接2)
        • 2.2.1 生成pytorch模型
        • 2.2.2 将pytorch模型转化为onnx
        • 2.2.3 将ONNX格式转成TensorRT格式
        • 2.2.4 测试生成的tensorrt模型
    • 3 容易出现的问题
      • 3.1 cuda 和 tensorrt 版本不匹配的问题
      • 3.2 出现编译和加载时不是同一个cuda cuBLAS/cuBLAS
    • 4 参考链接

本次使用的window环境是win 11,windows环境安装cuda(cuda版本为11.6.2)和cudnn(cudnn版本为8.8.0其实应该下载8.9.0tensorrt 8.6.1对应的cudnn版本是8.9.0,如下图1),anconda的安装就不用介绍了,如果不会安装,可以参考这篇文章
在这里插入图片描述 图 1 图1 1

1 安装tensorrt

1.1 下载最新的稳定的tensorrt 8.6.1(tensorrt对应的cuda、cudnn等版本是参考链接4)

从nvidia官方文件中可以看出,在windows上安装tensorrt只能通过Zip File Installation这个安装方式来进行安装。

  • 首先前往tensorrt官网下载,登录会出现不同版本的tensorrt资源,如图2,点击TensorRT 8
    在这里插入图片描述 图 2 图2 2
  • 然后,直接下载下图3中的TensorRT 8.6 GA for Windows 10 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7 and 11.8 ZIP Package
    在这里插入图片描述 图 3 图3 3

1.2 将tensorrt添加到环境变量

  • 下载完毕后,将其解压,并且进入lib子文件夹,如下图4所示,将路径D:\TensorRT-8.6.1.6\lib添加到系统环境变量中。
    在这里插入图片描述 图 4 图4 4

1.3 安装tensorrt依赖

创建一个anconda环境,python版本为python==3.10.12

conda create -n tensorrt python==3.10.12

激活环境

activate tensorrt

进入刚才解压后的TensorRT文件夹内的python子目录,根据python版本选择好对用的whl文件,如下图5所示,并执行下面代码

pip install D:\TensorRT-8.6.1.6\python\tensorrt-8.6.1-cp310-none-win_amd64.whl

在这里插入图片描述 图 5 图5 5上面代码执行结果如下所示

C:\Users\Administrator>conda create -n tensorrt python==3.10.12
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done

1.4 安装Pycuda

前往下载Pycuda的网站,找到Pycuda,并点击Pycuda,就会跳到下图6下载Pycuda版本的网站,然后下载pycuda‑2022.1+cuda116‑cp310‑cp310‑win_amd64.whl
在这里插入图片描述 图 6 图6 6进入tensorrt的conda虚拟环境,输入以下代码指令安装Pycuda

pip install C:\Users\Administrator\Downloads\pycuda‑2022.1+cuda116‑cp310‑cp310‑win_amd64.whl

执行结果如下所示,就代表成功了

(tensorrt) C:\Users\Administrator>pip install C:\Users\Administrator\Downloads\pycuda-2022.1+cuda116-cp310-cp310-win_amd64.whl
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing c:\users\administrator\downloads\pycuda-2022.1+cuda116-cp310-cp310-win_amd64.whl
Collecting pytools>=2011.2 (from pycuda==2022.1+cuda116)Downloading pytools-2023.1.1-py2.py3-none-any.whl.metadata (2.7 kB)
Collecting appdirs>=1.4.0 (from pycuda==2022.1+cuda116)Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting mako (from pycuda==2022.1+cuda116)Downloading Mako-1.3.0-py3-none-any.whl.metadata (2.9 kB)
Requirement already satisfied: platformdirs>=2.2.0 in c:\users\administrator\appdata\roaming\python\python310\site-packages (from pytools>=2011.2->pycuda==2022.1+cuda116) (4.1.0)
Collecting typing-extensions>=4.0 (from pytools>=2011.2->pycuda==2022.1+cuda116)Downloading typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB)
Collecting MarkupSafe>=0.9.2 (from mako->pycuda==2022.1+cuda116)Downloading MarkupSafe-2.1.4-cp310-cp310-win_amd64.whl.metadata (3.1 kB)
Downloading pytools-2023.1.1-py2.py3-none-any.whl (70 kB)---------------------------------------- 70.6/70.6 kB 256.7 kB/s eta 0:00:00
Downloading Mako-1.3.0-py3-none-any.whl (78 kB)---------------------------------------- 78.6/78.6 kB 1.5 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.4-cp310-cp310-win_amd64.whl (17 kB)
Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Installing collected packages: appdirs, typing-extensions, MarkupSafe, pytools, mako, pycuda
Successfully installed MarkupSafe-2.1.4 appdirs-1.4.4 mako-1.3.0 pycuda-2022.1+cuda116 pytools-2023.1.1 typing-extensions-4.9.0

1.5 安装pytorch

进入tensorrt虚拟环境中,安装pytorch,注意这个安装pytorch,一定要使用pip的方式安装,不要使用conda的方式安装

pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116

安装成功后,可以查看pytorch的cuda是不是可以用

import torch
torch.cuda.is_available()  # 为True则可以用

2 测试

2.1 测试TensorRT 样例(这个测试主要来源于参考链接1)

tensorrt官方提供了可供测试的样例,进入刚才下载好的tensorrt文件夹下面的samples\python\network_api_pytorch_mnist目录下,这里我们选择一个手写数字识别的示例,如下图7所示。
在这里插入图片描述 图 7 图7 7拷贝路径,在tensorrt的虚拟环境下,cd 此路径,然后输入如下指令

python sample.py

执行结果如下,就代表成功了

(tensorrt) D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist>python sample.py
Train Epoch: 1 [0/60000 (0%)]   Loss: 2.292776
Train Epoch: 1 [6400/60000 (11%)]       Loss: 0.694761
Train Epoch: 1 [12800/60000 (21%)]      Loss: 0.316812
Train Epoch: 1 [19200/60000 (32%)]      Loss: 0.101704
Train Epoch: 1 [25600/60000 (43%)]      Loss: 0.087654
Train Epoch: 1 [32000/60000 (53%)]      Loss: 0.230672
Train Epoch: 1 [38400/60000 (64%)]      Loss: 0.189763
Train Epoch: 1 [44800/60000 (75%)]      Loss: 0.157570
Train Epoch: 1 [51200/60000 (85%)]      Loss: 0.043530
Train Epoch: 1 [57600/60000 (96%)]      Loss: 0.107672Test set: Average loss: 0.0927, Accuracy: 9732/10000 (97%)Train Epoch: 2 [0/60000 (0%)]   Loss: 0.049581
Train Epoch: 2 [6400/60000 (11%)]       Loss: 0.063095
Train Epoch: 2 [12800/60000 (21%)]      Loss: 0.086241
Train Epoch: 2 [19200/60000 (32%)]      Loss: 0.100145
Train Epoch: 2 [25600/60000 (43%)]      Loss: 0.087662
Train Epoch: 2 [32000/60000 (53%)]      Loss: 0.064293
Train Epoch: 2 [38400/60000 (64%)]      Loss: 0.053872
Train Epoch: 2 [44800/60000 (75%)]      Loss: 0.153787
Train Epoch: 2 [51200/60000 (85%)]      Loss: 0.065774
Train Epoch: 2 [57600/60000 (96%)]      Loss: 0.067333Test set: Average loss: 0.0520, Accuracy: 9835/10000 (98%)[01/26/2024-20:43:07] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:112: DeprecationWarning: Use set_memory_pool_limit instead.config.max_workspace_size = common.GiB(1)
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:75: DeprecationWarning: Use add_convolution_nd instead.conv1 = network.add_convolution(
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:78: DeprecationWarning: Use stride_nd instead.conv1.stride = (1, 1)
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:80: DeprecationWarning: Use add_pooling_nd instead.pool1 = network.add_pooling(input=conv1.get_output(0), type=trt.PoolingType.MAX, window_size=(2, 2))
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:81: DeprecationWarning: Use stride_nd instead.pool1.stride = (2, 2)
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:85: DeprecationWarning: Use add_convolution_nd instead.conv2 = network.add_convolution(pool1.get_output(0), 50, (5, 5), conv2_w, conv2_b)
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:86: DeprecationWarning: Use stride_nd instead.conv2.stride = (1, 1)
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:88: DeprecationWarning: Use add_pooling_nd instead.pool2 = network.add_pooling(conv2.get_output(0), trt.PoolingType.MAX, (2, 2))
D:\TensorRT-8.6.1.6\samples\python\network_api_pytorch_mnist\sample.py:89: DeprecationWarning: Use stride_nd instead.pool2.stride = (2, 2)
[01/26/2024-20:43:11] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
Test Case: 2
Prediction: 2

2.2 测试trtexec是否可以使用(这个测试主要来源于参考链接2)

2.2.1 生成pytorch模型

使用pytorch官方提供的resnet34训练flower数据集,得到pytorch模型,代码来源https://github.com/WZMIAOMIAO/deep-learning-for-image-processing/tree/master/pytorch_classification/Test5_resnet,flower数据集的来源也是这个链接的上一文件的readme.md,训练完成后,会得到resNet34.pth,如下图8所示
在这里插入图片描述 图 8 图8 8

2.2.2 将pytorch模型转化为onnx

这里就是以Pytorch官方提供的ResNet34为例(也就是上面代码训练好的resNet34.pth),直接从torchvision中实例化ResNet34并载入自己在flower_photos数据集上训练好的权重,然后在转成ONNX格式,示例代码如下:

import torch
import torch.onnx
import onnx
import onnxruntime
import numpy as np
from torchvision.models import resnet34device = torch.device("cpu")def to_numpy(tensor):return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()def main():weights_path = "resNet34.pth"onnx_file_name = "resnet34.onnx"batch_size = 1img_h = 224img_w = 224img_channel = 3# create model and load pretrain weightsmodel = resnet34(pretrained=False, num_classes=5)model.load_state_dict(torch.load(weights_path, map_location='cpu'))model.eval()# input to the model# [batch, channel, height, width]x = torch.rand(batch_size, img_channel, img_h, img_w, requires_grad=True)torch_out = model(x)# export the modeltorch.onnx.export(model,             # model being runx,                 # model input (or a tuple for multiple inputs)onnx_file_name,    # where to save the model (can be a file or file-like object)input_names=["input"],output_names=["output"],verbose=False)# check onnx modelonnx_model = onnx.load(onnx_file_name)onnx.checker.check_model(onnx_model)ort_session = onnxruntime.InferenceSession(onnx_file_name)# compute ONNX Runtime output predictionort_inputs = {ort_session.get_inputs()[0].name: to_numpy(x)}ort_outs = ort_session.run(None, ort_inputs)# compare ONNX Runtime and Pytorch results# assert_allclose: Raises an AssertionError if two objects are not equal up to desired tolerance.np.testing.assert_allclose(to_numpy(torch_out), ort_outs[0], rtol=1e-03, atol=1e-05)print("Exported model has been tested with ONNXRuntime, and the result looks good!")if __name__ == '__main__':main()

在运行上面代码之前,还得在tensorrt虚拟环境中安装onnx(onnx==1.12.0)和onnxruntime(onnxruntime==1.12.0)(onnx和onnxruntime的版本对应关系可以参考这个链接,当然如果要查看最新的版本的,可以直接google哦)

pip install onnx==1.12.0
pip install onnxruntime==1.12.0

注意,这里将Pytorch模型转成ONNX后,又利用ONNXRUNTIME载入导出的模型,然后输入同样的数据利用np.testing.assert_allclose方法对比转换前后输出的差异,其中rtol代表相对偏差,atol代表绝对偏差,如果两者的差异超出指定的精度则会报错。在转换后,会在当前文件夹中生成一个resnet34.onnx文件。

2.2.3 将ONNX格式转成TensorRT格式

将ONNX转成TensorRT engine的方式有多种,其中最简单的就是使用trtexec工具。在上面2.2.2章节中已经将Pyotrch中的Resnet34转成ONNX格式了,接下来可以直接使用trtexec工具将其转为TensorRT engine格式:

trtexec --onnx=resnet34.onnx --saveEngine=trt_output/resnet34.trt

其中:

  • –onnx是指向生成的onnx模型文件路径
  • –saveEngine是保存TensorRT engine的文件路径(发现一个小问题,就是保存的目录必须提前创建好,如果没有创建的话就会报错)

在进行trtexec之前,还需要将trtexec.exeD:\TensorRT-8.6.1.6\bin添加到环境变量中,具体得添加过程就不赘述了,需要添加得路径如下图9所示
在这里插入图片描述 图 9 图9 9添加环境变量后,如果使用VScode,请务必将VScode关闭后,在再次打开在上述终端中执行指令;也可以通过win 11自带的cmd窗口执行上述命令,两种方法的都可以,执行结果如下所示:

(tensorrt) C:\Users\Administrator\Desktop\resnet>trtexec --onnx=resnet34.onnx --saveEngine=trt_output/resnet34.trt
&&&& RUNNING TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=resnet34.onnx --saveEngine=trt_output/resnet34.trt
[01/26/2024-22:16:49] [I] === Model Options ===
[01/26/2024-22:16:49] [I] Format: ONNX
[01/26/2024-22:16:49] [I] Model: resnet34.onnx
[01/26/2024-22:16:49] [I] Output:
[01/26/2024-22:16:49] [I] === Build Options ===
[01/26/2024-22:16:49] [I] Max batch: explicit batch
[01/26/2024-22:16:49] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[01/26/2024-22:16:49] [I] minTiming: 1
[01/26/2024-22:16:49] [I] avgTiming: 8
[01/26/2024-22:16:49] [I] Precision: FP32
[01/26/2024-22:16:49] [I] LayerPrecisions:
[01/26/2024-22:16:49] [I] Layer Device Types:
[01/26/2024-22:16:49] [I] Calibration:
[01/26/2024-22:16:49] [I] Refit: Disabled
[01/26/2024-22:16:49] [I] Version Compatible: Disabled
[01/26/2024-22:16:49] [I] TensorRT runtime: full
[01/26/2024-22:16:49] [I] Lean DLL Path:
[01/26/2024-22:16:49] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[01/26/2024-22:16:49] [I] Exclude Lean Runtime: Disabled
[01/26/2024-22:16:49] [I] Sparsity: Disabled
[01/26/2024-22:16:49] [I] Safe mode: Disabled
[01/26/2024-22:16:49] [I] Build DLA standalone loadable: Disabled
[01/26/2024-22:16:49] [I] Allow GPU fallback for DLA: Disabled
[01/26/2024-22:16:49] [I] DirectIO mode: Disabled
[01/26/2024-22:16:49] [I] Restricted mode: Disabled
[01/26/2024-22:16:49] [I] Skip inference: Disabled
[01/26/2024-22:16:49] [I] Save engine: trt_output/resnet34.trt
[01/26/2024-22:16:49] [I] Load engine:
[01/26/2024-22:16:49] [I] Profiling verbosity: 0
[01/26/2024-22:16:49] [I] Tactic sources: Using default tactic sources
[01/26/2024-22:16:49] [I] timingCacheMode: local
[01/26/2024-22:16:49] [I] timingCacheFile:
[01/26/2024-22:16:49] [I] Heuristic: Disabled
[01/26/2024-22:16:49] [I] Preview Features: Use default preview flags.
[01/26/2024-22:16:49] [I] MaxAuxStreams: -1
[01/26/2024-22:16:49] [I] BuilderOptimizationLevel: -1
[01/26/2024-22:16:49] [I] Input(s)s format: fp32:CHW
[01/26/2024-22:16:49] [I] Output(s)s format: fp32:CHW
[01/26/2024-22:16:49] [I] Input build shapes: model
[01/26/2024-22:16:49] [I] Input calibration shapes: model
[01/26/2024-22:16:49] [I] === System Options ===
......
[01/26/2024-22:17:07] [I] Average on 10 runs - GPU latency: 2.13367 ms - Host latency: 2.35083 ms (enqueue 0.329321 ms)
[01/26/2024-22:17:07] [I]
[01/26/2024-22:17:07] [I] === Performance summary ===
[01/26/2024-22:17:07] [I] Throughput: 408.164 qps
[01/26/2024-22:17:07] [I] Latency: min = 2.1969 ms, max = 11.5844 ms, mean = 2.34914 ms, median = 2.26282 ms, percentile(90%) = 2.5896 ms, percentile(95%) = 2.74451 ms, percentile(99%) = 3.15137 ms
[01/26/2024-22:17:07] [I] Enqueue Time: min = 0.214111 ms, max = 11.3787 ms, mean = 0.462134 ms, median = 0.360229 ms, percentile(90%) = 0.757202 ms, percentile(95%) = 0.912842 ms, percentile(99%) = 1.60339 ms
[01/26/2024-22:17:07] [I] H2D Latency: min = 0.20166 ms, max = 0.341309 ms, mean = 0.229284 ms, median = 0.223328 ms, percentile(90%) = 0.264771 ms, percentile(95%) = 0.274536 ms, percentile(99%) = 0.304443 ms
[01/26/2024-22:17:07] [I] GPU Compute Time: min = 1.9906 ms, max = 11.3264 ms, mean = 2.11385 ms, median = 2.0265 ms, percentile(90%) = 2.33765 ms, percentile(95%) = 2.4895 ms, percentile(99%) = 2.89893 ms
[01/26/2024-22:17:07] [I] D2H Latency: min = 0.00415039 ms, max = 0.0593262 ms, mean = 0.00600404 ms, median = 0.00463867 ms, percentile(90%) = 0.0119629 ms, percentile(95%) = 0.0145569 ms, percentile(99%) = 0.0292969 ms
[01/26/2024-22:17:07] [I] Total Host Walltime: 3.0037 s
[01/26/2024-22:17:07] [I] Total GPU Compute Time: 2.59158 s
[01/26/2024-22:17:07] [W] * GPU compute time is unstable, with coefficient of variance = 15.3755%.
[01/26/2024-22:17:07] [W]   If not already in use, locking GPU clock frequency or adding --useSpinWait may improve the stability.
[01/26/2024-22:17:07] [I] Explanations of the performance metrics are printed in the verbose logs.
[01/26/2024-22:17:07] [I]
&&&& PASSED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=resnet34.onnx --saveEngine=trt_output/resnet34.trt

执行结果生成的trt模型如下图10所示:
在这里插入图片描述 图 10 图10 10

2.2.4 测试生成的tensorrt模型

这个测试demo是参考链接2写的,在样例中对比ONNX和TensorRT的输出结果:

import numpy as np
import tensorrt as trt
import onnxruntime
import pycuda.driver as cuda
import pycuda.autoinitdef normalize(image: np.ndarray) -> np.ndarray:"""Normalize the image to the given mean and standard deviation"""image = image.astype(np.float32)mean = (0.485, 0.456, 0.406)std = (0.229, 0.224, 0.225)image /= 255.0image -= meanimage /= stdreturn imagedef onnx_inference(onnx_path: str, image: np.ndarray):# load onnx modelort_session = onnxruntime.InferenceSession(onnx_path)# compute onnx Runtime output predictionort_inputs = {ort_session.get_inputs()[0].name: image}res_onnx = ort_session.run(None, ort_inputs)[0]return res_onnxdef trt_inference(trt_path: str, image: np.ndarray):# Load the network in Inference Enginetrt_logger = trt.Logger(trt.Logger.WARNING)with open(trt_path, "rb") as f, trt.Runtime(trt_logger) as runtime:engine = runtime.deserialize_cuda_engine(f.read())with engine.create_execution_context() as context:# Set input shape based on image dimensions for inferencecontext.set_binding_shape(engine.get_binding_index("input"), (1, 3, image.shape[-2], image.shape[-1]))# Allocate host and device buffersbindings = []for binding in engine:binding_idx = engine.get_binding_index(binding)size = trt.volume(context.get_binding_shape(binding_idx))dtype = trt.nptype(engine.get_binding_dtype(binding))if engine.binding_is_input(binding):input_buffer = np.ascontiguousarray(image)input_memory = cuda.mem_alloc(image.nbytes)bindings.append(int(input_memory))else:output_buffer = cuda.pagelocked_empty(size, dtype)output_memory = cuda.mem_alloc(output_buffer.nbytes)bindings.append(int(output_memory))stream = cuda.Stream()# Transfer input data to the GPU.cuda.memcpy_htod_async(input_memory, input_buffer, stream)# Run inferencecontext.execute_async_v2(bindings=bindings, stream_handle=stream.handle)# Transfer prediction output from the GPU.cuda.memcpy_dtoh_async(output_buffer, output_memory, stream)# Synchronize the streamstream.synchronize()res_trt = np.reshape(output_buffer, (1, -1))return res_trtdef main():image_h = 224image_w = 224onnx_path = "resnet34.onnx"trt_path = "trt_output/resnet34.trt"image = np.random.randn(image_h, image_w, 3)normalized_image = normalize(image)# Convert the resized images to network input shape# [h, w, c] -> [c, h, w] -> [1, c, h, w]normalized_image = np.expand_dims(np.transpose(normalized_image, (2, 0, 1)), 0)onnx_res = onnx_inference(onnx_path, normalized_image)ir_res = trt_inference(trt_path, normalized_image)np.testing.assert_allclose(onnx_res, ir_res, rtol=1e-03, atol=1e-05)print("Exported model has been tested with TensorRT Runtime, and the result looks good!")if __name__ == '__main__':main()

执行结果如下:

(tensorrt) C:\Users\Administrator\Desktop\resnet>C:/ProgramData/anaconda3/envs/tensorrt/python.exe c:/Users/Administrator/Desktop/resnet/demo.py
[01/26/2024-22:19:08] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
c:\Users\Administrator\Desktop\resnet\demo.py:39: DeprecationWarning: Use get_tensor_name instead.context.set_binding_shape(engine.get_binding_index("input"), (1, 3, image.shape[-2], image.shape[-1]))
c:\Users\Administrator\Desktop\resnet\demo.py:39: DeprecationWarning: Use set_input_shape instead.context.set_binding_shape(engine.get_binding_index("input"), (1, 3, image.shape[-2], image.shape[-1]))
c:\Users\Administrator\Desktop\resnet\demo.py:43: DeprecationWarning: Use get_tensor_name instead.binding_idx = engine.get_binding_index(binding)
c:\Users\Administrator\Desktop\resnet\demo.py:44: DeprecationWarning: Use get_tensor_shape instead.size = trt.volume(context.get_binding_shape(binding_idx))
c:\Users\Administrator\Desktop\resnet\demo.py:45: DeprecationWarning: Use get_tensor_dtype instead.dtype = trt.nptype(engine.get_binding_dtype(binding))
c:\Users\Administrator\Desktop\resnet\demo.py:46: DeprecationWarning: Use get_tensor_mode instead.if engine.binding_is_input(binding):
Exported model has been tested with TensorRT Runtime, and the result looks good!

这个结果已经是成功了,只不过里面还有一些瑕疵,如结果中提到的CUDA lazy loading is not enabled的问题,可以使用在win 11的系统变量中添加变量名为CUDA_MODULE_LOADING和变量值为LAZY,如下图11所示:
在这里插入图片描述 图 11 图11 11添加之后,再次执行代码,得到结果,可以看出,上面的CUDA lazy loading is not enabled的问题已经解决了,剩下的问题原因就是tensorrt的版本为最新的8.6.1,这个测试demo是来源于参考链接2,使用的tensorrt版本为8.2.5,8.2.5中的很多API接口在8.6.1中更新了,都已经需要被8.6.1中的代替,这个就不解决了。

(tensorrt) C:\Users\Administrator\Desktop\resnet>C:/ProgramData/anaconda3/envs/tensorrt/python.exe c:/Users/Administrator/Desktop/resnet/demo.py
c:\Users\Administrator\Desktop\resnet\demo.py:39: DeprecationWarning: Use get_tensor_name instead.context.set_binding_shape(engine.get_binding_index("input"), (1, 3, image.shape[-2], image.shape[-1]))
c:\Users\Administrator\Desktop\resnet\demo.py:39: DeprecationWarning: Use set_input_shape instead.context.set_binding_shape(engine.get_binding_index("input"), (1, 3, image.shape[-2], image.shape[-1]))
c:\Users\Administrator\Desktop\resnet\demo.py:43: DeprecationWarning: Use get_tensor_name instead.binding_idx = engine.get_binding_index(binding)
c:\Users\Administrator\Desktop\resnet\demo.py:44: DeprecationWarning: Use get_tensor_shape instead.size = trt.volume(context.get_binding_shape(binding_idx))
c:\Users\Administrator\Desktop\resnet\demo.py:45: DeprecationWarning: Use get_tensor_dtype instead.dtype = trt.nptype(engine.get_binding_dtype(binding))
c:\Users\Administrator\Desktop\resnet\demo.py:46: DeprecationWarning: Use get_tensor_mode instead.if engine.binding_is_input(binding):
Exported model has been tested with TensorRT Runtime, and the result looks good!

3 容易出现的问题

3.1 cuda 和 tensorrt 版本不匹配的问题

这个问题可能会导致在执行2.2测试的时候出现退出程序的问题,在断点调试的时候,应该会出现下面的问题,尽可能的让cuda和tensorrt的版本一致

trt.volume(context.get_binding_shape(binding_idx))
[WinError 10054] 远程主机强迫关闭了一个现有的连接。

3.2 出现编译和加载时不是同一个cuda cuBLAS/cuBLAS

即是下面的问题

[TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1

最终要的解决方法前面已经提到过,就是anconda创建的虚拟环境tensorrt一定要使用pip安装pytorch,如果使用conda的方法安装pytorch,tensorrt会自动安装cudatoolkit,这可能会导致与win 11环境中安装的cuda版本不一致,导致出现上面的问题。
当然,如果不是上一段话引起的问题,使用pip的方法解决不了的话,可以阅读这一篇文章和这一篇文章。

4 参考链接

  1. TensorRT(一)Windows+Anaconda配置TensorRT环境 (Python版 )
  2. TensorRT安装记录(8.2.5)
  3. TensorRT的支持的cuda、cudnn等环境版本

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/649070.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

【Python】01快速上手爬虫案例一:搞定豆瓣读书

文章目录 前言一、VSCodePython环境搭建二、爬虫案例一1、爬取第一页数据2、爬取所有页数据3、格式化html数据4、导出excel文件 前言 实战是最好的老师,直接案例操作,快速上手。 案例一,爬取数据,最终效果图: 一、VS…

docker生命周期管理命令

文章目录 前言1、docker create2、docker run2.1、常用选项2.2、系统2.3、网络2.4、健康检查 3、docker start/stop/restart4、docker kill5、docker rm6、docker pause/unpause总结 前言 在云原生时代,Docker已成为必不可少的容器管理工具。通过掌握Docker常用的容…

uniCloud发行部署H5进行网页托管

生成文件,生成文件这个和我们平时用uniapp 生成H5的时候是一样的,我们可以选择hash 或者history 模式,默认的这是显示的根目录,如果我们在根目录下建立了H5目录,那么我们在发布H5的时候,是需要在manifest.j…

数据恢复与硬盘修理

目录 第1章 基础知识 1.1 数据恢复技术的发展和研究现状 1.2 数据恢复技术的层次与体系 1.网络层 2.网络存储层 DAS NAS 3.磁盘阵列层 4.磁盘层 5.文件系统层 6.文件层 7.覆盖恢复…

越活越“老“越值钱——来喝了这碗孟婆汤吧

目录 一、背景介绍二、思路&方案三、过程1.开头先从小编的行业说起2.扩展到各行各业,从小编的视角和认知的对比3.纵观人类发展和科技发展的对比4.我身边就有越活越"老"越值钱的人5.如何做到这一点的基本逻辑 四、总结 一、背景介绍 对于自己的未来&a…

Java强训day4(选择题编程题)

选择题 接口中的方法是为了让重写编程题 题目 import java.util.Scanner;public class Main {public static void main(String[] args) {Scanner sc new Scanner(System.in);int a_b sc.nextInt();int b_c sc.nextInt();int ab sc.nextInt();int bc sc.nextInt();for(in…

mysql索引失效

索引失效 1.违反最左前缀法则 查询要从索引的最左前列开始,不能跳过索引中的列 没有违反 违反 部分违反,只有最左边的命中索引 2.查询右边范围的列,不能使用索引 范维列不满足最左前缀 3.索引列上进行运算,索引会失效 改变…

假期刷题打卡--Day15

1、MT1152韩信又生气了 韩信点兵(大于10人),三个三个一排少1个人,五个五个一排又少1个人,七个七个一排还少1个人。韩信生气了,从别的队伍里调来一个人!这样不管是三个一排五个一排还是七个一排都完美了。问原本最少应该有多少人。…

每日一练 | 华为认证真题练习Day172

1、关于OSPF的ASBR-SUMMARY-LSA中LSA头部他、信息描述错误的是 A. LINK STATE ID表示ASBR的ROUTER ID B. ADVERTISING ROUTER表示该ABR的ROUTER ID C. ADVERTISING ROUTER字段永远不会改变 D. METRIC表示该ABR到达ASBR的OSPF开销 2、关于OSPF外部路由种类描述错误的是 A. …

神经网络进一步学习

一、代价函数 首先引入一些便于稍后讨论的新标记方法:假设神经网络的训练样本有m个,每个包含一组输入x和一组输出y,L表示神经网络层数,表示该层的神经元个数,将神经网络的分类定义为两种情况:二类分类和多类…

上位机图像处理和嵌入式模块部署(极致成本下的图像处理)

【 声明:版权所有,欢迎转载,请勿用于商业用途。 联系信箱:feixiaoxing 163.com】 目前,大家都习惯了特定的图像处理方式,要么是windows上位机来处理,要么是arm soc来进行处理,要么是…

亚信安慧AntDB:赋能业务核心 助力数据管理

在当今数据驱动的信息时代,数据库面临着前所未有的挑战与机遇。随着技术的不断进步和用户需求的多样化,传统的数据库解决方案已经难以满足运营商核心业务的复杂需求。亚信安慧AntDB数据库,作为一款强大数据库产品,旨在帮助运营商应…

[SwiftUI]Text对字符串中部分字符改变颜色和字体

如图,需要对字符串中部分字符改变颜色和字体。 在 SwiftUI 中合并带有不同样式的文本,应该使用不同的 Text 实例并将它们合并起来。将实例使用 运算符合并起来,每个 Text 实例都保持其自己的样式设置。这种方式可以正常编译并运行&#xff0…

蓝桥杯——每日一练(简单题)

题目 有n个整数,使前面各数顺序向后移m个位置,最后m个数变成前面m个数。写一函数:实现以上功能,在主函数中输入n个数和输出调整后的n个数。 解析 一、list()函数配合map()函数获得…

ps---1.使用置入嵌入的智能化对象命令制作拼贴画

ps—使用置入嵌入的智能化对象命令制作拼贴画 素材 步骤 打开背景图 置入嵌入对象(置入完,移动到正确的位置,回车或双击) 栅格化图层 重复上述步骤制作拼贴画

第二百八十八回

文章目录 1. 概念介绍2. 使用方法2.1 实现步骤2.2 具体细节 3. 示例代码4. 内容总结 我们在上一章回中介绍了"如何获取文件类型"相关的内容,本章回中将介绍如何播放视频.闲话休提,让我们一起Talk Flutter吧。 1. 概念介绍 播放视频是我们常用…

go语言(二十)---- 有缓冲的channel

有缓冲的channel 举个例子 package mainimport ("fmt""time" )func main() {c : make(chan int,3) //带有缓冲的 channelfmt.Println("len(c) ",len(c),",cap(c)",cap(c))go func() {defer fmt.Println("子go程结束")for i…

视频监控方案设计:EasyCVR视频智能监管系统方案技术特点与应用

随着科技的发展,视频监控平台在各个领域的应用越来越广泛。然而,当前的视频监控平台仍存在一些问题,如视频质量不高、监控范围有限、智能化程度不够等。这些问题不仅影响了监控效果,也制约了视频监控平台的发展。 为了解决这些问…

HTML-框架标签、实体、全局属性和元信息

HTML 1.框架标签 <iframe name"b站" src"https://www.bilibili.com" width"500" height"300" frameborder"0"></iframe>iframe 标签的实际应用&#xff1a; 在网页中嵌入广告。与超链接或表单的 target 配合&a…

QSqlQuery 执行Update 判断执行成功与否

1.执行更新操作的SQL语句 update s_info set name"009" where contact_number "13511112222" 怎么样判断是否确实更新操作是执行成功的 &#xff0c;可以通过下列语句判断 query.numRowsAffected() > 0 2.主要的几步操作如下: QSqlQuery query;query.…