TensorRT accelerated application ⭐

tensorrtx Download

TensorRT is a C + + version of deep learning architecture, which is parallel to tensorflow and pytorch. We want to deploy yolov5 to the development board with TensorRT, which is divided into two steps: 1 Install TensorRT, 2 Rewrite the network trained by each training framework with TensorRT syntax. But it's really troublesome to write the network by ourselves with TensorRT, and we won't. Then we can use tensorrtx, the great God, to reproduce various classic networks in TensorRT syntax:

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-d45qqq1jc-164232526242) (tensorrt accelerated application. assets/image-20220115200209423.png)]

There are so many networks that one screenshot can't fit. The basic mainstream networks have been able to transform

It is equivalent to helping us complete step 2, which is very important. It can transform the model files of each framework training into files supported by tensorrtx.

But he put forward requirements for our environment:

• Requirement 1: tensorrtx cuda10 0 / cudnn7. 6.5 / TensorRT7. zero

• Requirement 2: cuda10 0 requires the graphics card driver to be higher than 410.48

• Requirements for driver of graphics card:

  use ubuntu-drivers devices To view the supported graphics card drivers,
  If you find that none of the supported drivers exceed 410.48，Then this road won't work.
  If supported>= 410.48 But you don't use it now,
  Then it can be used sudo apt install nvidia-driver-xxx To install
  

• CUDA10. Requirements for 0(Other versions of CUDA are required here ): requirements for ubuntu version, system kernel, gcc, glibc,

That is, we should have:

• For the graphics card driver, go to the official website to install it, or click the command to install it
• >CUDA10. 0, you can refer to Deploy cuda and cudnn
• >cuDNN7. 6.5, refer to Deploy cuda and cudnn
• >TensorRT7. 0.0.11, refer to Deploy tensorrt
• >opencv3. 4.15, refer to Deploy opencv

Code download:

Two codes are required:

• yolov5 official source code, and configure the environment (preferably conda virtual environment)

  cd ~
  git clone https://github.com/ultralytics/yolov5
  cd yolov5
  pip install -r requirements.txt
  

  python detect.py
  

  After running, it can automatically help you download yolov5s PT needs to be connected to the Internet. If you are not sure, you can also go directly to github download , this pt file is very important and will be used later to accelerate the application

• tensorrtx source code

  cd ~
  git clone https://github.com/wang-xinyu/tensorrtx
  

get. wts file

The yolov5s will be downloaded Copy the PT file to the tensorrtx/yolov5 directory and run gen_wts can be obtained wts file, where you can share the configuration environment of yolov5

python gen_wts.py yolov5s.pt

After running, you will get yolov5s wts

Compile and run to get engine file

Before compiling and running, review whether TensorRT is installed with deb package or tar package. If it is installed from tar package, this step must not be omitted, otherwise the compilation cannot be completed

Edit cmakelists. In the tensorrtx/yolov5 directory txt:

# Replace these two lines
include_directories(/usr/include/x86_64-linux-gnu/)
link_directories(/usr/lib/x86_64-linux-gnu/)
 
include_directories(/home/{user_name}/TensorRT-{version}/include/)
link_directories(/home/{user_name}/TensorRT-{version}/targets/x86_64-linux-gnu/lib/)
# Save exit

{user_name}: user name

{version}: TensorRT version

cd ~/tensorrtx/yolov5
mkdir build
cd build
cp ../yolov5s.wts ./
cmake ..
make

# -The s command is used to generate engine file, the three parameters are wts file engine file and model identification [s/m/l/x /...]
./yolov5 -s yolov5s.wts yolov5s.engine s 
# -The d command is used to perform reasoning. The two parameters are engine and the directory to be tested
./yolov5 -d yolov5s.engine ../samples

If there is no error, you can see the framed Zidane in the build folder Jpg

The two parameters are engine and the directory to be tested
./yolov5 -d yolov5s.engine .../samples

If no error is reported during operation, you can build See framed in the folder zidane.jpg Yes