深度学习图像视觉 RKNN Toolkit2 部署 RK3588S边缘端过程全记录

认识RKNN Toolkit2 工程文件
- 学习路线：
Anaconda Miniconda安装
- .condarc 文件
- 配置镜像源
- 自定义conda虚拟环境路径
- 创建Conda虚拟环境
本地训练环境
本地转换环境
- 安装 RKNN-Toolkit2：
- 添加 linaro交叉编译工具链
- 安装 Cmake
- rknn_model_zoo 文件
- - 1. 使用 rknn_model_zoo 转换模型 model.rknn
  - 以转换yolov8n.onnx 为例：
  - - 测试：
  - 2. 编译模型 model.rknn 生成 install 文件
  - 以编译 yolov8 生成 install 为例
  - 全过程记录转换编译模型
- rknpu2 文件
- - 1. 以Aarch64 Linux Demo 编译构建
  - 以编译 Yolo-v5 demo 生成 install 为例：
  - 全过程记录转换编译模型
Aarch64设备部署
- rknn_model_zoo examples 文件
- - 使用 rknn_model_zoo 编译构建输出文件（install）推送到开发板
  - 终端设备执行操作全记录
- rknpu2 examples 文件
- - 使用 rknpu2 编译构建输出文件（install）推送到开发板
  - - 视频演示指南：
  - 终端设备执行操作全记录
aarch64设备部署 C API 推理

------ AArch64是由ARM公司为其ARMv8-A 64位指令集架构推出的一种新的编程模型，它是ARMv8-A架构的一个子集。这种编程模型主要是为了在64位模式下提高程序性能和安全性。与ARM32（即ARMv7-A）的指令集不同，AArch64在寄存器和指令方面有许多改进和增强。
------ 因此，ARM64和AArch64都是指同一个东西，只是名称不同，来源也不同。如果你在不同的地方看到这两个术语，不要感到困惑，它们都是指代同一个处理器架构。
作者：量子孤岛
来源：知乎

认识RKNN Toolkit2 工程文件

在这里插入图片描述

RKNN-Toolkit, RKNN Toolkit Lite, 和 RKNPU2 都与Rockchip芯片上的神经网络推理相关，但它们在用途和应用场景上有所不同：

RKNN-Toolkit:
这是一个用于PC平台的开发套件，主要用于在Rockchip芯片上进行NPU（神经处理单元）模型的转换、优化和性能评估。
它提供了全面的功能，包括模型导入（如ONNX、TensorFlow、Torchscript等），模型转换为RKNN格式，以及在目标硬件上进行推理的接口。
RKNN-Toolkit通常用于开发环境，允许开发者在强大的PC上进行模型的预处理和调试工作。
RKNN Toolkit Lite:
这是轻量级版本的RKNN工具，设计用于在Rockchip的嵌入式设备或开发板上直接运行。
它可能包含了基本的模型加载和推理功能，适合在资源有限的环境中使用。
RKNN Toolkit Lite可能不包含所有PC版本的高级特性，例如详细的性能分析或模型转换工具，但它更专注于在实际硬件上的高效运行。
RKNPU2:
RKNPU2似乎是指Rockchip NPU的第二代软件栈或SDK，它提供了与NPU硬件交互的底层接口。
这个SDK通常包含驱动程序、库文件和API，使得开发者能够直接控制NPU进行神经网络计算。
RKNPU2可能被用于实现更底层的性能优化或者在没有完整RKNN-Toolkit的情况下进行定制化开发。

RKNPU2 SDK
提供了C语言编程接口，专门设计用于带有Rockchip神经处理单元（NPU）的芯片平台。这个接口允许开发者直接在目标设备上编写C代码来集成和执行已经通过RKNN-Toolkit2转换和优化的RKNN模型。以下是一些关键点：

模型转换：
使用RKNN-Toolkit2，开发者可以将常见的深度学习框架（如TensorFlow, PyTorch等）的模型转换为针对RockchipNPU优化的RKNN格式。
C语言接口：
RKNPU2 SDK 提供的C API使得开发者能够在应用程序中加载、初始化和运行这些RKNN模型。这些API通常包括模型加载函数、推理接口、数据预处理和后处理函数等。
设备交互：
通过这些接口，开发者可以直接控制NPU进行高效的硬件加速计算，充分利用NPU的并行处理能力。API会处理与硬件的低级别通信，包括内存管理和指令调度。
资源管理：
开发者需要管理模型的内存分配和释放，以及在运行时管理输入和输出数据缓冲区。
性能优化：
RKNPU2可能还包括一些工具和指导，帮助开发者进行性能调优，比如批量处理、多线程支持等。
部署和测试：
一旦模型在目标设备上编译和链接，开发者可以通过编写C程序来实现模型的部署，并进行实际的推理任务。
错误处理和调试：
C语言接口也会提供错误处理机制，以便在遇到问题时能够捕获和诊断错误。通过这种方式，RKNPU2 SDK 和 C语言接口为开发者提供了一种灵活和高效的方法，将AI模型集成到嵌入式系统中，尤其适用于需要实时推理和低功耗要求的应用场景。

在这里插入图片描述

总结来说，RKNN-Toolkit是全面的开发和调试工具，适合在PC上进行模型准备；RKNN Toolkit Lite是简化版，适用于嵌入式设备上的推理；而RKNPU2是NPU的软件开发包，提供直接访问硬件的能力。根据开发需求和目标平台的不同，开发者会选择适合的工具。

学习路线：

在这里插入图片描述

Anaconda Miniconda安装

清华大学开源软件镜像站 Miniconda下载
Anaconda 镜像使用帮助
linux安装软件：安装过程中根据提示输入enter或yes

bash Miniconda3-py312_24.3.0-0-Linux-x86_64.sh

安装完后，通过conda命令进行使用。

.condarc 文件

Windows 用户无法直接创建名为 .condarc 的文件，可先执行

conda config --set show_channel_urls yes

配置镜像源

通过修改文件添加（推荐）
直接修改.condarc文件是最方便的。
找到系统用户下的 .condarc 的文件，记事本打开并添加镜像源。

channels:- defaults
show_channel_urls: true
default_channels:- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmsys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudbioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmenpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudsimpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/clouddeepmodeling: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/

自定义conda虚拟环境路径

找到系统用户下的 .condarc 的文件，记事本打开并添加路径，换成自己要保存的位置，建议放在非C盘中。

##windows
envs_dirs:- E://Miniconda3//envs

##linux
envs_dirs:- /home/wlj/.conda/envs/

创建Conda虚拟环境

# 列举所有环境
conda env list
# Python创建虚拟训练环境
conda create -n RKNN_yolov8_py3.10 python==3.10
conda activate RKNN_yolov8_py3.10 # Python创建虚拟转换环境
conda create -n RKNN_Toolkit2_py3.10 python==3.10
conda activate RKNN_Toolkit2_py3.10 # 退出虚拟环境
conda deactivate

本地训练环境

在这里插入图片描述

本地转换环境

在这里插入图片描述

安装 RKNN-Toolkit2：

注意事项：

使用大于或等于 1.4.0 的 rknn-toolkit2 版本。
使用自己训练的模型时，请对齐anchor等后处理参数，否则会导致后处理分析误差。
demo需要librga.so的支持,编译使用请参考 https://github.com/airockchip/librga 。

# 从官方RKNN-Toolkit2仓库拉取最新版本(toolkit2版本是/v2.0.0-beta0)
git clone https://github.com/airockchip/rknn-toolkit2/tree/v2.0.0-beta0# 配置pip源
pip3 config set global.index-url https://mirror.baidu.com/pypi/simple# 安装依赖库，根据rknn-toolkit2/packages/requirements_cp310-2.0.0b0.txt
pip3 install -r requirements_cp310-2.0.0b0.txt# 安装rknn_toolkit2 | rknn-toolkit2/packages/rknn_toolkit2-2.0.0b0+9bab5682-cp310-cp310-linux_x86_64.whl
# 根据系统的python版本选择不同的whl文件安装：
pip3 install rknn_toolkit2-2.0.0b0+9bab5682-cp310-cp310-linux_x86_64.whl

检测是否安装成功：

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:~$ python3
Python 3.10.14 (main, May  6 2024, 19:42:50) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from rknn.api import RKNN
>>>

添加 linaro交叉编译工具链

环境：ubnutu20.04.06
使用linaro交叉编译工具链下载
博客

解压交叉编译工具链
解压gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu

tar -xzvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz
tar -xvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar

sudo gedit ~/.bashrc 或 sudo vim ~/.bashrc
export GCC_COMPILER=/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu
source ~/.bashrc

安装 Cmake

第一种：指令安装

sudo apt install cmake

第二种：解压tar （自定义）
Cmake下载地址

tar -xzvf cmake-3.29.0-linux-x86_64.tar.gz

#打开个人path配置
gedit ~/.bashrc 或 sudo vim ~/.bashrc
#在末尾添加如下的内容
export PATH=/home/ubuntu/cmake-3.29.0-linux-x86_64/bin:$PATH
#接着在终端source一下.bashrc文件让path立即生效
source ~/.bashrc
#安装完毕测试版本
cmake --version

rknn_model_zoo 文件

rknn_model_zoo
├── 3rdparty # 第三方库
├── datasets # 数据集
├── examples # 示例代码
├── utils # 常用方法，如文件操作，画图等
├── build-android.sh # 用于目标为 Android 系统开发板的编译脚本
├── build-linux.sh # 用于目标为 Linux 系统开发板的编译脚本
└── ...

1. 使用 rknn_model_zoo 转换模型 model.rknn

# 从官方rknn_model_zoo仓库拉取最新版本
git clone https://github.com/airockchip/rknn_model_zoo/tree/v2.0.0

Convert to RKNN
在这里插入图片描述

以转换yolov8n.onnx 为例：

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ tree
.
├── README.md
├── cpp
│   ├── CMakeLists.txt
│   ├── main.cc
│   ├── postprocess.cc
│   ├── postprocess.h
│   ├── rknpu1
│   │   └── yolov8.cc
│   ├── rknpu2
│   │   ├── yolov8.cc
│   │   └── yolov8_rv1106_1103.cc
│   └── yolov8.h
├── model
│   ├── bus.jpg
│   ├── coco_80_labels_list.txt
│   ├── dataset.txt
│   ├── download_model.sh
│   └── yolov8n.onnx
├── model_comparison
│   ├── yolov8_graph_comparison.jpg
│   └── yolov8_output_comparison.jpg
├── python
│   ├── convert.py
│   └── yolov8.py
└── result.png

首先导入 GCC_COMPILER ，例如 export GCC_COMPILER=/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu（如系统添加，无需此步），然后执行：

cd python
# 运行 convert.py 脚本，将原始的 ONNX 模型转成 RKNN 模型
# 用法: python convert.py model_path [rk3566|rk3588|rk3562] [i8/fp] [output_path]
# output model will be saved as ../model/yolov8.rknn
python convert.py ../model/yolov8n.onnx rk3588 i8 ../model/yolov8n.rknn

测试：

# 如果想先在计算机端运行原始的 onnx 模型，可以参考以下命令
# 用法: python yolov5.py --model_path {onnx_model} --img_show
python yolov8.py --model_path ../model/yolov8n.onnx --img_show

在这里插入图片描述

2. 编译模型 model.rknn 生成 install 文件

For Linux develop board:

首先导入 GCC_COMPILER ，例如 export GCC_COMPILER=/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu（如系统添加，无需此步），然后执行：

./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]-t : target (rk356x/rk3588/rk3576/rv1106/rk1808/rv1126)-a : arch (aarch64/armhf)-d : demo name-b : build_type(Debug/Release)-m : enable address sanitizer, build_type need set to Debug
Note: 'rk356x' represents rk3562/rk3566/rk3568, 'rv1106' represents rv1103/rv1106, 'rv1126' represents rv1109/rv1126# Here is an example for compiling yolov5 demo for 64-bit Linux RK3566.
./build-linux.sh -t rk356x -a aarch64 -d yolov5

For Android development board:

# For Android develop boards, it's require to set path for Android NDK compilation tool path according to the user environment
export ANDROID_NDK_PATH=~/opts/ndk/android-ndk-r18b
./build-android.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]-t : target (rk356x/rk3588/rk3576)-a : arch (arm64-v8a/armeabi-v7a)-d : demo name-b : build_type (Debug/Release)-m : enable address sanitizer, build_type need set to Debug# Here is an example for compiling yolov5 demo for 64-bit Android RK3566.
./build-android.sh -t rk356x -a arm64-v8a -d yolov5

以编译 yolov8 生成 install 为例

位置在rknn_model_zoo/examples/yolov8

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/rknn_model_zoo/examples/yolov8$ tree
.
├── README.md
├── cpp  # C/C++ 版本的示例代码
│   ├── CMakeLists.txt
│   ├── main.cc
│   ├── postprocess.cc
│   ├── postprocess.h
│   ├── rknpu1
│   │   └── yolov8.cc
│   ├── rknpu2
│   │   ├── yolov8.cc
│   │   └── yolov8_rv1106_1103.cc
│   └── yolov8.h
├── model   # 模型、测试图片等文件
│   ├── bus.jpg
│   ├── coco_80_labels_list.txt
│   ├── dataset.txt
│   ├── download_model.sh
│   ├── yolov8.rknn
│   └── yolov8n.onnx
├── model_comparison
│   ├── yolov8_graph_comparison.jpg
│   └── yolov8_output_comparison.jpg
├── python  # 模型转换脚本和 Python 版本的示例代码
│   ├── convert.py
│   └── yolov8.py
└── result.png

cd /rknn_model_zoo
./build-linux.sh -t rk3588 -a aarch64 -d yolov8

rknn_model_zoo目录下生成 install

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/rknn_model_zoo/install$ tree
.
└── rk3588_linux_aarch64└── rknn_yolov8_demo├── lib    # 依赖库│   ├── librga.so│   └── librknnrt.so├── model  # 存放模型、测试图片等文件│   ├── bus.jpg│   ├── coco_80_labels_list.txt│   └── yolov8.rknn└── rknn_yolov8_demo   # 可执行文件

全过程记录转换编译模型

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ ls
README.md  cpp  model  model_comparison  python  result.png
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd python/
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/python$ python convert.py ../model/yolov8n.onnx rk3588
I rknn-toolkit2 version: 2.0.0b0+9bab5682
--> Config model
done
--> Loading model
I It is recommended onnx opset 19, but your onnx model opset is 12!
I Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert!
I Loading : 100%|██████████████████████████████████████████████| 136/136 [00:00<00:00, 35716.32it/s]
done
--> Building model
W build: found outlier value, this may affect quantization accuracyconst name                        abs_mean    abs_std     outlier valuemodel.0.conv.weight               2.44        2.47        -17.494model.22.cv3.2.1.conv.weight      0.09        0.14        -10.215model.22.cv3.1.1.conv.weight      0.12        0.19        13.361, 13.317model.22.cv3.0.1.conv.weight      0.18        0.20        -11.216
I GraphPreparing : 100%|████████████████████████████████████████| 161/161 [00:00<00:00, 5348.48it/s]
I Quantizating : 100%|████████████████████████████████████████████| 161/161 [00:05<00:00, 27.53it/s]
W build: The default input dtype of 'images' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '318' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 'onnx::ReduceSum_326' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '331' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '338' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 'onnx::ReduceSum_346' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '350' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '357' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 'onnx::ReduceSum_365' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '369' is changed from 'float32' to 'int8' in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
I rknn building ...
I rknn buiding done.
done
--> Export rknn model
done
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/python$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd model
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/model$ ls
bus.jpg  coco_80_labels_list.txt  dataset.txt  download_model.sh  yolov8.rknn  yolov8n.onnx
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/model$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo$ ./build-linux.sh -t rk3588 -a aarch64 -d yolov8
./build-linux.sh -t rk3588 -a aarch64 -d yolov8
/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu
===================================
BUILD_DEMO_NAME=yolov8
BUILD_DEMO_PATH=examples/yolov8/cpp
TARGET_SOC=rk3588
TARGET_ARCH=aarch64
BUILD_TYPE=Release
ENABLE_ASAN=OFF
INSTALL_DIR=/mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo
BUILD_DIR=/mnt/d/Desktop/RK3588/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3588_linux_aarch64_Release
CC=/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
CXX=/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
===================================
-- !!!!!!!!!!!CMAKE_SYSTEM_NAME: Linux
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/d/Desktop/RK3588/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3588_linux_aarch64_Release
[ 20%] Built target fileutils
[ 40%] Built target imagedrawing
[ 60%] Built target imageutils
[100%] Built target rknn_yolov8_demo
[ 20%] Built target fileutils
[ 40%] Built target imageutils
[ 60%] Built target imagedrawing
[100%] Built target rknn_yolov8_demo
Install the project...
-- Install configuration: "Release"
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo
-- Set runtime path of "/mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo" to "$ORIGIN/../lib"
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/bus.jpg
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/coco_80_labels_list.txt
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/yolov8.rknn
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/lib/librknnrt.so
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/lib/librga.so
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo$

rknpu2 文件

注意事项：

使用大于或等于 1.4.0 的 rknn-toolkit2 版本。
使用自己训练的模型时，请对齐anchor等后处理参数，否则会导致后处理分析误差。
官网和rk预训练模型均检测80种目标。如果训练自己的模型，则需要更改 include/postprocess.h 中的OBJ_CLASS_NUM和NMS_THRESH，BOX_THRESH后处理参数。
demo需要librga.so的支持,编译使用请参考 https://github.com/airockchip/librga 。
由于硬件限制，该demo的模型默认把 yolov5 模型的后处理部分，移至cpu实现。本demo附带的模型均使用relu为激活函数，相比silu激活函数精度略微下降，性能大幅上升。

1. 以Aarch64 Linux Demo 编译构建

首先导入 GCC_COMPILER ，例如 export GCC_COMPILER=/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu（如系统添加，无需此步），然后执行：

#进入rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ls
CMakeLists.txt  README.md  README_CN.md  build-android.sh  build-linux.sh  convert_rknn_demo  include  model  src  utils

以编译 Yolo-v5 demo 生成 install 为例：

cat@lubancat:~/rknpu2/examples/rknn_yolov5_demo$ tree
.
├── CMakeLists.txt
├── README.md
├── README_CN.md
├── build-android.sh
├── build-linux.sh
├── convert_rknn_demo
│   └── yolov5
│       ├── README.md
│       ├── README_CN.md
│       ├── bus.jpg
│       ├── dataset.txt
│       ├── onnx2rknn.py
│       └── onnx_models
│           ├── yolov5s_for_apk_demo.onnx
│           └── yolov5s_relu.onnx
├── include
│   ├── drm_func.h
│   ├── postprocess.h
│   ├── preprocess.h
│   └── rga_func.h
├── model
│   ├── RK3562
│   │   └── yolov5s-640-640.rknn
│   ├── RK3566_RK3568
│   │   └── yolov5s-640-640.rknn
│   ├── RK3576
│   │   └── yolov5s-640-640.rknn
│   ├── RK3588
│   │   └── yolov5s-640-640.rknn
│   ├── bus.jpg
│   └── coco_80_labels_list.txt
├── src
│   ├── main.cc
│   ├── main_video.cc
│   ├── postprocess.cc
│   └── preprocess.cc
└── utils├── drawing.cpp├── drawing.h├── mpp_decoder.cpp├── mpp_decoder.h├── mpp_encoder.cpp└── mpp_encoder.h11 directories, 32 files

# such as: 
# ./build-linux.sh -t <target> -a <arch> -b <build_type>]
./build-linux.sh -t rk3588 -a aarch64 -b Release

在rknn_yolov5_demo目录下生成 install
在这里插入图片描述

全过程记录转换编译模型

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ls
CMakeLists.txt  README.md  README_CN.md  build-android.sh  build-linux.sh  convert_rknn_demo  include  model  src  utils
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ll
total 20
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 ./
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 ../
-rwxrwxrwx 1 ubuntu20 ubuntu20 3948 Mar 25 11:53 CMakeLists.txt*
-rwxrwxrwx 1 ubuntu20 ubuntu20 3561 Mar 25 11:53 README.md*
-rwxrwxrwx 1 ubuntu20 ubuntu20 3496 Mar 25 11:53 README_CN.md*
-rwxrwxrwx 1 ubuntu20 ubuntu20 2632 Mar 25 11:53 build-android.sh*
-rwxrwxrwx 1 ubuntu20 ubuntu20 2722 Mar 25 11:53 build-linux.sh*
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 convert_rknn_demo/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 include/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 model/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 src/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 utils/
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ./build-linux.sh -t rk3588 -a aarch64 -b Release
./build-linux.sh -t rk3588 -a aarch64 -b Release
/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu
===================================
TARGET_SOC=RK3588
TARGET_ARCH=aarch64
BUILD_TYPE=Release
BUILD_DIR=/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
CC=/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
CXX=/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
===================================
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
-- Check for working CXX compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/3rdparty/opencv/opencv-linux-aarch64 (found version "3.4.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
Scanning dependencies of target rknn_yolov5_demo
Scanning dependencies of target rknn_yolov5_video_demo
[ 10%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/main_video.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/postprocess.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_decoder.cpp.o
[ 40%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_encoder.cpp.o
[ 50%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/drawing.cpp.o
[ 60%] Linking CXX executable rknn_yolov5_video_demo
[ 60%] Built target rknn_yolov5_video_demo
[ 70%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/main.cc.o
[ 90%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/postprocess.cc.o
[ 90%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/preprocess.cc.o
[100%] Linking CXX executable rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 60%] Built target rknn_yolov5_video_demo
[100%] Built target rknn_yolov5_demo
Install the project...
-- Install configuration: "Release"
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_demo
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librknnrt.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librga.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3588
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3588/yolov5s-640-640.rknn
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/bus.jpg
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/coco_80_labels_list.txt
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_video_demo
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librockchip_mpp.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/libmk_api.so
(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$

Aarch64设备部署

rknn_model_zoo examples 文件

使用 rknn_model_zoo 编译构建输出文件（install）推送到开发板

在 rknn_model_zoo 生成 install 文件夹
在这里插入图片描述

(RKNN_Toolkit2_py3.10) ubuntu20@DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/install$ tree
.
└── rk3588_linux_aarch64└── rknn_yolov8_demo├── lib│   ├── librga.so│   └── librknnrt.so├── model│   ├── bus.jpg│   ├── coco_80_labels_list.txt│   └── yolov8.rknn└── rknn_yolov8_demo

拷贝 install 到 aarch64设备，再

cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ chmod 777 *
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ export LD_LIBRARY_PATH=./lib
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ./rknn_yolov8_demo model/yolov8.rknn model/bus.jpg

运行一张的速度太慢了，批量去运行写一个batch_process.sh文件

cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$  vim batch_process.sh

export LD_LIBRARY_PATH=./lib
#!/bin/bash
# 指定图片文件夹路径
IMAGE_DIR="./image_file"
# 遍历图片文件夹中的所有图片文件
for image_file in "$IMAGE_DIR"/*.jpg; doif [ -f "$image_file" ]; thenecho "Processing $image_file"./rknn_yolov8_demo ./model/yolov8.rknn "$image_file"fi
done

终端设备执行操作全记录

cat@lubancat:~$ ls
Desktop    Downloads  Pictures  Python-3.10.14  Videos                      librga                rknn_mobilenet_demo_Linux  rknn_model_zoo
Documents  Music      Public    Templates       cmake-3.29.0-linux-aarch64  rk3588_linux_aarch64  rknn_model
cat@lubancat:~$ cd rk3588_linux_aarch64/
cat@lubancat:~/rk3588_linux_aarch64$ ls
rknn_yolov8_demo
cat@lubancat:~/rk3588_linux_aarch64$ cd rknn_yolov8_demo/
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ls
lib  model  rknn_yolov8_demo
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ll
total 984
drwxrwxr-x 4 cat cat   4096 May 13 15:06 ./
drwxrwxrwx 3 cat cat   4096 May 13 15:06 ../
drwxrwxr-x 2 cat cat   4096 May 13 15:06 lib/
drwxrwxr-x 2 cat cat   4096 May 13 15:06 model/
-rw-rw-r-- 1 cat cat 991216 May 13 15:06 rknn_yolov8_demo
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ chmod 777 *
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ export LD_LIBRARY_PATH=./lib
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ./rknn_yolov8_demo model/yolov8.rknn model/bus.jpg
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 9
input tensors:index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:index=0, name=318, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-58, scale=0.117659index=1, name=onnx::ReduceSum_326, n_dims=4, dims=[1, 80, 80, 80], n_elems=512000, size=512000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003104index=2, name=331, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003173index=3, name=338, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-45, scale=0.093747index=4, name=onnx::ReduceSum_346, n_dims=4, dims=[1, 80, 40, 40], n_elems=128000, size=128000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003594index=5, name=350, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003627index=6, name=357, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-34, scale=0.083036index=7, name=onnx::ReduceSum_365, n_dims=4, dims=[1, 80, 20, 20], n_elems=32000, size=32000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003874index=8, name=369, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=640x640 crop size=640x640
input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=1.000000 dst_box=(0 0 639 639) allow_slight_change=1 _left_offset=0 _top_offset=0 padding_w=0 padding_h=0
src width=640 height=640 fmt=0x1 virAddr=0x0x1d996ea0 fd=0
dst width=640 height=640 fmt=0x1 virAddr=0x0x1dac2eb0 fd=0
src_box=(0 0 639 639)
dst_box=(0 0 639 639)
color=0x72
rga_api version 1.10.1_[0]
rknn_run
person @ (211 241 282 506) 0.864
bus @ (96 136 549 449) 0.864
person @ (109 235 225 535) 0.860
person @ (477 226 560 522) 0.848
person @ (79 327 116 513) 0.306
write_image path: out.png width=640 height=640 channel=3 data=0x1d996ea0
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ls
lib  model  out.png  rknn_yolov8_demo
cat@lubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$

在这里插入图片描述

rknpu2 examples 文件

使用 rknpu2 编译构建输出文件（install）推送到开发板

在rknpu2 / examples / rknn_yolov5_demo目录下生成 install
在这里插入图片描述
推送到aarch64设备开发板 install / rknn_yolov5_demo_Linux

export LD_LIBRARY_PATH=./lib
#./rknn_yolov5_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn model/bus.jpg
chmod 777 *
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg

注意：如果在 lib 文件夹中找不到 librga.so，请尝试搜索 librga.so 的位置并将其添加到LD_LIBRARY_PATH。使用以下命令添加到LD_LIBRARY_PATH。

export LD_LIBRARY_PATH=./lib:<LOCATION_LIBRGA.SO>

GITHUB的airockchip/librga地址

效果：
在这里插入图片描述

视频演示指南：

添加 test.mp4 视频测试
在这里插入图片描述

H264 H264型

通过 ffmpeg 转换为 h264 ：

#ffmpeg -i xxx.mp4 -vcodec h264 xxx.h264
ffmpeg -i model/test.mp4 -vcodec h264 model/test.h264

#./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn xxx.h264 264
./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/test.h264 264

通过 ffmpeg 转换为 h265 ：

#ffmpeg -i xxx.mp4 -vcodec hevc xxx.hevc
ffmpeg -i model/test.mp4 -vcodec hevc model/test.hevc

H265

#./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn xxx.hevc 265./rknn_yolov5_video_demo  model/RK3588/yolov5s-640-640.rknn model/test.hevc 265

RTSP

#./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn <RTSP_URL> 265./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn <RTSP_URL> 265

效果：
在这里插入图片描述

终端设备执行操作全记录

cat@lubancat:~$ ls
Desktop    Music     Python-3.10.14  cmake-3.29.0-linux-aarch64  rknn_yolov5_demo_Linux
Documents  Pictures  Templates       librga
Downloads  Public    Videos          rk3588_linux_aarch64
cat@lubancat:~$ cd rknn_yolov5_demo_Linux/
cat@lubancat:~/rknn_yolov5_demo_Linux$ ls
lib  model  out.h264  out.jpg  rknn_yolov5_demo  rknn_yolov5_video_demo
cat@lubancat:~/rknn_yolov5_demo_Linux$ rm -rf out.h264
cat@lubancat:~/rknn_yolov5_demo_Linux$ ls
lib  model  out.jpg  rknn_yolov5_demo  rknn_yolov5_video_demo
cat@lubancat:~/rknn_yolov5_demo_Linux$ ./rknn_yolov5_video_demo  model/RK3588/yolov5s-640-640.rknn model/test.hevc 265
Loading mode...
省略....time_gap=-10found last frame reset decoder
waiting finish
cat@lubancat:~/rknn_yolov5_demo_Linux$ ls
lib  model  out.h264  out.jpg  rknn_yolov5_demo  rknn_yolov5_video_demo