Randall Zhuo
2 years ago
101 changed files with 2786 additions and 474 deletions
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
|
||||
1、目前RV1106/RV1103仅支持rknn_mobilenet_demo 和 rknn_yolov5_demo两个demo，在examples/RV1106_RV1103目录下，demo中的**build脚本用RV1106指代RV1106/RV1103** |
||||
|
||||
2、RK356X和RK3588支持example目录下，除examples/RV1106_RV1103文件夹外的所有demo |
||||
|
||||
3、**RV1106/RV1103设置LD_LIBRARY_PATH必须为全路径**，例如：export LD_LIBRARY_PATH=/userdata/lib |
||||
|
||||
4、RK356X和RK3588设置LD_LIBRARY_PATH为全路径和相对路径均可 |
||||
@ -0,0 +1,34 @@
@@ -0,0 +1,34 @@
|
||||
下述<TARGET_PLATFORM>是RV1106或RV1103 |
||||
|
||||
# Arm Linux Demo |
||||
|
||||
## 编译 |
||||
|
||||
修改目标平台的`build-linux_<TARGET_PLATFORM>.sh`上的`GCC_COMPILER`， |
||||
|
||||
然后执行 |
||||
|
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
连接设备并将构建输出推送到“/userdata” |
||||
|
||||
``` |
||||
adb push install/rknn_mobilenet_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_mobilenet_demo_Linux/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_mobilenet_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
|
||||
@ -0,0 +1,46 @@
@@ -0,0 +1,46 @@
|
||||
# Yolo-v5 demo |
||||
|
||||
# 导出rknn模型 |
||||
|
||||
请参考 https://github.com/airockchip/rknn_model_zoo/tree/main/models/CV/object_detection/yolo |
||||
|
||||
|
||||
|
||||
## Arm Linux Demo |
||||
|
||||
### 编译 |
||||
|
||||
RV1106/RV1103编译脚本均为 `build-linux_RV1106.sh`，设置交叉编译器所在目录的路径 `RK_RV1106_TOOLCHAIN`，例如修改成 |
||||
|
||||
```sh |
||||
export RK_RV1106_TOOLCHAIN=~/opts/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf |
||||
``` |
||||
|
||||
然后执行： |
||||
|
||||
```sh |
||||
./build-linux_RV1106.sh |
||||
``` |
||||
|
||||
### 推送执行文件到板子 |
||||
|
||||
连接板子的usb口到PC,将整个demo目录到 `/userdata`: |
||||
|
||||
```sh |
||||
adb push install/rknn_yolov5_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
### 运行 |
||||
|
||||
```sh |
||||
adb shell |
||||
cd /userdata/rknn_yolov5_demo_Linux/ |
||||
|
||||
export LD_LIBRARY_PATH=/userdata/rknn_yolov5_demo_Linux/lib |
||||
./rknn_yolov5_demo model/RV1106/yolov5s-640-640.rknn model/bus.jpg |
||||
``` |
||||
|
||||
Note: |
||||
|
||||
- LD_LIBRARY_PATH 必须采用全路径 |
||||
- 基于性能原因，demo中将 RKNN 模型的输出 fmt 设置为 RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR，以获取更好的推理性能。此时模型输出 buf 是以 NHWC 顺序进行排布的，比如第一个输出的原始 shape 是 1,255,80,80，此时RKNN输出的 shape 是1,80,80,255，此demo中的后处理也根据这个顺序做了相应的优化调整。 |
||||
|
Before Width: | Height: | Size: 177 KiB |
@ -1 +0,0 @@
@@ -1 +0,0 @@
|
||||
bus.jpg |
||||
@ -1,55 +0,0 @@
@@ -1,55 +0,0 @@
|
||||
|
||||
import cv2 |
||||
import numpy as np |
||||
|
||||
from rknn.api import RKNN |
||||
import os |
||||
|
||||
if __name__ == '__main__': |
||||
|
||||
exp = 'yolov5s' |
||||
Width = 640 |
||||
Height = 640 |
||||
MODEL_PATH = './onnx_models/yolov5s.onnx' |
||||
NEED_BUILD_MODEL = True |
||||
# NEED_BUILD_MODEL = False |
||||
im_file = './bus.jpg' |
||||
|
||||
# Create RKNN object |
||||
rknn = RKNN() |
||||
|
||||
OUT_DIR = "rknn_models" |
||||
RKNN_MODEL_PATH = './{}/{}.rknn'.format(OUT_DIR, exp+'-'+str(Width)+'-'+str(Height)) |
||||
if NEED_BUILD_MODEL: |
||||
DATASET = './dataset.txt' |
||||
rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform="rv1106") |
||||
# Load model |
||||
print('--> Loading model') |
||||
ret = rknn.load_onnx(MODEL_PATH, outputs=['334', '353', '372']) |
||||
if ret != 0: |
||||
print('load model failed!') |
||||
exit(ret) |
||||
print('done') |
||||
|
||||
# Build model |
||||
print('--> Building model') |
||||
ret = rknn.build(do_quantization=True, dataset=DATASET) |
||||
if ret != 0: |
||||
print('build model failed.') |
||||
exit(ret) |
||||
print('done') |
||||
|
||||
# Export rknn model |
||||
if not os.path.exists(OUT_DIR): |
||||
os.mkdir(OUT_DIR) |
||||
print('--> Export RKNN model: {}'.format(RKNN_MODEL_PATH)) |
||||
ret = rknn.export_rknn(RKNN_MODEL_PATH) |
||||
if ret != 0: |
||||
print('Export rknn model failed.') |
||||
exit(ret) |
||||
print('done') |
||||
else: |
||||
ret = rknn.load_rknn(RKNN_MODEL_PATH) |
||||
|
||||
rknn.release() |
||||
|
||||
Binary file not shown.
Binary file not shown.
@ -1,73 +1,69 @@
@@ -1,73 +1,69 @@
|
||||
# 说明 |
||||
Android平台有两种方式来调用RKNN API |
||||
1）应用直接链接librknnrt.so |
||||
2）应用链接Android平台HIDL实现的librknn_api_android.so |
||||
对于需要通过CTS/VTS测试的Android设备可以使用基于Android平台HIDL实现的RKNN API。如果不需要通过CTS/VTS测试的设备建议直接链接使用librknnrt.so，对各个接口调用流程的链路更短，可以提供更好的性能。 |
||||
# Instructions |
||||
|
||||
对于使用Android HIDL实现的RKNN API的代码位于RK3566_RK3568/RK3588 Android系统SDK的vendor/rockchip/hardware/interfaces/neuralnetworks目录下。当完成Android系统编译后，将会生成一些NPU相关的库（对于应用只需要链接使用librknn_api_android.so即可） |
||||
There are two ways to use the RKNN API on the Android platform: |
||||
|
||||
**本示例适用于librknn_api_android.so。** |
||||
1. Directly link to librknnrt.so. |
||||
2. Link to librknn_api_android.so, which is implemented based on Android platform HIDL. |
||||
|
||||
For Android devices that need to pass CTS/VTS testing, it is recommended to use the RKNN API implemented based on Android platform HIDL. If devices don't need to pass CTS/VTS testing, it is suggested to directly link and use librknnrt.so, which provides better performance due to shorter interface calling process. |
||||
|
||||
The code for using the RKNN API implemented with Android HIDL can be found in the vendor/rockchip/hardware/interfaces/neuralnetworks directory of the RK3566_RK3568/RK3588 Android system SDK. After completing the Android system compilation, some NPU-related libraries will be generated (for applications, only librknn_api_android.so needs to be linked). |
||||
|
||||
# 编译 |
||||
**This example is applicable to librknn_api_android.so.** |
||||
|
||||
- 编译librknn_api_android.so |
||||
## Compilation |
||||
|
||||
需要先下载RK3566_RK3568/RK3588 Android SDK，在Android SDK根目录执行 |
||||
- Compile librknn_api_android.so |
||||
|
||||
``` |
||||
source build/envsetup.sh |
||||
lunch your target ##需要根据自己的实际情况进行选择 |
||||
mmm vendor/rockchip/hardware/interfaces/neuralnetworks/ -j16 |
||||
First, download the RK3566_RK3568/RK3588 Android SDK, and in the root directory of the Android SDK, then execute following commands: |
||||
|
||||
``` |
||||
``` |
||||
source build/envsetup.sh |
||||
lunch your target ## Choose according to your actual situation |
||||
mmm vendor/rockchip/hardware/interfaces/neuralnetworks/ -j16 |
||||
``` |
||||
|
||||
将生成 |
||||
The following files will be generated: |
||||
|
||||
``` |
||||
/vendor/lib/librknn_api_android.so |
||||
/vendor/lib/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/librknn_api_android.so |
||||
/vendor/lib64/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0-adapter-helper.so |
||||
/vendor/lib64/hw/rockchip.hardware.neuralnetworks@1.0-impl.so |
||||
/vendor/bin/hw/rockchip.hardware.neuralnetworks@1.0-service |
||||
``` |
||||
``` |
||||
/vendor/lib/librknn_api_android.so |
||||
/vendor/lib/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/librknn_api_android.so |
||||
/vendor/lib64/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0-adapter-helper.so |
||||
/vendor/lib64/hw/rockchip.hardware.neuralnetworks@1.0-impl.so |
||||
/vendor/bin/hw/rockchip.hardware.neuralnetworks@1.0-service |
||||
``` |
||||
|
||||
- Compile this demo |
||||
|
||||
- 编译本demo |
||||
Copy $RKNPU2_SDK to the root directory of the Android SDK, and execute: |
||||
|
||||
将$RKNPU2_SDK拷贝到Android SDK根目录，并执行： |
||||
``` |
||||
mmm rknpu2/examples/librknn_api_android_demo |
||||
``` |
||||
|
||||
``` |
||||
mmm rknpu2/examples/librknn_api_android_demo |
||||
``` |
||||
The following file will be generated: |
||||
|
||||
将生成的vendor/bin/rknn_create_mem_demo |
||||
vendor/bin/rknn_create_mem_demo |
||||
|
||||
# 运行 |
||||
## Execution |
||||
|
||||
|
||||
- 将rknn_create_mem_demo推到板子/vendor/bin/目录 |
||||
|
||||
- 将model推到板子/data/目录 |
||||
|
||||
- 确保板子的rockchip.hardware.neuralnetworks@1.0-service已经运行 |
||||
- Push rknn_create_mem_demo to the /vendor/bin/ directory of the target device. |
||||
- Push the model to the /data/ directory of the target device. |
||||
- Make sure that the [rockchip.hardware.neuralnetworks@1.0-service](mailto:rockchip.hardware.neuralnetworks@1.0-service) is running on the device. |
||||
|
||||
``` |
||||
rknn_create_mem_demo /data/model/RK3566_RK3568/mobilenet_v1.rknn /data/model/dog_224x224.jpg |
||||
rknn_create_mem_demo /data/model/RK3566_RK3568/mobilenet_v1.rknn /data/model/dog_224x224.jpg |
||||
``` |
||||
|
||||
## FAQ |
||||
|
||||
- What should I do if the [rockchip.hardware.neuralnetworks@1.0-service](mailto:rockchip.hardware.neuralnetworks@1.0-service) is not running? |
||||
|
||||
# FAQ |
||||
|
||||
- rockchip.hardware.neuralnetworks@1.0-service服务没有运行怎么办 |
||||
|
||||
如果该服务没有运行，从Android SDK确保vendor/rockchip/hardware/interfaces/neuralnetworks/目录存在，并且重新编译系统固件，并重新烧写到板子上，具体步骤请参考SDK编译固件的说明。 |
||||
If the service is not running, make sure that the vendor/rockchip/hardware/interfaces/neuralnetworks/ directory exists in the Android SDK, recompile the system firmware, and burn it onto the device. Please refer to the instructions for compiling the SDK firmware for specific steps. |
||||
|
||||
- 遇到sizeof(rknn_tensor_attr) != sizeof(::rockchip::hardware::neuralnetworks::V1_0::RKNNTensorAttr)的错误 |
||||
- Encounter the error "sizeof(rknn_tensor_attr) != sizeof(::rockchip::hardware::neuralnetworks::V1_0::RKNNTensorAttr)" |
||||
|
||||
需要更新vendor/rockchip/hardware/interfaces/neuralnetworks到最新代码 |
||||
You need to update the vendor/rockchip/hardware/interfaces/neuralnetworks to the latest version. |
||||
|
||||
@ -0,0 +1,73 @@
@@ -0,0 +1,73 @@
|
||||
# 说明 |
||||
Android平台有两种方式来调用RKNN API |
||||
1）应用直接链接librknnrt.so |
||||
2）应用链接Android平台HIDL实现的librknn_api_android.so |
||||
对于需要通过CTS/VTS测试的Android设备可以使用基于Android平台HIDL实现的RKNN API。如果不需要通过CTS/VTS测试的设备建议直接链接使用librknnrt.so，对各个接口调用流程的链路更短，可以提供更好的性能。 |
||||
|
||||
对于使用Android HIDL实现的RKNN API的代码位于RK3566_RK3568/RK3588 Android系统SDK的vendor/rockchip/hardware/interfaces/neuralnetworks目录下。当完成Android系统编译后，将会生成一些NPU相关的库（对于应用只需要链接使用librknn_api_android.so即可） |
||||
|
||||
**本示例适用于librknn_api_android.so。** |
||||
|
||||
|
||||
|
||||
# 编译 |
||||
|
||||
- 编译librknn_api_android.so |
||||
|
||||
需要先下载RK3566_RK3568/RK3588 Android SDK，在Android SDK根目录执行 |
||||
|
||||
``` |
||||
source build/envsetup.sh |
||||
lunch your target ##需要根据自己的实际情况进行选择 |
||||
mmm vendor/rockchip/hardware/interfaces/neuralnetworks/ -j16 |
||||
|
||||
``` |
||||
|
||||
将生成 |
||||
|
||||
``` |
||||
/vendor/lib/librknn_api_android.so |
||||
/vendor/lib/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/librknn_api_android.so |
||||
/vendor/lib64/librknnhal_bridge.rockchip.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0.so |
||||
/vendor/lib64/rockchip.hardware.neuralnetworks@1.0-adapter-helper.so |
||||
/vendor/lib64/hw/rockchip.hardware.neuralnetworks@1.0-impl.so |
||||
/vendor/bin/hw/rockchip.hardware.neuralnetworks@1.0-service |
||||
``` |
||||
|
||||
|
||||
- 编译本demo |
||||
|
||||
将$RKNPU2_SDK拷贝到Android SDK根目录，并执行： |
||||
|
||||
``` |
||||
mmm rknpu2/examples/librknn_api_android_demo |
||||
``` |
||||
|
||||
将生成的vendor/bin/rknn_create_mem_demo |
||||
|
||||
# 运行 |
||||
|
||||
|
||||
- 将rknn_create_mem_demo推到板子/vendor/bin/目录 |
||||
|
||||
- 将model推到板子/data/目录 |
||||
|
||||
- 确保板子的rockchip.hardware.neuralnetworks@1.0-service已经运行 |
||||
|
||||
``` |
||||
rknn_create_mem_demo /data/model/RK3566_RK3568/mobilenet_v1.rknn /data/model/dog_224x224.jpg |
||||
``` |
||||
|
||||
|
||||
|
||||
# FAQ |
||||
|
||||
- rockchip.hardware.neuralnetworks@1.0-service服务没有运行怎么办 |
||||
|
||||
如果该服务没有运行，从Android SDK确保vendor/rockchip/hardware/interfaces/neuralnetworks/目录存在，并且重新编译系统固件，并重新烧写到板子上，具体步骤请参考SDK编译固件的说明。 |
||||
|
||||
- 遇到sizeof(rknn_tensor_attr) != sizeof(::rockchip::hardware::neuralnetworks::V1_0::RKNNTensorAttr)的错误 |
||||
|
||||
需要更新vendor/rockchip/hardware/interfaces/neuralnetworks到最新代码 |
||||
@ -0,0 +1,76 @@
@@ -0,0 +1,76 @@
|
||||
以下 <TARGET_PLATFORM> 表示RK3566_RK3568、RK3562或RK3588。 |
||||
# Aarch64 Linux 示例 |
||||
## 编译 |
||||
|
||||
将`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`修改成交叉编译器路径, 然后执行 |
||||
|
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
将 install/rknn_api_demo_Linux 拷贝到设备上。 |
||||
|
||||
- 如果使用Rockchip的EVB板，可以使用以下命令： |
||||
|
||||
连接设备并将程序和模型传输到`/userdata` |
||||
|
||||
``` |
||||
adb push install/rknn_api_demo_Linux /userdata/ |
||||
adb push ../rknn_mobilenet_demo/model/ /userdata/rknn_api_demo_Linux |
||||
``` |
||||
|
||||
- 如果你的板子有sshd服务，可以使用scp命令或者其他方式将程序和模型传输到板子上。 |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_api_demo_Linux/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_create_mem_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
./rknn_create_mem_with_rga_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
|
||||
# Android 示例 |
||||
## 编译 |
||||
|
||||
将`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`修改成平台对应的NDK，然后执行 |
||||
|
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
连接设备并将程序和模型传输到`/data` |
||||
|
||||
``` |
||||
adb push install/rknn_api_demo_Android /data/ |
||||
adb push ../rknn_mobilenet_demo/model/ /userdata/rknn_api_demo_Android |
||||
``` |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /data/rknn_api_demo_Android/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_create_mem_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
./rknn_create_mem_with_rga_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
./rknn_with_mmz_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
./rknn_set_internal_mem_from_fd_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
./rknn_set_internal_mem_from_phy_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
|
||||
# 注意： |
||||
- 你可能需要依赖系统中的MMZ实现更新libmpimmz.so和头文件。 |
||||
- 你可能需要依赖系统中的RGA实现更新librga.so和头文件。库地址：https://github.com/airockchip/librga。对于RK3562,librga库版本需要大于等于1.9.1。 |
||||
- 你可能需要使用r19c或者更老的NDK版本编译MMZ相关的Demo。 |
||||
@ -0,0 +1,64 @@
@@ -0,0 +1,64 @@
|
||||
以下 <TARGET_PLATFORM> 表示RK3566_RK3568、RK3562或RK3588。 |
||||
# Aarch64 Linux 示例 |
||||
## 编译 |
||||
|
||||
将`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`修改成交叉编译器路径, 然后执行 |
||||
|
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
将 install/rknn_api_demo_Linux 拷贝到设备上。 |
||||
|
||||
- 如果使用Rockchip的EVB板，可以使用以下命令： |
||||
|
||||
连接设备并将程序和模型传输到`/userdata` |
||||
|
||||
``` |
||||
adb push install/rknn_common_test_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果你的板子有sshd服务，可以使用scp命令或者其他方式将程序和模型传输到板子上。 |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_common_test_Linux/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_common_test model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
|
||||
# Android 示例 |
||||
## 编译 |
||||
|
||||
将`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`修改成平台对应的NDK，然后执行 |
||||
|
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
连接设备并将程序和模型传输到`/data` |
||||
|
||||
``` |
||||
adb push install/rknn_common_test_Android /data/ |
||||
``` |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /data/rknn_common_test_Android/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_common_test model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
@ -1,52 +1,73 @@
@@ -1,52 +1,73 @@
|
||||
# RKNN C API 动态形状输入Demo |
||||
这是一个使用RKNN C API进行动态形状输入推理的演示应用。您可以在这个应用中看到如何使用RKNN 动态形状 C API对图像进行分类。 |
||||
# RKNN C API Dynamic Shape Input Demo |
||||
|
||||
# 如何使用 |
||||
1. 克隆或下载此代码库ssh://git@10.10.10.59:8001/hpc/rknpu2.git。 |
||||
2. 在终端中进入动态形状推理Demo目录。 |
||||
``` |
||||
This is a demo that uses the RKNN C API for dynamic shape input inference. In this demo, you can see how to use the RKNN dynamic shape C API to perform image classification. |
||||
|
||||
## How to Use |
||||
|
||||
1. Clone or download this code repository: ssh://git@10.10.10.59:8001/hpc/rknpu2.git. |
||||
2. Navigate to the dynamic shape inference demo directory in your terminal. |
||||
|
||||
```shell |
||||
cd examples/rknn_dynamic_shape_input_demo |
||||
``` |
||||
3. 根据芯片平台，运行shell脚本编译应用程序,以RK3562 Android系统为例，命令如下: |
||||
``` |
||||
|
||||
3. Compile the application by running the shell script based on the chip platform. For example, for the RK3562 Android system, run the following command: |
||||
|
||||
```shell |
||||
./build-android_RK3562.sh |
||||
``` |
||||
4. 将Demo程序目录通过adb命令推送到开发板系统中，命令如下: |
||||
``` |
||||
|
||||
4. Push the demo program directory to the target board's system using the adb command. For example: |
||||
|
||||
```shell |
||||
#If using Android system, make sure to run adb root & adb remount first. |
||||
adb push ./install/rknn_dynshape_demo_Android/ /data |
||||
注意：如果是安卓系统，需要adb root & adb remount |
||||
``` |
||||
5. 设置runtime库链接路径 |
||||
|
||||
5. Set the runtime library path. |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
``` |
||||
6. 运行程序，以rk3562平台为例，./rknn_dynshape_inference model/RK3562/mobilenet_v2.rknn images/dog_224x224.jpg 命令对图像进行分类，其中 mobilenet_v2.rknn 是神经网络模型文件的名称，dog_224x224.jpg 是要分类的图像文件的名称。 |
||||
|
||||
6. Run the program. For example, on the RK3562 platform, use the command |
||||
|
||||
```shell |
||||
./rknn_dynshape_inference model/RK3562/mobilenet_v2.rknn images/dog_224x224.jpg |
||||
``` |
||||
|
||||
,where `mobilenet_v2.rknn` is the name of the neural network model file, and `dog_224x224.jpg` is the name of the image file to classify. |
||||
|
||||
## Compilation Instructions |
||||
|
||||
### Arm Linux |
||||
|
||||
Specify the cross-compiler path for the specific chip platform by modifying the `GCC_COMPILER` in `build-linux_<TARGET_PLATFORM>.sh`, where TARGET_PLATFORM is the chip name. Then execute: |
||||
|
||||
# 编译说明 |
||||
## Arm Linux系统 |
||||
为特定的芯片平台指定交叉编译器路径，修改`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`,其中TARGET_PLATFORM为芯片名，然后执行 |
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
## Android系统 |
||||
指定Android NDK的路径，修改`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`，其中TARGET_PLATFORM为芯片名，然后执行 |
||||
|
||||
### Android |
||||
|
||||
Specify the path to the Android NDK by modifying `ANDROID_NDK_PATH` in `build-android_<TARGET_PLATFORM>.sh`, where TARGET_PLATFORM is the chip name. Then execute: |
||||
|
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
# 包含的功能 |
||||
此演示应用程序包含以下功能： |
||||
## Included Features |
||||
|
||||
This demonstration application includes the following features: |
||||
|
||||
- 创建一个包含动态形状的神经网络模型。 |
||||
参考https://github.com/rockchip-linux/rknn-toolkit2仓库下的examples/functions/dynamic_input |
||||
- Creating a neural network model with dynamic shape inputs. Please refer to the examples/functions/dynamic_input directory in the https://github.com/rockchip-linux/rknn-toolkit2 repository for more information. |
||||
- Reading an image from a file and performing classification using the neural network model. The program follows these steps: |
||||
|
||||
- 从文件中读取一张图像，并使用神经网络模型对其进行分类。程序步骤如下： |
||||
1. 使用 rknn_init() 函数初始化 RKNN 上下文。 |
||||
2. 使用 rknn_set_input_shape() 函数设置模型输入的形状信息，包括形状、布局等。 |
||||
3. 使用 rknn_query() 函数查询当前设置的模型输入和输出的信息，包括形状、数据类型和大小等。 |
||||
4. 使用 rknn_inputs_set() 函数设置模型输入的数据，包括数据指针和数据大小等。 |
||||
5. 使用 rknn_run() 函数运行模型。 |
||||
6. 使用 rknn_outputs_get() 函数设置是否需要float类型结果并获取输出数据。 |
||||
7. 处理输出数据，得到分类结果和概率。 |
||||
8. 使用 rknn_release() 函数释放RKNN上下文。 |
||||
1. Initialize the RKNN context using the `rknn_init()` function. |
||||
2. Set the shape information of all the model inputs using the `rknn_set_input_shapes()` function, including shape and layout. |
||||
3. Query the current model input and output information, including shape, data type, and size, using the `rknn_query()` function. |
||||
4. Set the input data of the model using the `rknn_inputs_set()` function, including data pointer and size. |
||||
5. Run the model using the `rknn_run()` function. |
||||
6. Retrieve the output data by using the `rknn_outputs_get()` function, specifying the need for float-type results. |
||||
7. Process the output data to obtain the classification results and probabilities. |
||||
8. Release the RKNN context using the `rknn_release()` function. |
||||
@ -0,0 +1,52 @@
@@ -0,0 +1,52 @@
|
||||
# RKNN C API 动态形状输入Demo |
||||
这是一个使用RKNN C API进行动态形状输入推理的演示应用。您可以在这个应用中看到如何使用RKNN 动态形状 C API对图像进行分类。 |
||||
|
||||
# 如何使用 |
||||
1. 克隆或下载此代码库ssh://git@10.10.10.59:8001/hpc/rknpu2.git。 |
||||
2. 在终端中进入动态形状推理Demo目录。 |
||||
``` |
||||
cd examples/rknn_dynamic_shape_input_demo |
||||
``` |
||||
3. 根据芯片平台，运行shell脚本编译应用程序,以RK3562 Android系统为例，命令如下: |
||||
``` |
||||
./build-android_RK3562.sh |
||||
``` |
||||
4. 将Demo程序目录通过adb命令推送到开发板系统中，命令如下: |
||||
``` |
||||
adb push ./install/rknn_dynshape_demo_Android/ /data |
||||
注意：如果是安卓系统，需要adb root & adb remount |
||||
``` |
||||
5. 设置runtime库链接路径 |
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
``` |
||||
6. 运行程序，以rk3562平台为例，./rknn_dynshape_inference model/RK3562/mobilenet_v2.rknn images/dog_224x224.jpg 命令对图像进行分类，其中 mobilenet_v2.rknn 是神经网络模型文件的名称，dog_224x224.jpg 是要分类的图像文件的名称。 |
||||
|
||||
|
||||
# 编译说明 |
||||
## Arm Linux系统 |
||||
为特定的芯片平台指定交叉编译器路径，修改`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`,其中TARGET_PLATFORM为芯片名，然后执行 |
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
## Android系统 |
||||
指定Android NDK的路径，修改`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`，其中TARGET_PLATFORM为芯片名，然后执行 |
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
# 包含的功能 |
||||
此演示应用程序包含以下功能： |
||||
|
||||
- 创建一个包含动态形状的神经网络模型。 |
||||
参考https://github.com/rockchip-linux/rknn-toolkit2仓库下的examples/functions/dynamic_input |
||||
|
||||
- 从文件中读取一张图像，并使用神经网络模型对其进行分类。程序步骤如下： |
||||
1. 使用 rknn_init() 函数初始化 RKNN 上下文。 |
||||
2. 使用 rknn_set_input_shapes() 函数设置模型所有的输入的形状信息，包括形状、布局等。 |
||||
3. 使用 rknn_query() 函数查询当前设置的模型输入和输出的信息，包括形状、数据类型和大小等。 |
||||
4. 使用 rknn_inputs_set() 函数设置模型输入的数据，包括数据指针和数据大小等。 |
||||
5. 使用 rknn_run() 函数运行模型。 |
||||
6. 使用 rknn_outputs_get() 函数设置是否需要float类型结果并获取输出数据。 |
||||
7. 处理输出数据，得到分类结果和概率。 |
||||
8. 使用 rknn_release() 函数释放RKNN上下文。 |
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,904 @@
@@ -0,0 +1,904 @@
|
||||
/****************************************************************************
|
||||
* |
||||
* Copyright (c) 2017 - 2023 by Rockchip Corp. All rights reserved. |
||||
* |
||||
* The material in this file is confidential and contains trade secrets |
||||
* of Rockchip Corporation. This is proprietary information owned by |
||||
* Rockchip Corporation. No part of this work may be disclosed, |
||||
* reproduced, copied, transmitted, or used in any way for any purpose, |
||||
* without the express written permission of Rockchip Corporation. |
||||
* |
||||
*****************************************************************************/ |
||||
|
||||
/*-------------------------------------------
|
||||
Includes |
||||
-------------------------------------------*/ |
||||
#include "rk_mpi_mmz.h" |
||||
#include "rknn_api.h" |
||||
|
||||
#include <float.h> |
||||
#include <stdio.h> |
||||
#include <stdlib.h> |
||||
#include <string.h> |
||||
#include <sys/time.h> |
||||
|
||||
#include <string> |
||||
#include <vector> |
||||
|
||||
#define STB_IMAGE_IMPLEMENTATION |
||||
#include "stb/stb_image.h" |
||||
#define STB_IMAGE_RESIZE_IMPLEMENTATION |
||||
#include <stb/stb_image_resize.h> |
||||
|
||||
#define NPY_SUPPORT 1 |
||||
#if NPY_SUPPORT |
||||
#include "cnpy/cnpy.h" |
||||
using namespace cnpy; |
||||
#endif |
||||
|
||||
#define TIME_BEGIN(name) \ |
||||
struct timeval tv##name; \ |
||||
gettimeofday(&tv##name, NULL); \ |
||||
long val##name = tv##name.tv_usec; \ |
||||
long min##name = tv##name.tv_sec; |
||||
|
||||
#define TIME_END(name) \ |
||||
gettimeofday(&tv##name, NULL); \ |
||||
val##name = tv##name.tv_usec - val##name; \ |
||||
val##name += 1000000 * (tv##name.tv_sec - min##name); \ |
||||
printf("[%s]exectime is %ld us, %ld ms\n", #name, val##name, val##name / 1000); |
||||
|
||||
static int mb_flags = RK_MMZ_ALLOC_TYPE_IOMMU | RK_MMZ_ALLOC_CACHEABLE; |
||||
|
||||
#define SIZE_ALIGN(size, align) (((size) + ((align)-1)) & (~((align)-1))) |
||||
#define SIZE_ALIGN_128(size) SIZE_ALIGN(size, 128) |
||||
|
||||
/*-------------------------------------------
|
||||
Functions |
||||
-------------------------------------------*/ |
||||
static inline int64_t getCurrentTimeUs() |
||||
{ |
||||
struct timeval tv; |
||||
gettimeofday(&tv, NULL); |
||||
return tv.tv_sec * 1000000 + tv.tv_usec; |
||||
} |
||||
|
||||
static int rknn_GetTopN(float *pfProb, float *pfMaxProb, uint32_t *pMaxClass, uint32_t outputCount, uint32_t topNum) |
||||
{ |
||||
uint32_t i, j; |
||||
uint32_t top_count = outputCount > topNum ? topNum : outputCount; |
||||
|
||||
for (i = 0; i < topNum; ++i) |
||||
{ |
||||
pfMaxProb[i] = -FLT_MAX; |
||||
pMaxClass[i] = -1; |
||||
} |
||||
|
||||
for (j = 0; j < top_count; j++) |
||||
{ |
||||
for (i = 0; i < outputCount; i++) |
||||
{ |
||||
if ((i == *(pMaxClass + 0)) || (i == *(pMaxClass + 1)) || (i == *(pMaxClass + 2)) || (i == *(pMaxClass + 3)) || |
||||
(i == *(pMaxClass + 4))) |
||||
{ |
||||
continue; |
||||
} |
||||
|
||||
if (pfProb[i] > *(pfMaxProb + j)) |
||||
{ |
||||
*(pfMaxProb + j) = pfProb[i]; |
||||
*(pMaxClass + j) = i; |
||||
} |
||||
} |
||||
} |
||||
|
||||
return 1; |
||||
} |
||||
|
||||
static void dump_tensor_attr(rknn_tensor_attr *attr) |
||||
{ |
||||
printf(" index=%d, name=%s, n_dims=%d, dims=[%d, %d, %d, %d], n_elems=%d, size=%d, fmt=%s, type=%s, qnt_type=%s, " |
||||
"zp=%d, scale=%f\n", |
||||
attr->index, attr->name, attr->n_dims, attr->dims[0], attr->dims[1], attr->dims[2], attr->dims[3], |
||||
attr->n_elems, attr->size, get_format_string(attr->fmt), get_type_string(attr->type), |
||||
get_qnt_type_string(attr->qnt_type), attr->zp, attr->scale); |
||||
} |
||||
|
||||
static void dump_input_dynamic_range(rknn_input_range *dyn_range) |
||||
{ |
||||
std::string range_str = ""; |
||||
for (int n = 0; n < dyn_range->shape_number; ++n) |
||||
{ |
||||
range_str += n == 0 ? "[" : ",["; |
||||
range_str += dyn_range->n_dims < 1 ? "" : std::to_string(dyn_range->dyn_range[n][0]); |
||||
for (int i = 1; i < dyn_range->n_dims; ++i) |
||||
{ |
||||
range_str += ", " + std::to_string(dyn_range->dyn_range[n][i]); |
||||
} |
||||
range_str += "]"; |
||||
} |
||||
|
||||
printf(" index=%d, name=%s, shape_number=%d, range=[%s], fmt = %s\n", dyn_range->index, dyn_range->name, |
||||
dyn_range->shape_number, range_str.c_str(), get_format_string(dyn_range->fmt)); |
||||
} |
||||
|
||||
static unsigned char *load_image(const char *image_path, rknn_tensor_attr *input_attr) |
||||
{ |
||||
int req_height = 0; |
||||
int req_width = 0; |
||||
int req_channel = 0; |
||||
|
||||
switch (input_attr->fmt) |
||||
{ |
||||
case RKNN_TENSOR_NHWC: |
||||
req_height = input_attr->dims[1]; |
||||
req_width = input_attr->dims[2]; |
||||
req_channel = input_attr->dims[3]; |
||||
break; |
||||
case RKNN_TENSOR_NCHW: |
||||
req_height = input_attr->dims[2]; |
||||
req_width = input_attr->dims[3]; |
||||
req_channel = input_attr->dims[1]; |
||||
break; |
||||
default: |
||||
printf("meet unsupported layout\n"); |
||||
return NULL; |
||||
} |
||||
|
||||
int height = 0; |
||||
int width = 0; |
||||
int channel = 0; |
||||
|
||||
unsigned char *image_data = stbi_load(image_path, &width, &height, &channel, req_channel); |
||||
if (image_data == NULL) |
||||
{ |
||||
printf("load image failed!\n"); |
||||
return NULL; |
||||
} |
||||
|
||||
if (width != req_width || height != req_height) |
||||
{ |
||||
unsigned char *image_resized = (unsigned char *)STBI_MALLOC(req_width * req_height * req_channel); |
||||
if (!image_resized) |
||||
{ |
||||
printf("malloc image failed!\n"); |
||||
STBI_FREE(image_data); |
||||
return NULL; |
||||
} |
||||
if (stbir_resize_uint8(image_data, width, height, 0, image_resized, req_width, req_height, 0, channel) != 1) |
||||
{ |
||||
printf("resize image failed!\n"); |
||||
STBI_FREE(image_data); |
||||
return NULL; |
||||
} |
||||
STBI_FREE(image_data); |
||||
image_data = image_resized; |
||||
} |
||||
|
||||
return image_data; |
||||
} |
||||
|
||||
#if NPY_SUPPORT |
||||
static unsigned char *load_npy(const char *input_path, rknn_tensor_attr *input_attr, int *input_type, int *input_size, |
||||
int *type_bytes) |
||||
{ |
||||
printf("Loading %s\n", input_path); |
||||
|
||||
NpyArray npy_data = npy_load(input_path); |
||||
|
||||
*type_bytes = npy_data.word_size; |
||||
std::string typeName = npy_data.typeName; |
||||
|
||||
printf("npy data type:%s\n", typeName.c_str()); |
||||
|
||||
if (typeName == "int8") |
||||
{ |
||||
*input_type = RKNN_TENSOR_INT8; |
||||
} |
||||
else if (typeName == "uint8") |
||||
{ |
||||
*input_type = RKNN_TENSOR_UINT8; |
||||
} |
||||
else if (typeName == "float16") |
||||
{ |
||||
*input_type = RKNN_TENSOR_FLOAT16; |
||||
} |
||||
else if (typeName == "float32") |
||||
{ |
||||
*input_type = RKNN_TENSOR_FLOAT32; |
||||
} |
||||
else if (typeName == "8") |
||||
{ |
||||
*input_type = RKNN_TENSOR_BOOL; |
||||
} |
||||
else if (typeName == "int64") |
||||
{ |
||||
*input_type = RKNN_TENSOR_INT64; |
||||
} |
||||
|
||||
// npy shape = NHWC
|
||||
std::vector<int> npy_shape; |
||||
for (size_t i = 0; i < npy_data.shape.size(); ++i) |
||||
{ |
||||
npy_shape.emplace_back(npy_data.shape[i]); |
||||
} |
||||
|
||||
int height = npy_shape.size() > 1 ? npy_shape[1] : 1; |
||||
int width = npy_shape.size() > 2 ? npy_shape[2] : 1; |
||||
int channel = npy_shape.size() > 3 ? npy_shape[3] : 1; |
||||
|
||||
switch (input_attr->fmt) |
||||
{ |
||||
case RKNN_TENSOR_NHWC: |
||||
input_attr->dims[0] = npy_shape[0]; |
||||
input_attr->dims[1] = height; |
||||
input_attr->dims[2] = width; |
||||
input_attr->dims[3] = channel; |
||||
break; |
||||
case RKNN_TENSOR_UNDEFINED: |
||||
for (int idx = 0; idx < input_attr->n_dims; ++idx) |
||||
{ |
||||
input_attr->dims[idx] = npy_shape[idx]; |
||||
} |
||||
break; |
||||
default: |
||||
fprintf(stderr, "load_npy error, unsupport model input layout: %s\n", get_format_string(input_attr->fmt)); |
||||
break; |
||||
} |
||||
|
||||
unsigned char *data = (unsigned char *)malloc(npy_data.num_bytes()); |
||||
if (!data) |
||||
{ |
||||
return NULL; |
||||
} |
||||
|
||||
// TODO: copy
|
||||
memcpy(data, npy_data.data<unsigned char>(), npy_data.num_bytes()); |
||||
|
||||
*input_size = npy_data.num_bytes(); |
||||
|
||||
return data; |
||||
} |
||||
|
||||
static void save_npy(const char *output_path, float *output_data, rknn_tensor_attr *output_attr) |
||||
{ |
||||
std::vector<size_t> output_shape; |
||||
|
||||
for (uint32_t i = 0; i < output_attr->n_dims; ++i) |
||||
{ |
||||
output_shape.push_back(output_attr->dims[i]); |
||||
} |
||||
|
||||
npy_save<float>(output_path, output_data, output_shape); |
||||
} |
||||
#endif |
||||
|
||||
static std::vector<std::string> split(const std::string &str, const std::string &pattern) |
||||
{ |
||||
std::vector<std::string> res; |
||||
if (str == "") |
||||
return res; |
||||
std::string strs = str + pattern; |
||||
size_t pos = strs.find(pattern); |
||||
while (pos != strs.npos) |
||||
{ |
||||
std::string temp = strs.substr(0, pos); |
||||
res.push_back(temp); |
||||
strs = strs.substr(pos + 1, strs.size()); |
||||
pos = strs.find(pattern); |
||||
} |
||||
return res; |
||||
} |
||||
|
||||
uint32_t get_file_size(char *file_name) |
||||
{ |
||||
FILE *fid = fopen(file_name, "rb"); |
||||
if (fid == NULL) |
||||
{ |
||||
printf("open file error\n"); |
||||
fclose(fid); |
||||
return -1; |
||||
} |
||||
|
||||
fseek(fid, 0, SEEK_END); |
||||
uint32_t size = ftell(fid); |
||||
|
||||
fclose(fid); |
||||
return size; |
||||
} |
||||
|
||||
int read_bin_file(char *file_name, void *out_ptr, unsigned int size) |
||||
{ |
||||
FILE *fid = fopen(file_name, "rb"); |
||||
if (fid == NULL) |
||||
{ |
||||
printf("open file error\n"); |
||||
fclose(fid); |
||||
return -1; |
||||
} |
||||
fread(out_ptr, 1, size, fid); |
||||
|
||||
fclose(fid); |
||||
return 0; |
||||
} |
||||
|
||||
/*-------------------------------------------
|
||||
Main Functions |
||||
-------------------------------------------*/ |
||||
int main(int argc, char *argv[]) |
||||
{ |
||||
if (argc < 3) |
||||
{ |
||||
printf("Usage:%s model_path input_path [loop_count] [core_mask] [output_dir]\n", argv[0]); |
||||
return -1; |
||||
} |
||||
|
||||
char *model_path = argv[1]; |
||||
char *input_paths = argv[2]; |
||||
std::vector<std::string> input_paths_split = split(input_paths, "#"); |
||||
|
||||
int loop_count = 1; |
||||
if (argc > 3) |
||||
{ |
||||
loop_count = atoi(argv[3]); |
||||
} |
||||
|
||||
uint32_t core_mask = 1; |
||||
if (argc > 4) |
||||
{ |
||||
// core_mask = atoi(argv[4]);
|
||||
core_mask = strtoul(argv[4], NULL, 10); |
||||
} |
||||
|
||||
char *output_dir = NULL; |
||||
if (argc > 5) |
||||
{ |
||||
output_dir = argv[5]; |
||||
} |
||||
|
||||
rknn_context ctx = 0; |
||||
|
||||
// Allocate model memory in outside
|
||||
MB_BLK model_mb; |
||||
uint32_t model_size = get_file_size(model_path); |
||||
int ret = RK_MPI_MMZ_Alloc(&model_mb, model_size, mb_flags); |
||||
if (ret < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Alloc failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
void *model_virt = RK_MPI_MMZ_Handle2VirAddr(model_mb); |
||||
if (model_virt == NULL) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2VirAddr failed!\n"); |
||||
return -1; |
||||
} |
||||
ret = read_bin_file(model_path, model_virt, model_size); |
||||
if (ret < 0) |
||||
{ |
||||
printf("read_bin_file failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
|
||||
TIME_BEGIN(dummpy_rknn_init); |
||||
ret = rknn_init(&ctx, model_virt, model_size, RKNN_FLAG_COLLECT_MODEL_INFO_ONLY | RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL); |
||||
TIME_END(dummpy_rknn_init); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_init with RKNN_FLAG_COLLECT_MODEL_INFO_ONLY fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
|
||||
// [dummpy_rknn_init] Get weight and internal mem size
|
||||
rknn_mem_size mem_size; |
||||
ret = rknn_query(ctx, RKNN_QUERY_MEM_SIZE, &mem_size, sizeof(mem_size)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("[dummpy init] total weight size: %d, total internal size: %d\n", mem_size.total_weight_size, |
||||
mem_size.total_internal_size); |
||||
|
||||
// Load RKNN Model
|
||||
TIME_BEGIN(rknn_init); |
||||
ret = rknn_init(&ctx, model_virt, model_size, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL); |
||||
TIME_END(rknn_init); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_init fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
|
||||
// Get sdk and driver version
|
||||
rknn_sdk_version sdk_ver; |
||||
ret = rknn_query(ctx, RKNN_QUERY_SDK_VERSION, &sdk_ver, sizeof(sdk_ver)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("rknn_api/rknnrt version: %s, driver version: %s\n", sdk_ver.api_version, sdk_ver.drv_version); |
||||
|
||||
// Get weight and internal mem size
|
||||
ret = rknn_query(ctx, RKNN_QUERY_MEM_SIZE, &mem_size, sizeof(mem_size)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("total weight size: %u, total internal size: %u\n", mem_size.total_weight_size, mem_size.total_internal_size); |
||||
|
||||
// Get Model Input Output Info
|
||||
rknn_input_output_num io_num; |
||||
ret = rknn_query(ctx, RKNN_QUERY_IN_OUT_NUM, &io_num, sizeof(io_num)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("model input num: %d, output num: %d\n", io_num.n_input, io_num.n_output); |
||||
|
||||
printf("default input tensors:\n"); |
||||
rknn_tensor_attr input_attrs[io_num.n_input]; |
||||
memset(input_attrs, 0, io_num.n_input * sizeof(rknn_tensor_attr)); |
||||
for (uint32_t i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
input_attrs[i].index = i; |
||||
// query info
|
||||
ret = rknn_query(ctx, RKNN_QUERY_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr)); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_init error! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
dump_tensor_attr(&input_attrs[i]); |
||||
} |
||||
|
||||
printf("default output tensors:\n"); |
||||
rknn_tensor_attr output_attrs[io_num.n_output]; |
||||
memset(output_attrs, 0, io_num.n_output * sizeof(rknn_tensor_attr)); |
||||
for (uint32_t i = 0; i < io_num.n_output; i++) |
||||
{ |
||||
output_attrs[i].index = i; |
||||
// query info
|
||||
ret = rknn_query(ctx, RKNN_QUERY_OUTPUT_ATTR, &(output_attrs[i]), sizeof(rknn_tensor_attr)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
dump_tensor_attr(&output_attrs[i]); |
||||
} |
||||
|
||||
// Get custom string
|
||||
rknn_custom_string custom_string; |
||||
ret = rknn_query(ctx, RKNN_QUERY_CUSTOM_STRING, &custom_string, sizeof(custom_string)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("custom string: %s\n", custom_string.string); |
||||
|
||||
printf("dynamic inputs shape range:\n"); |
||||
rknn_input_range dyn_range[io_num.n_input]; |
||||
memset(dyn_range, 0, io_num.n_input * sizeof(rknn_input_range)); |
||||
for (uint32_t i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
dyn_range[i].index = i; |
||||
ret = rknn_query(ctx, RKNN_QUERY_INPUT_DYNAMIC_RANGE, &dyn_range[i], sizeof(rknn_input_range)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
dump_input_dynamic_range(&dyn_range[i]); |
||||
} |
||||
|
||||
unsigned char *input_data[io_num.n_input]; |
||||
int input_type[io_num.n_input]; |
||||
int input_layout[io_num.n_input]; |
||||
int input_size[io_num.n_input]; |
||||
int type_bytes[io_num.n_input]; |
||||
for (int i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
input_data[i] = NULL; |
||||
input_type[i] = RKNN_TENSOR_UINT8; |
||||
input_layout[i] = RKNN_TENSOR_NHWC; |
||||
input_size[i] = input_attrs[i].size; |
||||
type_bytes[i] = 1; |
||||
} |
||||
|
||||
// Load input
|
||||
if (io_num.n_input != input_paths_split.size()) |
||||
{ |
||||
return -1; |
||||
} |
||||
for (int i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
if (strstr(input_paths_split[i].c_str(), ".npy")) |
||||
{ |
||||
// Load npy
|
||||
#if NPY_SUPPORT |
||||
input_data[i] = |
||||
load_npy(input_paths_split[i].c_str(), &input_attrs[i], &input_type[i], &input_size[i], &type_bytes[i]); |
||||
#else |
||||
return -1; |
||||
#endif |
||||
} |
||||
else |
||||
{ |
||||
// Load image
|
||||
for (int i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
input_data[i] = load_image(input_paths_split[i].c_str(), &input_attrs[i]); |
||||
} |
||||
} |
||||
|
||||
if (!input_data[i]) |
||||
{ |
||||
return -1; |
||||
} |
||||
} |
||||
|
||||
ret = rknn_set_input_shapes(ctx, io_num.n_input, input_attrs); |
||||
if (ret < 0) |
||||
{ |
||||
fprintf(stderr, "rknn_set_input_shapes error! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
|
||||
// Allocate weight memory in outside
|
||||
MB_BLK weight_mb; |
||||
rknn_tensor_mem *weight_mem; |
||||
ret = RK_MPI_MMZ_Alloc(&weight_mb, SIZE_ALIGN_128(mem_size.total_weight_size), mb_flags); |
||||
if (ret < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Alloc failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
void *weight_virt = RK_MPI_MMZ_Handle2VirAddr(weight_mb); |
||||
if (weight_virt == NULL) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2VirAddr failed!\n"); |
||||
return -1; |
||||
} |
||||
int weight_fd = RK_MPI_MMZ_Handle2Fd(weight_mb); |
||||
if (weight_fd < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2Fd failed!\n"); |
||||
return -1; |
||||
} |
||||
weight_mem = rknn_create_mem_from_fd(ctx, weight_fd, weight_virt, mem_size.total_weight_size, 0); |
||||
ret = rknn_set_weight_mem(ctx, weight_mem); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_set_weight_mem fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("weight mb info: virt = %p, fd = %d, size: %d\n", weight_virt, weight_fd, mem_size.total_weight_size); |
||||
|
||||
// Allocate internal memory in outside
|
||||
MB_BLK internal_mb; |
||||
rknn_tensor_mem *internal_mem; |
||||
ret = RK_MPI_MMZ_Alloc(&internal_mb, SIZE_ALIGN_128(mem_size.total_internal_size), mb_flags); |
||||
if (ret < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Alloc failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
void *internal_virt = RK_MPI_MMZ_Handle2VirAddr(internal_mb); |
||||
if (internal_virt == NULL) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2VirAddr failed!\n"); |
||||
return -1; |
||||
} |
||||
int internal_fd = RK_MPI_MMZ_Handle2Fd(internal_mb); |
||||
if (internal_fd < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2Fd failed!\n"); |
||||
return -1; |
||||
} |
||||
internal_mem = rknn_create_mem_from_fd(ctx, internal_fd, internal_virt, mem_size.total_internal_size, 0); |
||||
ret = rknn_set_internal_mem(ctx, internal_mem); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_set_internal_mem fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("internal mb info: virt = %p, fd = %d, size: %d\n", internal_virt, internal_fd, mem_size.total_internal_size); |
||||
|
||||
printf("current input tensors:\n"); |
||||
rknn_tensor_attr cur_input_attrs[io_num.n_input]; |
||||
memset(cur_input_attrs, 0, io_num.n_input * sizeof(rknn_tensor_attr)); |
||||
for (uint32_t i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
cur_input_attrs[i].index = i; |
||||
// query info
|
||||
ret = rknn_query(ctx, RKNN_QUERY_CURRENT_INPUT_ATTR, &(cur_input_attrs[i]), sizeof(rknn_tensor_attr)); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_init error! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
dump_tensor_attr(&cur_input_attrs[i]); |
||||
} |
||||
|
||||
printf("current output tensors:\n"); |
||||
rknn_tensor_attr cur_output_attrs[io_num.n_output]; |
||||
memset(cur_output_attrs, 0, io_num.n_output * sizeof(rknn_tensor_attr)); |
||||
for (uint32_t i = 0; i < io_num.n_output; i++) |
||||
{ |
||||
cur_output_attrs[i].index = i; |
||||
// query info
|
||||
ret = rknn_query(ctx, RKNN_QUERY_CURRENT_OUTPUT_ATTR, &(cur_output_attrs[i]), sizeof(rknn_tensor_attr)); |
||||
if (ret != RKNN_SUCC) |
||||
{ |
||||
printf("rknn_query fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
dump_tensor_attr(&cur_output_attrs[i]); |
||||
} |
||||
|
||||
// Allocate inputs memory in outside
|
||||
MB_BLK input_mbs[io_num.n_input]; |
||||
void *input_virts[io_num.n_input]; |
||||
int input_fds[io_num.n_input]; |
||||
for (uint32_t i = 0; i < io_num.n_input; ++i) |
||||
{ |
||||
int input_size = cur_input_attrs[i].size_with_stride; |
||||
|
||||
ret = RK_MPI_MMZ_Alloc(&input_mbs[i], SIZE_ALIGN_128(input_size), mb_flags); |
||||
if (ret < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Alloc failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
input_virts[i] = RK_MPI_MMZ_Handle2VirAddr(input_mbs[i]); |
||||
if (input_virts[i] == NULL) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2VirAddr failed!\n"); |
||||
return -1; |
||||
} |
||||
input_fds[i] = RK_MPI_MMZ_Handle2Fd(input_mbs[i]); |
||||
if (input_fds[i] < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2Fd failed!\n"); |
||||
return -1; |
||||
} |
||||
printf("input%d mb info: virt = %p, fd = %d, size = %d\n", i, input_virts[i], input_fds[i], input_size); |
||||
} |
||||
|
||||
// Allocate outputs memory in outside
|
||||
MB_BLK output_mbs[io_num.n_output]; |
||||
void *output_virts[io_num.n_output]; |
||||
int output_fds[io_num.n_output]; |
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
// default output type is depend on model, this require float32 to compute top5
|
||||
cur_output_attrs[i].type = RKNN_TENSOR_FLOAT32; |
||||
int output_size = cur_output_attrs[i].n_elems * sizeof(float); |
||||
cur_output_attrs[i].size = output_size; |
||||
ret = RK_MPI_MMZ_Alloc(&output_mbs[i], SIZE_ALIGN_128(output_size), mb_flags); |
||||
if (ret < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Alloc failed, ret: %d\n", ret); |
||||
return ret; |
||||
} |
||||
output_virts[i] = RK_MPI_MMZ_Handle2VirAddr(output_mbs[i]); |
||||
if (output_virts[i] == NULL) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2VirAddr failed!\n"); |
||||
return -1; |
||||
} |
||||
output_fds[i] = RK_MPI_MMZ_Handle2Fd(output_mbs[i]); |
||||
if (output_fds[i] < 0) |
||||
{ |
||||
printf("RK_MPI_MMZ_Handle2Fd failed!\n"); |
||||
return -1; |
||||
} |
||||
printf("output%d mb info: virt = %p, fd = %d, size = %d\n", i, output_virts[i], output_fds[i], output_size); |
||||
} |
||||
|
||||
// Create input tensor memory
|
||||
rknn_tensor_mem *input_mems[io_num.n_input]; |
||||
for (int i = 0; i < io_num.n_input; i++) |
||||
{ |
||||
// default input type is int8 (normalize and quantize need compute in outside)
|
||||
// if set uint8, will fuse normalize and quantize to npu
|
||||
cur_input_attrs[i].type = RKNN_TENSOR_UINT8; |
||||
// default fmt is NHWC, npu only support NHWC in zero copy mode
|
||||
cur_input_attrs[i].fmt = RKNN_TENSOR_NHWC; |
||||
|
||||
input_mems[i] = rknn_create_mem_from_fd(ctx, input_fds[i], input_virts[i], cur_input_attrs[i].size_with_stride, 0); |
||||
|
||||
// Copy input data to input tensor memory
|
||||
int width = cur_input_attrs[i].dims[2]; |
||||
int stride = cur_input_attrs[i].w_stride; |
||||
|
||||
if (width == stride) |
||||
{ |
||||
memcpy(input_mems[i]->virt_addr, input_data[i], input_size[i]); |
||||
} |
||||
else |
||||
{ |
||||
int height = cur_input_attrs[i].dims[1]; |
||||
int channel = cur_input_attrs[i].dims[3]; |
||||
// copy from src to dst with stride
|
||||
uint8_t *src_ptr = input_data[i]; |
||||
uint8_t *dst_ptr = (uint8_t *)input_mems[i]->virt_addr; |
||||
// width-channel elements
|
||||
int src_wc_elems = width * channel; |
||||
int dst_wc_elems = stride * channel; |
||||
for (int b = 0; b < cur_input_attrs[i].dims[0]; b++) |
||||
{ |
||||
for (int h = 0; h < height; ++h) |
||||
{ |
||||
memcpy(dst_ptr, src_ptr, src_wc_elems); |
||||
src_ptr += src_wc_elems; |
||||
dst_ptr += dst_wc_elems; |
||||
} |
||||
} |
||||
} |
||||
} |
||||
|
||||
// Create output tensor memory
|
||||
rknn_tensor_mem *output_mems[io_num.n_output]; |
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
output_mems[i] = rknn_create_mem_from_fd(ctx, output_fds[i], output_virts[i], cur_output_attrs[i].size, 0); |
||||
} |
||||
|
||||
// Set input tensor memory
|
||||
for (uint32_t i = 0; i < io_num.n_input; ++i) |
||||
{ |
||||
ret = rknn_set_io_mem(ctx, input_mems[i], &cur_input_attrs[i]); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_set_io_mem fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
} |
||||
|
||||
// Set output tensor memory
|
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
// set output memory and attribute
|
||||
ret = rknn_set_io_mem(ctx, output_mems[i], &cur_output_attrs[i]); |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn_set_io_mem fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
} |
||||
|
||||
// weight flush cache
|
||||
ret = RK_MPI_MMZ_FlushCacheEnd(weight_mb, 0, SIZE_ALIGN_128(mem_size.total_weight_size), RK_MMZ_SYNC_RW); |
||||
if (ret < 0) |
||||
{ |
||||
printf("weight_mb FlushCacheEnd fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
|
||||
// Run
|
||||
printf("Begin perf ...\n"); |
||||
for (int i = 0; i < loop_count; ++i) |
||||
{ |
||||
ret = RK_MPI_MMZ_FlushCacheEnd(internal_mb, 0, SIZE_ALIGN_128(mem_size.total_internal_size), RK_MMZ_SYNC_RW); |
||||
if (ret < 0) |
||||
{ |
||||
printf("internal_mb FlushCacheEnd fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
|
||||
for (uint32_t i = 0; i < io_num.n_input; ++i) |
||||
{ |
||||
ret = |
||||
RK_MPI_MMZ_FlushCacheEnd(input_mbs[i], 0, SIZE_ALIGN_128(cur_input_attrs[i].size_with_stride), RK_MMZ_SYNC_RW); |
||||
if (ret < 0) |
||||
{ |
||||
printf("input_mbs FlushCacheEnd fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
} |
||||
|
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
ret = RK_MPI_MMZ_FlushCacheEnd(output_mbs[i], 0, SIZE_ALIGN_128(cur_output_attrs[i].n_elems * sizeof(float)), |
||||
RK_MMZ_SYNC_RW); |
||||
if (ret < 0) |
||||
{ |
||||
printf("output_mbs FlushCacheEnd fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
} |
||||
|
||||
int64_t start_us = getCurrentTimeUs(); |
||||
ret = rknn_run(ctx, NULL); |
||||
int64_t elapse_us = getCurrentTimeUs() - start_us; |
||||
if (ret < 0) |
||||
{ |
||||
printf("rknn run error %d\n", ret); |
||||
return -1; |
||||
} |
||||
printf("%4d: Elapse Time = %.2fms, FPS = %.2f\n", i, elapse_us / 1000.f, 1000.f * 1000.f / elapse_us); |
||||
|
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
ret = RK_MPI_MMZ_FlushCacheStart(output_mbs[i], 0, SIZE_ALIGN_128(cur_output_attrs[i].n_elems * sizeof(float)), |
||||
RK_MMZ_SYNC_RW); |
||||
if (ret < 0) |
||||
{ |
||||
printf("output_mbs FlushCacheStart fail! ret=%d\n", ret); |
||||
return -1; |
||||
} |
||||
} |
||||
} |
||||
|
||||
// 处理输出结果
|
||||
for (uint32_t i = 0; i < io_num.n_output; i++) |
||||
{ |
||||
float *output = (float *)output_mems[i]->virt_addr; |
||||
int out_elems = cur_output_attrs[i].n_elems; |
||||
std::vector<std::pair<float, int>> results; |
||||
for (int i = 0; i < out_elems; i++) |
||||
{ |
||||
results.emplace_back(output[i], i); |
||||
} |
||||
std::partial_sort(results.begin(), results.begin() + 5, results.end(), std::greater<std::pair<float, int>>()); |
||||
printf(" --- Top5 ---\n"); |
||||
for (int i = 0; i < 5; i++) |
||||
{ |
||||
printf("%-3d: %.2f%%\n", results[i].second, results[i].first * 100); |
||||
} |
||||
} |
||||
|
||||
// save output
|
||||
for (uint32_t i = 0; i < io_num.n_output; i++) |
||||
{ |
||||
char output_path[PATH_MAX]; |
||||
sprintf(output_path, "%s/rt_output%d.npy", output_dir ? output_dir : ".", i); |
||||
rknn_tensor_attr cur_output_attrs_npy = cur_output_attrs[i]; |
||||
if (cur_output_attrs[i].fmt == RKNN_TENSOR_NHWC && cur_output_attrs[i].n_dims == 4) |
||||
{ |
||||
std::vector<int> axis = {0, 2, 3, 1}; |
||||
for (int j = 0; j < axis.size(); j++) |
||||
{ |
||||
cur_output_attrs_npy.dims[j] = cur_output_attrs[i].dims[axis[j]]; |
||||
} |
||||
} |
||||
save_npy(output_path, (float *)output_mems[i]->virt_addr, &cur_output_attrs_npy); |
||||
} |
||||
|
||||
// free mb blk memory
|
||||
RK_MPI_MMZ_Free(model_mb); |
||||
RK_MPI_MMZ_Free(weight_mb); |
||||
RK_MPI_MMZ_Free(internal_mb); |
||||
for (uint32_t i = 0; i < io_num.n_input; ++i) |
||||
{ |
||||
RK_MPI_MMZ_Free(input_mbs[i]); |
||||
} |
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
RK_MPI_MMZ_Free(output_mbs[i]); |
||||
} |
||||
|
||||
// Destroy rknn memory
|
||||
for (uint32_t i = 0; i < io_num.n_input; ++i) |
||||
{ |
||||
rknn_destroy_mem(ctx, input_mems[i]); |
||||
free(input_data[i]); |
||||
} |
||||
for (uint32_t i = 0; i < io_num.n_output; ++i) |
||||
{ |
||||
rknn_destroy_mem(ctx, output_mems[i]); |
||||
} |
||||
rknn_destroy_mem(ctx, weight_mem); |
||||
rknn_destroy_mem(ctx, internal_mem); |
||||
|
||||
// destroy
|
||||
rknn_destroy(ctx); |
||||
|
||||
return 0; |
||||
} |
||||
@ -0,0 +1,112 @@
@@ -0,0 +1,112 @@
|
||||
# rknn_internal_mem_reuse_demo |
||||
|
||||
## 说明 |
||||
|
||||
本工程主要用于**RKNN_FLAG_MEM_ALLOC_OUTSIDE** 及 **rknn_set_internal_mem**的使用演示。 |
||||
|
||||
RKNN_FLAG_MEM_ALLOC_OUTSIDE：主要有两方面的作用: |
||||
- 所有内存均是用户自行分配，便于对整个系统内存进行统筹安排 |
||||
- 用于内存复用，特别是针对RV1103/RV1106这种内存极为紧张的情况。 |
||||
|
||||
|
||||
假设有模型A、B 两个模型，这两个模型在设计上串行运行的，那么这两个模型的中间tensor的内存就可以复用。示例代码如下： |
||||
``` |
||||
rknn_context ctx_a, ctx_b; |
||||
|
||||
rknn_init(&ctx_a, model_path_a, 0, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL); |
||||
rknn_query(ctx_a, RKNN_QUERY_MEM_SIZE, &mem_size_a, sizeof(mem_size_a)); |
||||
|
||||
rknn_init(&ctx_b, model_path_b, 0, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL); |
||||
rknn_query(ctx_b, RKNN_QUERY_MEM_SIZE, &mem_size_b, sizeof(mem_size_b)); |
||||
|
||||
max_internal_size = MAX(mem_size_a.total_internal_size, mem_size_b.total_internal_size); |
||||
internal_mem_max = rknn_create_mem(ctx_a, max_internal_size); |
||||
|
||||
internal_mem_a = rknn_create_mem_from_fd(ctx_a, internal_mem_max->fd, |
||||
internal_mem_max->virt_addr, mem_size_a.total_internal_size, 0); |
||||
rknn_set_internal_mem(ctx_a, internal_mem_a); |
||||
|
||||
internal_mem_b = rknn_create_mem_from_fd(ctx_b, internal_mem_max->fd, |
||||
internal_mem_max->virt_addr, mem_size_a.total_internal_size, 0); |
||||
rknn_set_internal_mem(ctx_b, internal_mem_b); |
||||
``` |
||||
|
||||
|
||||
|
||||
注意：本工程使用了上级目录的rknn_mobilenet_demo及rknn_yolov5_demo两个工程的rknn模型，编译之前请确保其存在。 |
||||
|
||||
## Android Demo |
||||
|
||||
### 编译 |
||||
|
||||
根据指定平台修改 `build-android_<TARGET_PLATFORM>.sh`中的Android NDK的路径 `ANDROID_NDK_PATH`，<TARGET_PLATFORM>可以是RK3566_RK3568、RK3562或RK3588 例如修改成： |
||||
|
||||
```sh |
||||
ANDROID_NDK_PATH=~/opt/tool_chain/android-ndk-r17 |
||||
``` |
||||
|
||||
然后执行： |
||||
|
||||
```sh |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
### 推送执行文件到板子 |
||||
|
||||
连接板子的usb口到PC,将整个demo目录到 `/data`: |
||||
|
||||
```sh |
||||
adb root |
||||
adb remount |
||||
adb push install/rknn_internal_mem_reuse_demo_Android /data/ |
||||
``` |
||||
|
||||
### 运行 |
||||
|
||||
```sh |
||||
adb shell |
||||
cd /data/rknn_internal_mem_reuse_demo_Android/ |
||||
|
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_internal_mem_reuse_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn model/bus.jpg model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/cat_224x224.jpg |
||||
``` |
||||
|
||||
## Aarch64 Linux Demo |
||||
|
||||
### 编译 |
||||
|
||||
根据指定平台修改 `build-linux_<TARGET_PLATFORM>.sh`中的交叉编译器所在目录的路径 `GCC_COMPILER`，例如修改成 |
||||
|
||||
```sh |
||||
export GCC_COMPILER=prebuilts/gcc/linux-x86/aarch64/gcc-buildroot-9.3.0-2020.03-x86_64_aarch64-rockchip-linux-gnu/bin/aarch64-linux |
||||
``` |
||||
|
||||
然后执行： |
||||
|
||||
```sh |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
### 推送执行文件到板子 |
||||
|
||||
|
||||
将 install/rknn_internal_mem_reuse_demo_Linux 拷贝到板子的/userdata/目录. |
||||
|
||||
- 如果使用rockchip的EVB板子，可以使用adb将文件推到板子上： |
||||
|
||||
``` |
||||
adb push install/rknn_internal_mem_reuse_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果使用其他板子，可以使用scp等方式将install/rknn_internal_mem_reuse_demo_Linux拷贝到板子的/userdata/目录 |
||||
|
||||
### 运行 |
||||
|
||||
```sh |
||||
adb shell |
||||
cd /userdata/rknn_internal_mem_reuse_demo_Linux/ |
||||
|
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_internal_mem_reuse_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn model/bus.jpg model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/cat_224x224.jpg |
||||
``` |
||||
|
||||
@ -0,0 +1,74 @@
@@ -0,0 +1,74 @@
|
||||
rknn_matmul_api_demo是一个使用matmul C API在NPU上执行int8矩阵乘法的示例。 |
||||
|
||||
|
||||
用法: |
||||
``` |
||||
./rknn_matmul_api_demo |
||||
``` |
||||
以下 <TARGET_PLATFORM> 表示RK3566_RK3568、RK3562或RK3588。 |
||||
|
||||
# Aarch64 Linux 示例 |
||||
## 编译 |
||||
|
||||
将`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`修改成交叉编译器路径, 然后执行 |
||||
|
||||
|
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
## 安装 |
||||
|
||||
将 install/rknn_matmul_api_demo_Linux 拷贝到设备上。 |
||||
|
||||
- 如果使用Rockchip的EVB板，可以使用以下命令： |
||||
|
||||
连接设备并将程序传输到`/userdata` |
||||
|
||||
``` |
||||
adb push install/rknn_matmul_api_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果你的板子有sshd服务，可以使用scp命令或者其他方式将程序和模型传输到板子上。 |
||||
|
||||
## 运行 |
||||
|
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_matmul_api_demo_Linux/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_matmul_api_demo |
||||
``` |
||||
|
||||
# Android 示例 |
||||
## 编译 |
||||
|
||||
将`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`修改成平台对应的NDK，然后执行 |
||||
|
||||
|
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
连接设备并将程序传输到`/data` |
||||
|
||||
``` |
||||
adb push install/rknn_matmul_api_demo_Android /data/ |
||||
``` |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /data/rknn_matmul_api_demo_Android/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_matmul_api_demo |
||||
``` |
||||
@ -0,0 +1,185 @@
@@ -0,0 +1,185 @@
|
||||
#ifndef _RKNPU2_RKNN_MATMUL_API_DEMO_H_ |
||||
#define _RKNPU2_RKNN_MATMUL_API_DEMO_H_ |
||||
|
||||
namespace rknpu2 { |
||||
|
||||
using ushort = unsigned short; |
||||
|
||||
typedef union suf32 |
||||
{ |
||||
int i; |
||||
unsigned u; |
||||
float f; |
||||
} suf32; |
||||
|
||||
class float16 |
||||
{ |
||||
public: |
||||
float16() {} |
||||
explicit float16(float x) { w = bits(x); } |
||||
|
||||
operator float() const |
||||
{ |
||||
suf32 out; |
||||
|
||||
unsigned t = ((w & 0x7fff) << 13) + 0x38000000; |
||||
unsigned sign = (w & 0x8000) << 16; |
||||
unsigned e = w & 0x7c00; |
||||
|
||||
out.u = t + (1 << 23); |
||||
out.u = (e >= 0x7c00 ? t + 0x38000000 : e == 0 ? (static_cast<void>(out.f -= 6.103515625e-05f), out.u) : t) | sign; |
||||
return out.f; |
||||
} |
||||
|
||||
static float16 fromBits(ushort b) |
||||
{ |
||||
float16 result; |
||||
result.w = b; |
||||
return result; |
||||
} |
||||
static float16 zero() |
||||
{ |
||||
float16 result; |
||||
result.w = (ushort)0; |
||||
return result; |
||||
} |
||||
ushort bits() const { return w; } |
||||
|
||||
static ushort bits(float x) |
||||
{ |
||||
suf32 in; |
||||
in.f = x; |
||||
unsigned sign = in.u & 0x80000000; |
||||
in.u ^= sign; |
||||
ushort w; |
||||
|
||||
if (in.u >= 0x47800000) |
||||
w = (ushort)(in.u > 0x7f800000 ? 0x7e00 : 0x7c00); |
||||
else { |
||||
if (in.u < 0x38800000) { |
||||
in.f += 0.5f; |
||||
w = (ushort)(in.u - 0x3f000000); |
||||
} else { |
||||
unsigned t = in.u + 0xc8000fff; |
||||
w = (ushort)((t + ((in.u >> 13) & 1)) >> 13); |
||||
} |
||||
} |
||||
|
||||
w = (ushort)(w | (sign >> 16)); |
||||
|
||||
return w; |
||||
} |
||||
|
||||
float16& operator=(float x) |
||||
{ |
||||
w = bits(x); |
||||
return *this; |
||||
} |
||||
|
||||
float16& operator+=(float x) |
||||
{ |
||||
w = bits(float() + x); |
||||
return *this; |
||||
} |
||||
|
||||
float16& operator/(float x) |
||||
{ |
||||
w = bits(float() / x); |
||||
return *this; |
||||
} |
||||
|
||||
inline bool is_nan() const { return ((w & 0x7c00u) == 0x7c00u) && ((w & 0x03ffu) != 0x0000u); } |
||||
|
||||
inline bool greater(const float16& x) const |
||||
{ |
||||
bool sign = w & 0x8000; |
||||
bool sign_x = x.w & 0x8000; |
||||
if (sign) { |
||||
if (sign_x) |
||||
return w < x.w; |
||||
else |
||||
return false; |
||||
} else { |
||||
if (sign_x) |
||||
/* Signed zeros are equal, have to check for it */ |
||||
return (w != 0 || x.w != 0x8000); |
||||
else |
||||
return w > x.w; |
||||
} |
||||
return false; |
||||
} |
||||
|
||||
inline bool less(const float16& x) const |
||||
{ |
||||
bool sign = w & 0x8000; |
||||
bool sign_x = x.w & 0x8000; |
||||
if (sign) { |
||||
if (sign_x) |
||||
return w > x.w; |
||||
else |
||||
/* Signed zeros are equal, have to check for it */ |
||||
return (w != 0x8000 || x.w != 0); |
||||
} else { |
||||
if (sign_x) |
||||
return false; |
||||
else |
||||
return w < x.w; |
||||
} |
||||
return false; |
||||
} |
||||
|
||||
inline bool operator>(const float16& x) const |
||||
{ |
||||
if (is_nan() || x.is_nan()) { |
||||
return false; |
||||
} |
||||
return greater(x); |
||||
} |
||||
|
||||
inline bool operator<(const float16& x) const |
||||
{ |
||||
if (is_nan() || x.is_nan()) { |
||||
return false; |
||||
} |
||||
return less(x); |
||||
} |
||||
|
||||
inline bool operator>=(const float16& x) const |
||||
{ |
||||
if (is_nan() || x.is_nan()) { |
||||
return false; |
||||
} |
||||
return !less(x); |
||||
} |
||||
|
||||
inline bool operator<=(const float16& x) const |
||||
{ |
||||
if (is_nan() || x.is_nan()) { |
||||
return false; |
||||
} |
||||
return !greater(x); |
||||
} |
||||
|
||||
inline bool operator==(const float16& x) const |
||||
{ |
||||
/*
|
||||
* The equality cases are as follows: |
||||
* - If either value is NaN, never equal. |
||||
* - If the values are equal, equal. |
||||
* - If the values are both signed zeros, equal. |
||||
*/ |
||||
if (is_nan() || x.is_nan()) { |
||||
return false; |
||||
} |
||||
return (w == x.w || ((w | x.w) & 0x7fff) == 0); |
||||
} |
||||
|
||||
inline bool operator!=(const float16& x) const { return !((*this) == x); } |
||||
|
||||
protected: |
||||
ushort w = 0; |
||||
}; |
||||
|
||||
} // namespace rknn
|
||||
|
||||
#endif /* _RKNPU2_RKNN_MATMUL_API_DEMO_H_ */ |
||||
@ -0,0 +1,68 @@
@@ -0,0 +1,68 @@
|
||||
以下 <TARGET_PLATFORM> 表示RK3566_RK3568、RK3562或RK3588。 |
||||
|
||||
# rknn模型来源说明 |
||||
各个平台的rknn模型来源于RKNN-Toolkit2的的examples/tflite/mobilenet_v1示例。 |
||||
|
||||
# Aarch64 Linux 示例 |
||||
## 编译 |
||||
|
||||
将`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`修改成交叉编译器路径, 然后执行 |
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
将 install/rknn_api_demo_Linux 拷贝到设备上。 |
||||
|
||||
- 如果使用Rockchip的EVB板，可以使用以下命令： |
||||
|
||||
连接设备并将程序和模型传输到`/userdata` |
||||
|
||||
``` |
||||
adb push install/rknn_mobilenet_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果你的板子有sshd服务，可以使用scp命令或者其他方式将程序和模型传输到板子上。 |
||||
|
||||
## 运行 |
||||
|
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_mobilenet_demo_Linux/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_mobilenet_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
|
||||
# Android 示例 |
||||
## 编译 |
||||
|
||||
将`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`修改成平台对应的NDK，然后执行 |
||||
|
||||
``` |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
连接设备并将程序和模型传输到`/data` |
||||
|
||||
``` |
||||
adb push install/rknn_mobilenet_demo_Android /data/ |
||||
``` |
||||
|
||||
## 运行 |
||||
|
||||
``` |
||||
adb shell |
||||
cd /data/rknn_mobilenet_demo_Android/ |
||||
``` |
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_mobilenet_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg |
||||
``` |
||||
@ -0,0 +1,69 @@
@@ -0,0 +1,69 @@
|
||||
|
||||
以下 <TARGET_PLATFORM> 表示RK3566_RK3568、RK3562或RK3588。 |
||||
# Aarch64 Linux 示例 |
||||
## 编译 |
||||
|
||||
将`build-linux_<TARGET_PLATFORM>.sh`中的`GCC_COMPILER`修改成交叉编译器路径, 然后执行 |
||||
|
||||
``` |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
将 install/rknn_api_demo_Linux 拷贝到设备上。 |
||||
|
||||
- 如果使用Rockchip的EVB板，可以使用以下命令： |
||||
|
||||
连接设备并将程序和模型传输到`/userdata` |
||||
|
||||
``` |
||||
adb push install/rknn_multiple_input_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果你的板子有sshd服务，可以使用scp命令或者其他方式将程序和模型传输到板子上。 |
||||
|
||||
## 运行 |
||||
|
||||
|
||||
``` |
||||
adb shell |
||||
cd /userdata/rknn_multiple_input_demo_Linux/ |
||||
``` |
||||
|
||||
|
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_multiple_input_demo model/<TARGET_PLATFORM>/multiple_input_demo.rknn model/input1.bin#model/input2.bin |
||||
``` |
||||
|
||||
# Android 示例 |
||||
## 编译 |
||||
|
||||
将`build-android_<TARGET_PLATFORM>.sh`中的`ANDROID_NDK_PATH`修改成平台对应的NDK，然后执行 |
||||
|
||||
``` |
||||
./build-android.sh |
||||
``` |
||||
|
||||
## 安装 |
||||
|
||||
connect device and push build output into `/data` |
||||
|
||||
``` |
||||
adb push install/rknn_multiple_input_demo_Android /data/ |
||||
``` |
||||
|
||||
## 运行 |
||||
``` |
||||
adb shell |
||||
cd /data/rknn_multiple_input_demo_Android/ |
||||
``` |
||||
|
||||
|
||||
|
||||
``` |
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_multiple_input_demo model/<TARGET_PLATFORM>/multiple_input_demo.rknn model/input1.bin#model/input2.bin |
||||
``` |
||||
@ -0,0 +1,78 @@
@@ -0,0 +1,78 @@
|
||||
# 简介 |
||||
- rknn_yolov5_android_apk_demo 是RK3566_RK3568, RK3562或RK3588上如何调用NPU的demo，该demo的基础模型是yolov5s |
||||
|
||||
|
||||
|
||||
# 使用说明 |
||||
|
||||
- 可以使用android studio 2021.2.1 （Windows）或者android studio 2021.3.1（Linux/macOS） 编译该工程 |
||||
- yolov5s.rknn是使用rknn toolkit2将yolov5s.onnx转换而来，其中激活函数用relu替换silu，具体转换方法参考 https://github.com/airockchip/rknn_model_zoo/tree/main/models/CV/object_detection/yolo |
||||
|
||||
|
||||
|
||||
# 代码说明 |
||||
|
||||
## 代码分为两大部分： |
||||
- JAVA: com.rockchip.gpadc.demo: 读取camera输入，并调用jni进行inference，并将结果显示出来 |
||||
|
||||
- JNI: 调用rknnrt进行实际inference |
||||
|
||||
|
||||
|
||||
# 权限 |
||||
运行本程序需要Camera及EXTERNAL_STORAGE读写权限 |
||||
|
||||
|
||||
# FAQ |
||||
|
||||
## 编译时出现“files found with path 'lib/arm64-v8a/xxx.so' from inputs:”类似错误 |
||||
``` |
||||
Execution failed for task ':app:mergeDebugNativeLibs'. |
||||
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeNativeLibsTask$MergeNativeLibsTaskWorkAction |
||||
> 2 files found with path 'lib/arm64-v8a/librga.so' from inputs: |
||||
- rknpu2/examples/rknn_yolov5_android_apk_demo/app/build/intermediates/merged_jni_libs/debug/out/arm64-v8a/librga.so |
||||
- rknpu2/examples/rknn_yolov5_android_apk_demo/app/build/intermediates/cxx/Debug/3f40b265/obj/arm64-v8a/librga.so |
||||
If you are using jniLibs and CMake IMPORTED targets, see |
||||
https://developer.android.com/r/tools/jniLibs-vs-imported-targets |
||||
|
||||
* Try: |
||||
> Run with --info or --debug option to get more log output. |
||||
> Run with --scan to get full insights. |
||||
``` |
||||
或者 |
||||
``` |
||||
2 files found with path 'lib/arm64-v8a/xxx.so' from inputs: |
||||
- rknn_yolov5_android_apk_demo/app/build/intermediates/merged_jni_libs/debug/out/arm64-v8a/librga.so |
||||
- rknn_yolov5_android_apk_demo/app/build/intermediates/cxx/Debug/3f40b265/obj/arm64-v8a/librga.so |
||||
If you are using jniLibs and CMake IMPORTED targets, see |
||||
https://developer.android.com/r/tools/jniLibs-vs-imported-targets |
||||
``` |
||||
则需要添加 app/build.gradle中的"jniLibs.srcDirs = ['libs']"，具体原因参考“https://developer.android.com/r/tools/jniLibs-vs-imported-targets” |
||||
|
||||
|
||||
|
||||
## app启动时，出现"E/SurfaceView: Exception configuring surface"的错误 |
||||
|
||||
``` |
||||
D/rkyolo: camera facing: 1 |
||||
V/rkyolo: Camera Supported Preview Size = 160x90 |
||||
V/rkyolo: Camera Supported Preview Size = 320x180 |
||||
V/rkyolo: Camera Supported Preview Size = 432x240 |
||||
V/rkyolo: Camera Supported Preview Size = 640x360 |
||||
V/rkyolo: Camera Supported Preview Size = 800x448 |
||||
V/rkyolo: Camera Supported Preview Size = 864x480 |
||||
V/rkyolo: Camera Supported Preview Size = 1024x576 |
||||
V/rkyolo: Camera Supported Preview Size = 1280x720 |
||||
V/rkyolo: Camera Supported Preview Size = 1600x896 |
||||
V/rkyolo: Camera Supported Preview Size = 1920x1080 |
||||
E/SurfaceView: Exception configuring surface |
||||
java.lang.RuntimeException: setParameters failed |
||||
at android.hardware.Camera.native_setParameters(Native Method) |
||||
at android.hardware.Camera.setParameters(Camera.java:2068) |
||||
at com.rockchip.gpadc.demo.MainActivity.setCameraParameters(MainActivity.java:295) |
||||
at com.rockchip.gpadc.demo.MainActivity.startCamera(MainActivity.java:245) |
||||
at com.rockchip.gpadc.demo.MainActivity.access$300(MainActivity.java:47) |
||||
at com.rockchip.gpadc.demo.MainActivity$TSurfaceHolderCallback.surfaceCreated(MainActivity.java:199) |
||||
``` |
||||
|
||||
是因为app设置的分辨率该camera不支持，需要修改"app/src/main/java/com/rockchip/gpadc/demo/rga/HALDefine.java"中CAMERA_PREVIEW_WIDTH及CAMERA_PREVIEW_HEIGHT为摄像头支持的分辨率。 |
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,127 @@
@@ -0,0 +1,127 @@
|
||||
# Yolo-v5 demo |
||||
|
||||
## 导出rknn模型步骤 |
||||
|
||||
请参考 https://github.com/airockchip/rknn_model_zoo/tree/main/models/CV/object_detection/yolo |
||||
|
||||
|
||||
## 注意事项 |
||||
|
||||
1. 使用rknn-toolkit2版本大于等于1.4.0。 |
||||
2. 切换成自己训练的模型时，请注意对齐anchor等后处理参数，否则会导致后处理解析出错。 |
||||
3. 官网和rk预训练模型都是检测80类的目标，如果自己训练的模型,需要更改include/postprocess.h中的OBJ_CLASS_NUM以及NMS_THRESH,BOX_THRESH后处理参数。 |
||||
4. demo需要librga.so的支持,编译使用请参考 https://github.com/airockchip/librga |
||||
5. 由于硬件限制，该demo的模型默认把 yolov5 模型的后处理部分，移至cpu实现。本demo附带的模型均使用relu为激活函数，相比silu激活函数精度略微下降，性能大幅上升。 |
||||
|
||||
|
||||
|
||||
## Android Demo |
||||
|
||||
### 编译 |
||||
|
||||
根据指定平台修改 `build-android_<TARGET_PLATFORM>.sh`中的Android NDK的路径 `ANDROID_NDK_PATH`，<TARGET_PLATFORM>可以是RK3566_RK3568, RK3562或RK3588 例如修改成： |
||||
|
||||
```sh |
||||
ANDROID_NDK_PATH=~/opt/tool_chain/android-ndk-r17 |
||||
``` |
||||
|
||||
然后执行： |
||||
|
||||
```sh |
||||
./build-android_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
### 推送执行文件到板子 |
||||
|
||||
连接板子的usb口到PC,将整个demo目录到 `/data`: |
||||
|
||||
```sh |
||||
adb root |
||||
adb remount |
||||
adb push install/rknn_yolov5_demo /data/ |
||||
``` |
||||
|
||||
### 运行 |
||||
|
||||
```sh |
||||
adb shell |
||||
cd /data/rknn_yolov5_demo/ |
||||
|
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_yolov5_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn model/bus.jpg |
||||
``` |
||||
|
||||
## Aarch64 Linux Demo |
||||
|
||||
### 编译 |
||||
|
||||
根据指定平台修改 `build-linux_<TARGET_PLATFORM>.sh`中的交叉编译器所在目录的路径 `TOOL_CHAIN`，例如修改成 |
||||
|
||||
```sh |
||||
export TOOL_CHAIN=~/opt/tool_chain/gcc-9.3.0-x86_64_aarch64-linux-gnu/host |
||||
``` |
||||
|
||||
然后执行： |
||||
|
||||
```sh |
||||
./build-linux_<TARGET_PLATFORM>.sh |
||||
``` |
||||
|
||||
### 推送执行文件到板子 |
||||
|
||||
|
||||
将 install/rknn_yolov5_demo_Linux 拷贝到板子的/userdata/目录. |
||||
|
||||
- 如果使用rockchip的EVB板子，可以使用adb将文件推到板子上： |
||||
|
||||
``` |
||||
adb push install/rknn_yolov5_demo_Linux /userdata/ |
||||
``` |
||||
|
||||
- 如果使用其他板子，可以使用scp等方式将install/rknn_yolov5_demo_Linux拷贝到板子的/userdata/目录 |
||||
|
||||
### 运行 |
||||
|
||||
```sh |
||||
adb shell |
||||
cd /userdata/rknn_yolov5_demo_Linux/ |
||||
|
||||
export LD_LIBRARY_PATH=./lib |
||||
./rknn_yolov5_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn model/bus.jpg |
||||
``` |
||||
|
||||
Note: Try searching the location of librga.so and add it to LD_LIBRARY_PATH if the librga.so is not found on the lib folder. |
||||
Using the following commands to add to LD_LIBRARY_PATH. |
||||
|
||||
```sh |
||||
export LD_LIBRARY_PATH=./lib:<LOCATION_LIBRGA.SO> |
||||
``` |
||||
|
||||
## 视频流Demo运行命令参考如下： |
||||
- h264视频 |
||||
``` |
||||
./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn xxx.h264 264 |
||||
``` |
||||
注意需要使用h264码流视频，可以使用如下命令转换得到： |
||||
``` |
||||
ffmpeg -i xxx.mp4 -vcodec h264 out.h264 |
||||
``` |
||||
|
||||
- h265视频 |
||||
``` |
||||
./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn xxx.hevc 265 |
||||
``` |
||||
注意需要使用h265码流视频，可以使用如下命令转换得到： |
||||
``` |
||||
ffmpeg -i xxx.mp4 -vcodec hevc out.hevc |
||||
``` |
||||
- rtsp视频流 |
||||
``` |
||||
./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn <RTSP_URL> 265 |
||||
``` |
||||
|
||||
### 注意 |
||||
|
||||
- 需要根据系统的rga驱动选择正确的librga库，具体依赖请参考： https://github.com/airockchip/librga |
||||
- **rk3562 目前仅支持h264视频流** |
||||
- **rtsp 视频流Demo仅在Linux系统上支持，Android上目前还不支持** |
||||
@ -1,16 +1,17 @@
@@ -1,16 +1,17 @@
|
||||
# onnx模型 |
||||
# onnx model |
||||
|
||||
* onnx_models/yolov5s_rm_transpose.onnx |
||||
模型基于官方yolov5s基础魔改而来，与rknn-toolkit2工具SDK中examples/onnx/yolov5/yolov5s.onnx相同，用于rknpu2的examples/rknn_yolov5_demo |
||||
* onnx_models/yolov5s_relu.onnx |
||||
|
||||
# 转换rknn模型 |
||||
The model comes from https://github.com/airockchip/rknn_model_zoo |
||||
|
||||
1. 将onnx2rknn.py以下的参数修改成对应的平台，例如RK3566_RK3568模型，修改为： |
||||
# convert rknn model |
||||
|
||||
```C++ |
||||
1. Modify the following parameters of onnx2rknn.py to the corresponding platform, such as the RK3566_RK3568 model, modify as follows: |
||||
|
||||
```python |
||||
platform="rk3566" |
||||
``` |
||||
|
||||
2. 执行python onnx2rknn.py |
||||
2. Execute python onnx2rknn.py |
||||
|
||||
3. rknn模型生成在rknn_models目录 |
||||
3. The rknn model is generated in the rknn_models directory |
||||
@ -0,0 +1,18 @@
@@ -0,0 +1,18 @@
|
||||
# onnx模型 |
||||
|
||||
* onnx_models/yolov5s_relu.onnx |
||||
|
||||
|
||||
模型来源：https://github.com/airockchip/rknn_model_zoo |
||||
|
||||
# 转换rknn模型 |
||||
|
||||
1. 将onnx2rknn.py以下的参数修改成对应的平台，例如RK3566_RK3568模型，修改为： |
||||
|
||||
```python |
||||
platform="rk3566" |
||||
``` |
||||
|
||||
2. 执行python onnx2rknn.py |
||||
|
||||
3. rknn模型生成在rknn_models目录 |
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue