File size: 3,804 Bytes
0db70ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
# Simple ONNX RT C++ Example for the NPU
A simple C++ program which runs an ONNX model on the CPU and then on the NPU (if the system configuration allows it).
## Requirements
To build the example, the following software must be installed:
- Ryzen AI 1.4
- Cmake
- Visual Studio 2022
## Instructions
- Build the example
```
conda activate <RAI env>
compile.bat
```
- Run the example
```
run.bat
```
Note: it is not necessary to run the example from within the RyzenAI conda environment.
## Notes
- The CMakeLists.txt copies all the required files (DLLs, XCLBINs) from the RyzenAI installation tree into the folder where the executable is created. With this approach, there is no runtime dependency on the RyzenAI installation tree. Once built, the example can be run on machines without RyzenAI installed.
- This example shows that it is possible to run without any dependency on Python.
- The example uses the test_model.onnx model from the quicktest folder.
- Like the Python quicktest, this example runs the model with random data and doesn't test the output results.
- The application uses the VAI-EP session options to specify the XCLBIN, and it explicitly unsets the XLNX_VART_FIRMWARE and XLNX_TARGET_NAME env vars.
- The example uses the helper functions in `npu_util.cpp` to check whether VAI-EP 1.4 can be used on the deployment machine, and to determine the type of NPU present (PHX/HPT or STX/KRK).
- The test program only supports models with exactly 1 input node and 1 output node. But the code can easily be extended to support an arbitrary number of nodes if need be.
## Sample Output
Sample output of a successful run:
```
>run.bat
-------------------------------------------------------
Running quicktest on CPU
-------------------------------------------------------
Creating ORT env
Initializing session options
Creating ONNX Session
ONNX model : c:\temp\quicktest_cpp\build\Release\test_model.onnx
input -1x3x32x32
output -1x10
Dynamic batch size detected. Setting batch size to 1.
Running the model
-------------------------------------------------------
Test PASSED!
-------------------------------------------------------
-------------------------------------------------------
Performing compatibility check for VitisAI EP 1.4
-------------------------------------------------------
- NPU Device ID : 0x1502
- NPU Device Name : NPU Compute Accelerator Device
- NPU Driver Version: 32.0.203.252
Environment compatible for VitisAI EP
-------------------------------------------------------
Running quicktest on NPU
-------------------------------------------------------
Creating ORT env
Initializing session options
Configuring VAI EP
Creating ONNX Session
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1046] Vitis AI EP Load ONNX Model Success
I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1047] Graph Input Node Name/Shape (1)
I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1051] input : [-1x3x32x32]
I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1057] Graph Output Node Name/Shape (1)
I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1061] output : [-1x10]
[Vitis AI EP] No. of Operators : CPU 2 NPU 18e
[Vitis AI EP] No. of Subgraphs : CPU 1 NPU 1 Actually running on NPU 1
ONNX model : C:\temp\quicktest_cpp\build\Release\test_model.onnx
input -1x3x32x32
output -1x10
Dynamic batch size detected. Setting batch size to 1.
Running the model
-------------------------------------------------------
Test PASSED!
-------------------------------------------------------
```
|