| # Simple ONNX RT C++ Example for the NPU | |
| A simple C++ program which runs an ONNX model on the CPU and then on the NPU (if the system configuration allows it). | |
| ## Requirements | |
| To build the example, the following software must be installed: | |
| - Ryzen AI 1.4 | |
| - Cmake | |
| - Visual Studio 2022 | |
| ## Instructions | |
| - Build the example | |
| ``` | |
| conda activate <RAI env> | |
| compile.bat | |
| ``` | |
| - Run the example | |
| ``` | |
| run.bat | |
| ``` | |
| Note: it is not necessary to run the example from within the RyzenAI conda environment. | |
| ## Notes | |
| - The CMakeLists.txt copies all the required files (DLLs, XCLBINs) from the RyzenAI installation tree into the folder where the executable is created. With this approach, there is no runtime dependency on the RyzenAI installation tree. Once built, the example can be run on machines without RyzenAI installed. | |
| - This example shows that it is possible to run without any dependency on Python. | |
| - The example uses the test_model.onnx model from the quicktest folder. | |
| - Like the Python quicktest, this example runs the model with random data and doesn't test the output results. | |
| - The application uses the VAI-EP session options to specify the XCLBIN, and it explicitly unsets the XLNX_VART_FIRMWARE and XLNX_TARGET_NAME env vars. | |
| - The example uses the helper functions in `npu_util.cpp` to check whether VAI-EP 1.4 can be used on the deployment machine, and to determine the type of NPU present (PHX/HPT or STX/KRK). | |
| - The test program only supports models with exactly 1 input node and 1 output node. But the code can easily be extended to support an arbitrary number of nodes if need be. | |
| ## Sample Output | |
| Sample output of a successful run: | |
| ``` | |
| >run.bat | |
| ------------------------------------------------------- | |
| Running quicktest on CPU | |
| ------------------------------------------------------- | |
| Creating ORT env | |
| Initializing session options | |
| Creating ONNX Session | |
| ONNX model : c:\temp\quicktest_cpp\build\Release\test_model.onnx | |
| input -1x3x32x32 | |
| output -1x10 | |
| Dynamic batch size detected. Setting batch size to 1. | |
| Running the model | |
| ------------------------------------------------------- | |
| Test PASSED! | |
| ------------------------------------------------------- | |
| ------------------------------------------------------- | |
| Performing compatibility check for VitisAI EP 1.4 | |
| ------------------------------------------------------- | |
| - NPU Device ID : 0x1502 | |
| - NPU Device Name : NPU Compute Accelerator Device | |
| - NPU Driver Version: 32.0.203.252 | |
| Environment compatible for VitisAI EP | |
| ------------------------------------------------------- | |
| Running quicktest on NPU | |
| ------------------------------------------------------- | |
| Creating ORT env | |
| Initializing session options | |
| Configuring VAI EP | |
| Creating ONNX Session | |
| WARNING: Logging before InitGoogleLogging() is written to STDERR | |
| I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1046] Vitis AI EP Load ONNX Model Success | |
| I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1047] Graph Input Node Name/Shape (1) | |
| I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1051] input : [-1x3x32x32] | |
| I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1057] Graph Output Node Name/Shape (1) | |
| I20250221 12:48:40.532923 76424 vitisai_compile_model.cpp:1061] output : [-1x10] | |
| [Vitis AI EP] No. of Operators : CPU 2 NPU 18e | |
| [Vitis AI EP] No. of Subgraphs : CPU 1 NPU 1 Actually running on NPU 1 | |
| ONNX model : C:\temp\quicktest_cpp\build\Release\test_model.onnx | |
| input -1x3x32x32 | |
| output -1x10 | |
| Dynamic batch size detected. Setting batch size to 1. | |
| Running the model | |
| ------------------------------------------------------- | |
| Test PASSED! | |
| ------------------------------------------------------- | |
| ``` | |