Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure
View the Project on GitHub onnx/onnx-mlir
This project is maintained by onnx
Hosted on GitHub Pages — Theme by orderedlist
Neural Network Processing Assist Facility (NNPA) is implemented on processor units of IBM z16. Onnx-mlir can use it via IBM Z Deep Neural Network Library (zDNN). Building and lit tests runs on other IBM Z systems(eg. z15), but numerical tests need to run on z16.
Add following CMake option to build onnx-mlir for NNPA. Regarding build command for Linux OS, see here
-DONNX_MLIR_ACCELERATORS=NNPA
The lit tests for NNPA are included in test/mlir/accelerators/nnpa
. When building onnx-mlir for NNPA, these lit tests also run with the following same command with CPU.
cmake --build . --target check-onnx-lit
Numerical tests for NNPA are provided in test/accelerators/NNPA/numerical
. Currently tests for Conv2D, MatMul2D, Gemm, LSTM, and GRU are provided and run using following command. These tests can check if a zDNN instruction is included in the generated shared library using an environment variable TEST_INSTRUCTION
. Also, to check the accuracy of the results, ATOL and RTOL can be set by using environment TEST_ATOL
and TEST_RTOL
. An environment variable TEST_DATARANGE
are provided to set lower and upper bound of data range. They can be set “
cmake --build . --config Release --target check-onnx-numerical-nnpa
These tests uses the same test code with numerical tests for CPU (test/modellib
and test/numerial
), but uses different cmake file(test/accelerator/NNPA/numerical/CMakeLists.txt
).
Since Conv2D in zDNN library only supports the case where dilation equals to one, dilation is always set to one in the test. Also, padding types are set as VALID and SAME_UPPER since they are only suppored. All dimensions are static since dynamic height and weight dimension are currently not supported. These configurations are set automatically when using --maccel=NNPA
, which are equivalent to manually setting the environment variable TEST_CONFIG
to “-dim=static -dilation=1 -padding=valid_upper”.
alpha
and beta
in Gemm are always one, which are supported case by zDNN library. These configurations are set automatically when using --maccel=NNPA
, which are equivalent to manually setting the environment variable TEST_CONFIG
to “-alpha=1 -beta=1”.
Peephole tensor is not tested since LSTM in zDNN library does not support it. These configurations are set automatically when using --maccel=NNPA
, which are equivalent to manually setting the environment variable TEST_CONFIG
to “-peephole=0”.
GRU of zDNN library supports only the case where the linear transformation is applied before multiplying by the output of the reset gata. It is configured automatically when using --maccel=NNPA
, which are equivalent to manually setting the environment variable TEST_CONFIG
to “-linearBeforeReset=1”.
Backend tests for NNPA are provided in test/accelerators/NNPA/backend
. It can be run with following command. Only test cases supported by zDNN runs as listed in test/accelerators/NNPA/backend/CMakeLists.txt
.
cmake --build . --config Release --target check-onnx-backend-nnpa
ATOL and RTOL for NNPA are set using environment variables TEST_ATOL
and TEST_RTOL
in the CMakeLists.txt
.
Also, the environment variables TEST_INSTRUCTION_CHECK
and TEST_CASE_BY_USER
allow you to check if the NNPA instruction is generated in the shared library. In CMakeLists.txt
, TEST_INSTRUCTION_CHECK
is set to true and TEST_CASE_BY_USER
contains the test case and instruction name. If the instruction name is not found in the shared library, the test will fail.