Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

View the Project on GitHub onnx/onnx-mlir


Using PyRuntime
Perform Inference Using ONNX-MLIR Runtime API


ONNX Dialect
OMTensor C99 Runtime API
OMTensorList C99 Runtime API
Generate ONNX Dialect
About Documentation


Testing Guidelines


debug.py - Debug Numerical Errors
DocCheck - Handling Necessary Code Duplication

This project is maintained by onnx

Hosted on GitHub Pages — Theme by orderedlist


In onnx-mlir, there are three types of tests to ensure correctness of implementation:

ONNX Backend Tests

Backend tests are end-to-end tests for onnx-mlir based on onnx node tests. To invoke the test, use the following command:

cmake --build . --config Release --target check-onnx-backend

Packages, such as third_party/onnx and ssl, needs to be installed to run the backend test.

The node tests in onnx that will be run by check-onnx-backend is defined by variable test_to_enable in test/backend/test.py. User can test one test case by environment variable TEST_CASE_BY_USER. For example,

TEST_CASE_BY_USER=selected_test_name cmake --build . --config Release --target check-onnx-backend

With TEST_CASE_BY_USER specified, the intermediate result, the .onnx file and .so file, are kept in build/test/backend for debugging.

When the ONNX-to-Krnl conversion of an operator is added, the corresponding backend tests for this operator should be added to test.py. The available test cases can be found in third_part/onnx/onnx/backend/test/case/node. Please note to add suffix _cpu to the onnx test name.

Tests with unknown dimensions

Testing with dynamic tensor sizes is most easily performed by using the following command, also used by our checkers.

cmake --build . --config Release --target check-onnx-backend-dynamic

The onnx node tests usually have known dimension size for input tensors. So, to test tensor with unknown dimension, the model importer (Build/FrontendONNXTransformer.cpp) provides a functionality to generate such cases. When the environment variable, IMPORTER_FORCE_DYNAMIC, is set, the frontend import will turn the all the dimensions (by default) of all the input tensors of the model into -1. For example,

IMPORTER_FORCE_DYNAMIC='-1:-1' all dimensions of all the inputs will be changed
IMPORTER_FORCE_DYNAMIC='0:-1' all dimensions of the first input will be changed
IMPORTER_FORCE_DYNAMIC='0:-1|1:0,1' all dimensions of the first input and the 1st and 2nd dimensions of the second input will be changed

The Backus–Naur Form (BNF) for IMPORTER_FORCE_DYNAMIC is as follows.

<ImportForceDymanicExpr> :== `'` <expr> `'`
                  <expr> ::= <inputString> | <inputString> `|` <expr>
            <inputString ::= <inputIndex> `:` <dimString>
             <dimString> ::= <dimIndex> | <dimIndex> `,` <dimString>
            <inputIndex> ::= <index>
              <dimIndex> ::= <index>
                 <index> ::= -1 | <number>
                <number> ::= <digit> | <digit><number>
                 <digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

Value -1 semantically represents all inputs or all dimensions, and it has the highest priority. E.g. '0: -1, 0' means all dimensions of the first input will be changed. Input and dimension indices start from 0.

For example, the default model for test_add_cpu is:

func @main_graph(%arg0: tensor<3x4x5xf32>, %arg1: tensor<3x4x5xf32>) -> tensor<3x4x5xf32>

with IMPORTER_FORCE_DYNAMIC=’-1:-1’, the result is:

func @main_graph(%arg0: tensor<?x?x?xf32>, %arg1: tensor<?x?x?xf32>) -> tensor<?x?x?xf32>

with IMPORTER_FORCE_DYNAMIC=’0:-1’, the result is:

func @main_graph(%arg0: tensor<?x?x?xf32>, %arg1: tensor<3x4x5xf32>) -> tensor<3x4x5xf32>.

with IMPORTER_FORCE_DYNAMIC=’0:0,2 1:1’, the result is:

func @main_graph(%arg0: tensor<?x4x?xf32>, %arg1: tensor<3x?x5xf32>) -> tensor<3x4x5xf32>.

This is a way to use existing node test for dynamic tensors. Since not all test case can pass with dynamic tensor, there is a list in test/backend/test.py, test_not_for_dynamic, to specify which test can not pass with IMPORTER_FORCE_DYNAMIC is defined.

LLVM FileCheck Tests


Numerical Tests

Numerical tests are used to test for numerical correctness in addition to the tests provided by the ONNX package. The goal is to provide extensive numerical value based unit tests; this is very important for ensuring that optimization transformations are valid and correct: more corner cases will arise as we specialize for specific architecture parameters (like vector width). Numerical tests generates extensive amount of numerical value-based unit tests based on simple, naive (and extremely slow) implementation of operations being tested, used to verify the correctness of our operation lowering and optimization.

Numerical tests should be structured such that the following two components are independent and separate:

The motivation is that there are two ways we want to generate test case parameters:

  // RapidCheck test case generation.
  rc::check("convolution implementation correctness", []() {
    const auto N = *rc::gen::inRange(1, 10);
    const auto C = *rc::gen::inRange(1, 20);
    const auto H = *rc::gen::inRange(5, 20);
    const auto W = *rc::gen::inRange(5, 20);

    const auto kH = *rc::gen::inRange(1, 15);
    const auto kW = *rc::gen::inRange(1, 15);

    // We don't want an entire window of padding.
    const auto pHBegin = *rc::gen::inRange(0, kH - 1);
    const auto pHEnd = *rc::gen::inRange(0, kH - 1);
    const auto pWBegin = *rc::gen::inRange(0, kW - 1);
    const auto pWEnd = *rc::gen::inRange(0, kW - 1);

    // Make sure we have at least 1 output per dimension.
    RC_PRE((H >= kH) && (W > kW));

        N, C, H, W, kH, kW, pHBegin, pHEnd, pWBegin, pWEnd));