NonMaxSuppression¶

NonMaxSuppression - 11¶

Version¶

name: NonMaxSuppression (GitHub)
domain: main
since_version: 11
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 11.

Summary¶

Filter out boxes that have high intersection-over-union (IOU) overlap with previously selected boxes. Bounding boxes with score less than score_threshold are removed. Bounding box format is indicated by attribute center_point_box. Boxes are suppressed if their IOU with a previously selected box is strictly greater than iou_threshold (i.e., boxes with IOU exactly equal to the threshold are kept). Note that this algorithm is agnostic to where the origin is in the coordinate system and more generally is invariant to orthogonal transformations and translations of the coordinate system; thus translating or reflections of the coordinate system result in the same boxes being selected by the algorithm. The selected_indices output is a set of integers indexing into the input collection of bounding boxes representing the selected boxes. The bounding box coordinates corresponding to the selected indices can then be obtained using the Gather or GatherND operation.

Attributes¶

center_point_box - INT (default is 0):

Integer indicate the format of the box data. The default is 0. 0 - the box data is supplied as [y1, x1, y2, x2] where (y1, x1) and (y2, x2) are the coordinates of any diagonal pair of box corners and the coordinates can be provided as normalized (i.e., lying in the interval [0, 1]) or absolute. Mostly used for TF models. 1 - the box data is supplied as [x_center, y_center, width, height]. Mostly used for Pytorch models.

Inputs¶

Between 2 and 5 inputs.

boxes (heterogeneous) - tensor(float):

An input tensor with shape [num_batches, spatial_dimension, 4]. The single box data format is indicated by center_point_box.
scores (heterogeneous) - tensor(float):

An input tensor with shape [num_batches, num_classes, spatial_dimension]
max_output_boxes_per_class (optional, heterogeneous) - tensor(int64):

Integer representing the maximum number of boxes to be selected per batch per class. It is a scalar. Default to 0, which means no output.
iou_threshold (optional, heterogeneous) - tensor(float):

Float representing the threshold for deciding whether boxes overlap too much with respect to IOU. Boxes with IoU strictly greater than this threshold are suppressed. It is scalar. Value range [0, 1]. Default to 0.
score_threshold (optional, heterogeneous) - tensor(float):

Float representing the threshold for deciding when to remove boxes based on score. It is a scalar.

Outputs¶

selected_indices (heterogeneous) - tensor(int64):

selected indices from the boxes tensor. [num_selected_indices, 3], the selected index format is [batch_index, class_index, box_index].

Examples¶

_nonmaxsuppression_suppress_by_IOU¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_suppress_by_IOU",
)

_nonmaxsuppression_suppress_by_IOU_and_scores¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.4]).astype(np.float32)
selected_indices = np.array([[0, 0, 3], [0, 0, 0]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_suppress_by_IOU_and_scores",
)

_nonmaxsuppression_flipped_coordinates¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [1.0, 1.0, 0.0, 0.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, 0.9, 1.0, -0.1],
            [0.0, 10.0, 1.0, 11.0],
            [1.0, 10.1, 0.0, 11.1],
            [1.0, 101.0, 0.0, 100.0],
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_flipped_coordinates",
)

_nonmaxsuppression_limit_output_size¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]).astype(np.float32)
max_output_boxes_per_class = np.array([2]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 3], [0, 0, 0]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_limit_output_size",
)

_nonmaxsuppression_single_box¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array([[[0.0, 0.0, 1.0, 1.0]]]).astype(np.float32)
scores = np.array([[[0.9]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 0]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_single_box",
)

_nonmaxsuppression_identical_boxes¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.0, 1.0, 1.0],
        ]
    ]
).astype(np.float32)
scores = np.array(
    [[[0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]]]
).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 0]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_identical_boxes",
)

_nonmaxsuppression_center_point_box_format¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
    center_point_box=1,
)
boxes = np.array(
    [
        [
            [0.5, 0.5, 1.0, 1.0],
            [0.5, 0.6, 1.0, 1.0],
            [0.5, 0.4, 1.0, 1.0],
            [0.5, 10.5, 1.0, 1.0],
            [0.5, 10.6, 1.0, 1.0],
            [0.5, 100.5, 1.0, 1.0],
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_center_point_box_format",
)

_nonmaxsuppression_two_classes¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ]
    ]
).astype(np.float32)
scores = np.array(
    [[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3], [0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]
).astype(np.float32)
max_output_boxes_per_class = np.array([2]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array(
    [[0, 0, 3], [0, 0, 0], [0, 1, 3], [0, 1, 0]]
).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_two_classes",
)

_nonmaxsuppression_two_batches¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ],
        [
            [0.0, 0.0, 1.0, 1.0],
            [0.0, 0.1, 1.0, 1.1],
            [0.0, -0.1, 1.0, 0.9],
            [0.0, 10.0, 1.0, 11.0],
            [0.0, 10.1, 1.0, 11.1],
            [0.0, 100.0, 1.0, 101.0],
        ],
    ]
).astype(np.float32)
scores = np.array(
    [[[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]], [[0.9, 0.75, 0.6, 0.95, 0.5, 0.3]]]
).astype(np.float32)
max_output_boxes_per_class = np.array([2]).astype(np.int64)
iou_threshold = np.array([0.5]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
selected_indices = np.array(
    [[0, 0, 3], [0, 0, 0], [1, 0, 3], [1, 0, 0]]
).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_two_batches",
)

_nonmaxsuppression_iou_threshold_boundary¶

import numpy as np
import onnx

"""Test boundary condition where IoU exactly equals threshold.

This test verifies that the comparison is strict (>), not inclusive (>=).
When IoU exactly equals the threshold, boxes should be KEPT, not suppressed.
This follows PyTorch's NMS implementation.
"""
node = onnx.helper.make_node(
    "NonMaxSuppression",
    inputs=[
        "boxes",
        "scores",
        "max_output_boxes_per_class",
        "iou_threshold",
        "score_threshold",
    ],
    outputs=["selected_indices"],
)
# Two boxes with 50% overlap in each dimension
# box1=[0,0,1,1], box2=[0.5,0.5,1.5,1.5]
# Intersection area = 0.5 * 0.5 = 0.25
# Union area = 1.0 + 1.0 - 0.25 = 1.75
# IoU = 0.25 / 1.75 (exact value computed below as float32)
boxes = np.array(
    [
        [
            [0.0, 0.0, 1.0, 1.0],  # box 0
            [0.5, 0.5, 1.5, 1.5],  # box 1 - overlaps box 0
        ]
    ]
).astype(np.float32)
scores = np.array([[[0.9, 0.8]]]).astype(np.float32)
max_output_boxes_per_class = np.array([3]).astype(np.int64)
# Compute the exact IoU value and use it as threshold
# This ensures the threshold exactly equals the IoU
exact_iou = np.float32(0.25 / 1.75)
iou_threshold = np.array([exact_iou]).astype(np.float32)
score_threshold = np.array([0.0]).astype(np.float32)
# Both boxes should be selected because IoU == threshold (not > threshold)
selected_indices = np.array([[0, 0, 0], [0, 0, 1]]).astype(np.int64)

expect(
    node,
    inputs=[
        boxes,
        scores,
        max_output_boxes_per_class,
        iou_threshold,
        score_threshold,
    ],
    outputs=[selected_indices],
    name="test_nonmaxsuppression_iou_threshold_boundary",
)

NonMaxSuppression - 10 vs 11