(l-onnx-doc-RoiAlign)=

# RoiAlign


(l-onnx-op-roialign-22)=

## RoiAlign - 22

### Version

- **name**: [RoiAlign (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#RoiAlign)
- **domain**: `main`
- **since_version**: `22`
- **function**: `False`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 22**.

### Summary

Region of Interest (RoI) align operation described in the
[Mask R-CNN paper](https://arxiv.org/abs/1703.06870).
RoiAlign consumes an input tensor X and region of interests (rois)
to apply pooling across each RoI; it produces a 4-D tensor of shape
(num_rois, C, output_height, output_width).

RoiAlign is proposed to avoid the misalignment by removing
quantizations while converting from original image into feature
map and from feature map into RoI feature; in each ROI bin,
the value of the sampled locations are computed directly
through bilinear interpolation.

### Attributes

* **coordinate_transformation_mode - STRING** (default is `half_pixel`):

  Allowed values are 'half_pixel' and 'output_half_pixel'. Use the value 'half_pixel' to pixel shift the input coordinates by -0.5 (the recommended behavior). Use the value 'output_half_pixel' to omit the pixel shift for the input (use this for a backward-compatible behavior).

* **mode - STRING** (default is `avg`):

  The pooling method. Two modes are supported: 'avg' and 'max'. Default is 'avg'.

* **output_height - INT** (default is `1`):

  default 1; Pooled output Y's height.

* **output_width - INT** (default is `1`):

  default 1; Pooled output Y's width.

* **sampling_ratio - INT** (default is `0`):

  Number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If == 0, then an adaptive number of grid points are used (computed as ceil(roi_width / output_width), and likewise for height). Default is 0.

* **spatial_scale - FLOAT** (default is `1.0`):

  Multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling, i.e., spatial scale of the input feature map X relative to the input image. E.g.; default is 1.0f.

### Inputs

- **X** (heterogeneous) - **T1**:

  Input data tensor from the previous operator; 4-D feature map of shape (N, C, H, W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data.
- **rois** (heterogeneous) - **T1**:

  RoIs (Regions of Interest) to pool over; rois is 2-D input of shape (num_rois, 4) given as [[x1, y1, x2, y2], ...]. The RoIs' coordinates are in the coordinate system of the input image. Each coordinate set has a 1:1 correspondence with the 'batch_indices' input.
- **batch_indices** (heterogeneous) - **T2**:

  1-D tensor of shape (num_rois,) with each element denoting the index of the corresponding image in the batch.

### Outputs

- **Y** (heterogeneous) - **T1**:

  RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element Y[r-1] is a pooled feature map corresponding to the r-th RoI X[r-1].

### Type Constraints

* **T1** in ( `tensor(bfloat16)`, `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Constrain types to float tensors.
* **T2** in ( `tensor(int64)` ):

  Constrain types to int tensors.

### Examples

#### _roialign_aligned_false

```python
import numpy as np
import onnx

node = onnx.helper.make_node(
    "RoiAlign",
    inputs=["X", "rois", "batch_indices"],
    outputs=["Y"],
    spatial_scale=1.0,
    output_height=5,
    output_width=5,
    sampling_ratio=2,
    coordinate_transformation_mode="output_half_pixel",
)

X, batch_indices, rois = get_roi_align_input_values()
# (num_rois, C, output_height, output_width)
Y = np.array(
    [
        [
            [
                [0.4664, 0.4466, 0.3405, 0.5688, 0.6068],
                [0.3714, 0.4296, 0.3835, 0.5562, 0.3510],
                [0.2768, 0.4883, 0.5222, 0.5528, 0.4171],
                [0.4713, 0.4844, 0.6904, 0.4920, 0.8774],
                [0.6239, 0.7125, 0.6289, 0.3355, 0.3495],
            ]
        ],
        [
            [
                [0.3022, 0.4305, 0.4696, 0.3978, 0.5423],
                [0.3656, 0.7050, 0.5165, 0.3172, 0.7015],
                [0.2912, 0.5059, 0.6476, 0.6235, 0.8299],
                [0.5916, 0.7389, 0.7048, 0.8372, 0.8893],
                [0.6227, 0.6153, 0.7097, 0.6154, 0.4585],
            ]
        ],
        [
            [
                [0.2384, 0.3379, 0.3717, 0.6100, 0.7601],
                [0.3767, 0.3785, 0.7147, 0.9243, 0.9727],
                [0.5749, 0.5826, 0.5709, 0.7619, 0.8770],
                [0.5355, 0.2566, 0.2141, 0.2796, 0.3600],
                [0.4365, 0.3504, 0.2887, 0.3661, 0.2349],
            ]
        ],
    ],
    dtype=np.float32,
)

expect(
    node,
    inputs=[X, rois, batch_indices],
    outputs=[Y],
    name="test_roialign_aligned_false",
)
```

#### _roialign_aligned_true

```python
import numpy as np
import onnx

node = onnx.helper.make_node(
    "RoiAlign",
    inputs=["X", "rois", "batch_indices"],
    outputs=["Y"],
    spatial_scale=1.0,
    output_height=5,
    output_width=5,
    sampling_ratio=2,
    coordinate_transformation_mode="half_pixel",
)

X, batch_indices, rois = get_roi_align_input_values()
# (num_rois, C, output_height, output_width)
Y = np.array(
    [
        [
            [
                [0.5178, 0.3434, 0.3229, 0.4474, 0.6344],
                [0.4031, 0.5366, 0.4428, 0.4861, 0.4023],
                [0.2512, 0.4002, 0.5155, 0.6954, 0.3465],
                [0.3350, 0.4601, 0.5881, 0.3439, 0.6849],
                [0.4932, 0.7141, 0.8217, 0.4719, 0.4039],
            ]
        ],
        [
            [
                [0.3070, 0.2187, 0.3337, 0.4880, 0.4870],
                [0.1871, 0.4914, 0.5561, 0.4192, 0.3686],
                [0.1433, 0.4608, 0.5971, 0.5310, 0.4982],
                [0.2788, 0.4386, 0.6022, 0.7000, 0.7524],
                [0.5774, 0.7024, 0.7251, 0.7338, 0.8163],
            ]
        ],
        [
            [
                [0.2393, 0.4075, 0.3379, 0.2525, 0.4743],
                [0.3671, 0.2702, 0.4105, 0.6419, 0.8308],
                [0.5556, 0.4543, 0.5564, 0.7502, 0.9300],
                [0.6626, 0.5617, 0.4813, 0.4954, 0.6663],
                [0.6636, 0.3721, 0.2056, 0.1928, 0.2478],
            ]
        ],
    ],
    dtype=np.float32,
)

expect(
    node,
    inputs=[X, rois, batch_indices],
    outputs=[Y],
    name="test_roialign_aligned_true",
)
```

#### _roialign_mode_max

```python
import numpy as np
import onnx

X = np.array(
    [
        [
            [
                [
                    0.2764,
                    0.715,
                    0.1958,
                    0.3416,
                    0.4638,
                    0.0259,
                    0.2963,
                    0.6518,
                    0.4856,
                    0.725,
                ],
                [
                    0.9637,
                    0.0895,
                    0.2919,
                    0.6753,
                    0.0234,
                    0.6132,
                    0.8085,
                    0.5324,
                    0.8992,
                    0.4467,
                ],
                [
                    0.3265,
                    0.8479,
                    0.9698,
                    0.2471,
                    0.9336,
                    0.1878,
                    0.4766,
                    0.4308,
                    0.34,
                    0.2162,
                ],
                [
                    0.0206,
                    0.172,
                    0.2155,
                    0.4394,
                    0.0653,
                    0.3406,
                    0.7724,
                    0.3921,
                    0.2541,
                    0.5799,
                ],
                [
                    0.4062,
                    0.2194,
                    0.4473,
                    0.4687,
                    0.7109,
                    0.9327,
                    0.9815,
                    0.632,
                    0.1728,
                    0.6119,
                ],
                [
                    0.3097,
                    0.1283,
                    0.4984,
                    0.5068,
                    0.4279,
                    0.0173,
                    0.4388,
                    0.043,
                    0.4671,
                    0.7119,
                ],
                [
                    0.1011,
                    0.8477,
                    0.4726,
                    0.1777,
                    0.9923,
                    0.4042,
                    0.1869,
                    0.7795,
                    0.9946,
                    0.9689,
                ],
                [
                    0.1366,
                    0.3671,
                    0.7011,
                    0.6234,
                    0.9867,
                    0.5585,
                    0.6985,
                    0.5609,
                    0.8788,
                    0.9928,
                ],
                [
                    0.5697,
                    0.8511,
                    0.6711,
                    0.9406,
                    0.8751,
                    0.7496,
                    0.165,
                    0.1049,
                    0.1559,
                    0.2514,
                ],
                [
                    0.7012,
                    0.4056,
                    0.7879,
                    0.3461,
                    0.0415,
                    0.2998,
                    0.5094,
                    0.3727,
                    0.5482,
                    0.0502,
                ],
            ]
        ]
    ],
    dtype=np.float32,
)
rois = np.array(
    [[0.0, 0.0, 9.0, 9.0], [0.0, 5.0, 4.0, 9.0], [5.0, 5.0, 9.0, 9.0]],
    dtype=np.float32,
)
batch_indices = np.array([0, 0, 0], dtype=np.int64)

Y = np.array(
    [
        [
            [
                [0.3445228, 0.37310338, 0.37865096, 0.446696, 0.37991184],
                [0.4133513, 0.5455125, 0.6651902, 0.55805874, 0.27110294],
                [0.21223956, 0.40924096, 0.8417618, 0.792561, 0.37196714],
                [0.46835402, 0.39741728, 0.8012819, 0.4969306, 0.5495158],
                [0.3595896, 0.5196813, 0.5403741, 0.23814403, 0.19992709],
            ]
        ],
        [
            [
                [0.30517197, 0.5086199, 0.3189761, 0.4054401, 0.47630402],
                [0.50862, 0.8477, 0.37808004, 0.24936005, 0.79384017],
                [0.17620805, 0.29368007, 0.44870415, 0.4987201, 0.63148826],
                [0.51066005, 0.8511, 0.5368801, 0.9406, 0.70008016],
                [0.4487681, 0.51066035, 0.5042561, 0.5643603, 0.42004836],
            ]
        ],
        [
            [
                [0.21062402, 0.3510401, 0.37416005, 0.5967599, 0.46507207],
                [0.32336006, 0.31180006, 0.6236001, 0.9946, 0.7751202],
                [0.35744014, 0.5588001, 0.35897616, 0.7030401, 0.6353923],
                [0.5996801, 0.27940005, 0.17948808, 0.35152006, 0.31769615],
                [0.3598083, 0.40752012, 0.2385281, 0.43856013, 0.26313624],
            ]
        ],
    ],
    dtype=np.float32,
)

node = onnx.helper.make_node(
    "RoiAlign",
    inputs=["X", "rois", "batch_indices"],
    mode="max",
    outputs=["Y"],
    spatial_scale=1.0,
    output_height=5,
    output_width=5,
    sampling_ratio=2,
    coordinate_transformation_mode="output_half_pixel",
)

expect(
    node,
    inputs=[X, rois, batch_indices],
    outputs=[Y],
    name="test_roialign_mode_max",
)
```

```{toctree}
text_diff_RoiAlign_16_22
```

(l-onnx-op-roialign-16)=

## RoiAlign - 16

### Version

- **name**: [RoiAlign (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#RoiAlign)
- **domain**: `main`
- **since_version**: `16`
- **function**: `False`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 16**.

### Summary

Region of Interest (RoI) align operation described in the
[Mask R-CNN paper](https://arxiv.org/abs/1703.06870).
RoiAlign consumes an input tensor X and region of interests (rois)
to apply pooling across each RoI; it produces a 4-D tensor of shape
(num_rois, C, output_height, output_width).

RoiAlign is proposed to avoid the misalignment by removing
quantizations while converting from original image into feature
map and from feature map into RoI feature; in each ROI bin,
the value of the sampled locations are computed directly
through bilinear interpolation.

### Attributes

* **coordinate_transformation_mode - STRING** (default is `half_pixel`):

  Allowed values are 'half_pixel' and 'output_half_pixel'. Use the value 'half_pixel' to pixel shift the input coordinates by -0.5 (the recommended behavior). Use the value 'output_half_pixel' to omit the pixel shift for the input (use this for a backward-compatible behavior).

* **mode - STRING** (default is `avg`):

  The pooling method. Two modes are supported: 'avg' and 'max'. Default is 'avg'.

* **output_height - INT** (default is `1`):

  default 1; Pooled output Y's height.

* **output_width - INT** (default is `1`):

  default 1; Pooled output Y's width.

* **sampling_ratio - INT** (default is `0`):

  Number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If == 0, then an adaptive number of grid points are used (computed as ceil(roi_width / output_width), and likewise for height). Default is 0.

* **spatial_scale - FLOAT** (default is `1.0`):

  Multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling, i.e., spatial scale of the input feature map X relative to the input image. E.g.; default is 1.0f.

### Inputs

- **X** (heterogeneous) - **T1**:

  Input data tensor from the previous operator; 4-D feature map of shape (N, C, H, W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data.
- **rois** (heterogeneous) - **T1**:

  RoIs (Regions of Interest) to pool over; rois is 2-D input of shape (num_rois, 4) given as [[x1, y1, x2, y2], ...]. The RoIs' coordinates are in the coordinate system of the input image. Each coordinate set has a 1:1 correspondence with the 'batch_indices' input.
- **batch_indices** (heterogeneous) - **T2**:

  1-D tensor of shape (num_rois,) with each element denoting the index of the corresponding image in the batch.

### Outputs

- **Y** (heterogeneous) - **T1**:

  RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element Y[r-1] is a pooled feature map corresponding to the r-th RoI X[r-1].

### Type Constraints

* **T1** in ( `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Constrain types to float tensors.
* **T2** in ( `tensor(int64)` ):

  Constrain types to int tensors.

```{toctree}
text_diff_RoiAlign_10_22
text_diff_RoiAlign_10_16
```

(l-onnx-op-roialign-10)=

## RoiAlign - 10

### Version

- **name**: [RoiAlign (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#RoiAlign)
- **domain**: `main`
- **since_version**: `10`
- **function**: `False`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 10**.

### Summary

Region of Interest (RoI) align operation described in the
[Mask R-CNN paper](https://arxiv.org/abs/1703.06870).
RoiAlign consumes an input tensor X and region of interests (rois)
to apply pooling across each RoI; it produces a 4-D tensor of shape
(num_rois, C, output_height, output_width).

RoiAlign is proposed to avoid the misalignment by removing
quantizations while converting from original image into feature
map and from feature map into RoI feature; in each ROI bin,
the value of the sampled locations are computed directly
through bilinear interpolation.

### Attributes

* **mode - STRING** (default is `avg`):

  The pooling method. Two modes are supported: 'avg' and 'max'. Default is 'avg'.

* **output_height - INT** (default is `1`):

  default 1; Pooled output Y's height.

* **output_width - INT** (default is `1`):

  default 1; Pooled output Y's width.

* **sampling_ratio - INT** (default is `0`):

  Number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If == 0, then an adaptive number of grid points are used (computed as ceil(roi_width / output_width), and likewise for height). Default is 0.

* **spatial_scale - FLOAT** (default is `1.0`):

  Multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling, i.e., spatial scale of the input feature map X relative to the input image. E.g.; default is 1.0f.

### Inputs

- **X** (heterogeneous) - **T1**:

  Input data tensor from the previous operator; 4-D feature map of shape (N, C, H, W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data.
- **rois** (heterogeneous) - **T1**:

  RoIs (Regions of Interest) to pool over; rois is 2-D input of shape (num_rois, 4) given as [[x1, y1, x2, y2], ...]. The RoIs' coordinates are in the coordinate system of the input image. Each coordinate set has a 1:1 correspondence with the 'batch_indices' input.
- **batch_indices** (heterogeneous) - **T2**:

  1-D tensor of shape (num_rois,) with each element denoting the index of the corresponding image in the batch.

### Outputs

- **Y** (heterogeneous) - **T1**:

  RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element Y[r-1] is a pooled feature map corresponding to the r-th RoI X[r-1].

### Type Constraints

* **T1** in ( `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Constrain types to float tensors.
* **T2** in ( `tensor(int64)` ):

  Constrain types to int tensors.