InstanceNormalization

InstanceNormalization - 22

Version

This version of the operator has been available since version 22.

Summary

Carries out instance normalization as described in the paper https://arxiv.org/abs/1607.08022.

y = scale * (x - mean) / sqrt(variance + epsilon) + B, where mean and variance are computed per instance per channel.

Attributes

  • epsilon - FLOAT (default is 1e-05):

    The epsilon value to use to avoid division by zero.

Inputs

  • input (heterogeneous) - T:

    Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size.

  • scale (heterogeneous) - T:

    The input 1-dimensional scale tensor of size C.

  • B (heterogeneous) - T:

    The input 1-dimensional bias tensor of size C.

Outputs

  • output (heterogeneous) - T:

    The output tensor of the same shape as input.

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.

Examples

default

import numpy as np
import onnx

def _instancenorm_test_mode(
    x: np.ndarray, s: np.ndarray, bias: np.ndarray, epsilon: float = 1e-5
) -> np.ndarray:
    dims_x = len(x.shape)
    axis = tuple(range(2, dims_x))
    mean = np.mean(x, axis=axis, keepdims=True)
    var = np.var(x, axis=axis, keepdims=True)
    dim_ones = (1,) * (dims_x - 2)
    s = s.reshape(-1, *dim_ones)
    bias = bias.reshape(-1, *dim_ones)
    return s * (x - mean) / np.sqrt(var + epsilon) + bias

# input size: (1, 2, 1, 3)
x = np.array([[[[-1, 0, 1]], [[2, 3, 4]]]]).astype(np.float32)
s = np.array([1.0, 1.5]).astype(np.float32)
bias = np.array([0, 1]).astype(np.float32)
y = _instancenorm_test_mode(x, s, bias).astype(np.float32)

node = onnx.helper.make_node(
    "InstanceNormalization",
    inputs=["x", "s", "bias"],
    outputs=["y"],
)

# output size: (1, 2, 1, 3)
expect(node, inputs=[x, s, bias], outputs=[y], name="test_instancenorm_example")

# input size: (2, 3, 4, 5)
x = np.random.randn(2, 3, 4, 5).astype(np.float32)
s = np.random.randn(3).astype(np.float32)
bias = np.random.randn(3).astype(np.float32)
epsilon = 1e-2
y = _instancenorm_test_mode(x, s, bias, epsilon).astype(np.float32)

node = onnx.helper.make_node(
    "InstanceNormalization",
    inputs=["x", "s", "bias"],
    outputs=["y"],
    epsilon=epsilon,
)

# output size: (2, 3, 4, 5)
expect(node, inputs=[x, s, bias], outputs=[y], name="test_instancenorm_epsilon")

InstanceNormalization - 6

Version

This version of the operator has been available since version 6.

Summary

Carries out instance normalization as described in the paper https://arxiv.org/abs/1607.08022.

y = scale * (x - mean) / sqrt(variance + epsilon) + B, where mean and variance are computed per instance per channel.

Attributes

  • epsilon - FLOAT (default is 1e-05):

    The epsilon value to use to avoid division by zero.

Inputs

  • input (heterogeneous) - T:

    Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size.

  • scale (heterogeneous) - T:

    The input 1-dimensional scale tensor of size C.

  • B (heterogeneous) - T:

    The input 1-dimensional bias tensor of size C.

Outputs

  • output (heterogeneous) - T:

    The output tensor of the same shape as input.

Type Constraints

  • T in ( tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.

InstanceNormalization - 1

Version

This version of the operator has been available since version 1.

Summary

Carries out instance normalization as described in the paper https://arxiv.org/abs/1607.08022.

y = scale * (x - mean) / sqrt(variance + epsilon) + B, where mean and variance are computed per instance per channel.

Attributes

  • consumed_inputs - INTS :

    legacy optimization attribute.

  • epsilon - FLOAT (default is 1e-05):

    The epsilon value to use to avoid division by zero, default is 1e-5f.

Inputs

  • input (heterogeneous) - T:

    The input 4-dimensional tensor of shape NCHW.

  • scale (heterogeneous) - T:

    The input 1-dimensional scale tensor of size C.

  • B (heterogeneous) - T:

    The input 1-dimensional bias tensor of size C.

Outputs

  • output (heterogeneous) - T:

    The output 4-dimensional tensor of the same shape as input.

Type Constraints

  • T in ( tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.