MeanVarianceNormalization

MeanVarianceNormalization - 29

Version

This version of the operator has been available since version 29.

Summary

Computes mean-variance normalization of X: Y = (X - mean) / (sqrt(variance) + epsilon), where mean = ReduceMean(X, axes) and variance = ReduceMean((X - mean)^2, axes). Each reduction computes the arithmetic mean over axes and retains the reduced dimensions with size 1 for broadcasting. An empty axes list reduces over all dimensions.

The default epsilon is 1e-9 and is added after the square root. For float16 and bfloat16 inputs, X is converted to float32 before computation. Otherwise, computation uses the input type. epsilon is converted to the computation type, and the result is cast to the input type. Y has the same shape and element type as X.

Attributes

  • axes - INTS (default is [0, 2, 3]):

    Axes over which to compute mean and variance. Values must be in [-r, r-1], where r is the rank of X; negative values count from the last dimension. An empty list reduces over all dimensions. The default is [0, 2, 3], which computes per-channel statistics for an NCHW tensor.

  • epsilon - FLOAT (default is 0.0):

    Value added to the standard deviation in the denominator.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to floating-point tensors.

Examples

default

import numpy as np
import onnx

node = onnx.helper.make_node(
    "MeanVarianceNormalization", inputs=["X"], outputs=["Y"]
)

input_data = np.array(
    [
        [
            [[0.8439683], [0.5665144], [0.05836735]],
            [[0.02916367], [0.12964272], [0.5060197]],
            [[0.79538304], [0.9411346], [0.9546573]],
        ],
        [
            [[0.17730942], [0.46192095], [0.26480448]],
            [[0.6746842], [0.01665257], [0.62473077]],
            [[0.9240844], [0.9722341], [0.11965699]],
        ],
        [
            [[0.41356155], [0.9129373], [0.59330076]],
            [[0.81929934], [0.7862604], [0.11799799]],
            [[0.69248444], [0.54119414], [0.07513223]],
        ],
    ],
    dtype=np.float32,
)

# Calculate expected output data
data_mean = np.mean(input_data, axis=(0, 2, 3), keepdims=1)
centered = input_data - data_mean
std = np.sqrt(np.mean(np.square(centered), axis=(0, 2, 3), keepdims=True))
expected_output = (input_data - data_mean) / (std + 1e-9)

expect(node, inputs=[input_data], outputs=[expected_output], name="test_mvn")

_epsilon

import numpy as np
import onnx

epsilon = 1.0
node = onnx.helper.make_node(
    "MeanVarianceNormalization",
    inputs=["X"],
    outputs=["Y"],
    axes=[1, -1],
    epsilon=epsilon,
)

input_data = np.arange(24, dtype=np.float64).reshape(2, 3, 4)

# Calculate expected output with custom epsilon
data_mean = np.mean(input_data, axis=(1, -1), keepdims=True)
centered = input_data - data_mean
std = np.sqrt(np.mean(np.square(centered), axis=(1, -1), keepdims=True))
expected_output = (input_data - data_mean) / (std + epsilon)

expect(
    node,
    inputs=[input_data],
    outputs=[expected_output],
    name="test_mvn_epsilon",
)

_zero_variance

import numpy as np
import onnx

node = onnx.helper.make_node(
    "MeanVarianceNormalization",
    inputs=["X"],
    outputs=["Y"],
    axes=[0],
)
input_data = np.ones(2, dtype=np.float32)
mean = np.mean(input_data, axis=0, keepdims=True)
centered = input_data - mean
expected_output = centered / (
    np.sqrt(np.mean(np.square(centered), axis=0, keepdims=True)) + 1e-9
)

expect(
    node,
    inputs=[input_data],
    outputs=[expected_output],
    name="test_mvn_zero_variance",
)

MeanVarianceNormalization - 13

Version

This version of the operator has been available since version 13.

Summary

Computes mean-variance normalization of X: Y = (X - mean) / (sqrt(variance) + 1e-9), where mean = ReduceMean(X, axes) and variance = ReduceMean(X^2, axes) - mean^2. Each reduction computes the arithmetic mean over axes and retains the reduced dimensions with size 1 for broadcasting.

Function Body

The function definition for this operator.

<
  domain: "",
  opset_import: ["" : 18]
>
MeanVarianceNormalization <axes>(X) => (Y)
{
   Exponent = Constant <value: tensor = float {2}> ()
   Epsilon = Constant <value: tensor = float {1e-09}> ()
   axes = Constant <value_ints: ints = @axes> ()
   X_RM = ReduceMean (X, axes)
   EX_squared = Pow (X_RM, Exponent)
   X_squared = Pow (X, Exponent)
   E_Xsquared = ReduceMean (X_squared, axes)
   Variance = Sub (E_Xsquared, EX_squared)
   STD = Sqrt (Variance)
   X_variance = Sub (X, X_RM)
   Processed_STD = Add (STD, Epsilon)
   Y = Div (X_variance, Processed_STD)
}

Attributes

  • axes - INTS (default is [0, 2, 3]):

    A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to all numeric tensors.

MeanVarianceNormalization - 9

Version

This version of the operator has been available since version 9.

Summary

A MeanVarianceNormalization Function: Perform mean variance normalization on the input tensor X using formula: <br/> (X-EX)/sqrt(E(X-EX)^2)

Function Body

The function definition for this operator.

&lt;
  domain: &#34;&#34;,
  opset_import: [&#34;&#34; : 9]
&gt;
MeanVarianceNormalization &lt;axes&gt;(X) =&gt; (Y)
{
   Exponent = Constant &lt;value: tensor = float {2}&gt; ()
   Epsilon = Constant &lt;value: tensor = float {1e-09}&gt; ()
   X_RM = ReduceMean &lt;axes: ints = @axes&gt; (X)
   EX_squared = Pow (X_RM, Exponent)
   X_squared = Pow (X, Exponent)
   E_Xsquared = ReduceMean &lt;axes: ints = @axes&gt; (X_squared)
   Variance = Sub (E_Xsquared, EX_squared)
   STD = Sqrt (Variance)
   X_variance = Sub (X, X_RM)
   Processed_STD = Add (STD, Epsilon)
   Y = Div (X_variance, Processed_STD)
}

Attributes

  • axes - INTS (default is [0, 2, 3]):

    A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to all numeric tensors.