MeanVarianceNormalization¶
MeanVarianceNormalization - 29¶
Version¶
domain:
mainsince_version:
29function:
Truesupport_level:
SupportType.COMMONshape inference:
True
This version of the operator has been available since version 29.
Summary¶
Computes mean-variance normalization of X:
Y = (X - mean) / (sqrt(variance) + epsilon), where
mean = ReduceMean(X, axes) and
variance = ReduceMean((X - mean)^2, axes).
Each reduction computes the arithmetic mean over axes and retains the
reduced dimensions with size 1 for broadcasting.
An empty axes list reduces over all dimensions.
The default epsilon is 1e-9 and is added after the square root.
For float16 and bfloat16 inputs, X is converted to float32 before computation.
Otherwise, computation uses the input type. epsilon is converted to the
computation type, and the result is cast to the input type.
Y has the same shape and element type as X.
Attributes¶
axes - INTS (default is
[0, 2, 3]):Axes over which to compute mean and variance. Values must be in [-r, r-1], where r is the rank of X; negative values count from the last dimension. An empty list reduces over all dimensions. The default is [0, 2, 3], which computes per-channel statistics for an NCHW tensor.
epsilon - FLOAT (default is
0.0):Value added to the standard deviation in the denominator.
Inputs¶
X (heterogeneous) - T:
Input tensor
Outputs¶
Y (heterogeneous) - T:
Output tensor
Type Constraints¶
T in (
tensor(bfloat16),tensor(double),tensor(float),tensor(float16)):Constrain input and output types to floating-point tensors.
Examples¶
default¶
import numpy as np
import onnx
node = onnx.helper.make_node(
"MeanVarianceNormalization", inputs=["X"], outputs=["Y"]
)
input_data = np.array(
[
[
[[0.8439683], [0.5665144], [0.05836735]],
[[0.02916367], [0.12964272], [0.5060197]],
[[0.79538304], [0.9411346], [0.9546573]],
],
[
[[0.17730942], [0.46192095], [0.26480448]],
[[0.6746842], [0.01665257], [0.62473077]],
[[0.9240844], [0.9722341], [0.11965699]],
],
[
[[0.41356155], [0.9129373], [0.59330076]],
[[0.81929934], [0.7862604], [0.11799799]],
[[0.69248444], [0.54119414], [0.07513223]],
],
],
dtype=np.float32,
)
# Calculate expected output data
data_mean = np.mean(input_data, axis=(0, 2, 3), keepdims=1)
centered = input_data - data_mean
std = np.sqrt(np.mean(np.square(centered), axis=(0, 2, 3), keepdims=True))
expected_output = (input_data - data_mean) / (std + 1e-9)
expect(node, inputs=[input_data], outputs=[expected_output], name="test_mvn")
_epsilon¶
import numpy as np
import onnx
epsilon = 1.0
node = onnx.helper.make_node(
"MeanVarianceNormalization",
inputs=["X"],
outputs=["Y"],
axes=[1, -1],
epsilon=epsilon,
)
input_data = np.arange(24, dtype=np.float64).reshape(2, 3, 4)
# Calculate expected output with custom epsilon
data_mean = np.mean(input_data, axis=(1, -1), keepdims=True)
centered = input_data - data_mean
std = np.sqrt(np.mean(np.square(centered), axis=(1, -1), keepdims=True))
expected_output = (input_data - data_mean) / (std + epsilon)
expect(
node,
inputs=[input_data],
outputs=[expected_output],
name="test_mvn_epsilon",
)
_zero_variance¶
import numpy as np
import onnx
node = onnx.helper.make_node(
"MeanVarianceNormalization",
inputs=["X"],
outputs=["Y"],
axes=[0],
)
input_data = np.ones(2, dtype=np.float32)
mean = np.mean(input_data, axis=0, keepdims=True)
centered = input_data - mean
expected_output = centered / (
np.sqrt(np.mean(np.square(centered), axis=0, keepdims=True)) + 1e-9
)
expect(
node,
inputs=[input_data],
outputs=[expected_output],
name="test_mvn_zero_variance",
)
MeanVarianceNormalization - 13¶
Version¶
domain:
mainsince_version:
13function:
Truesupport_level:
SupportType.COMMONshape inference:
False
This version of the operator has been available since version 13.
Summary¶
Computes mean-variance normalization of X:
Y = (X - mean) / (sqrt(variance) + 1e-9), where
mean = ReduceMean(X, axes) and
variance = ReduceMean(X^2, axes) - mean^2.
Each reduction computes the arithmetic mean over axes and retains the
reduced dimensions with size 1 for broadcasting.
Function Body¶
The function definition for this operator.
<
domain: "",
opset_import: ["" : 18]
>
MeanVarianceNormalization <axes>(X) => (Y)
{
Exponent = Constant <value: tensor = float {2}> ()
Epsilon = Constant <value: tensor = float {1e-09}> ()
axes = Constant <value_ints: ints = @axes> ()
X_RM = ReduceMean (X, axes)
EX_squared = Pow (X_RM, Exponent)
X_squared = Pow (X, Exponent)
E_Xsquared = ReduceMean (X_squared, axes)
Variance = Sub (E_Xsquared, EX_squared)
STD = Sqrt (Variance)
X_variance = Sub (X, X_RM)
Processed_STD = Add (STD, Epsilon)
Y = Div (X_variance, Processed_STD)
}
Attributes¶
axes - INTS (default is
[0, 2, 3]):A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance.
Inputs¶
X (heterogeneous) - T:
Input tensor
Outputs¶
Y (heterogeneous) - T:
Output tensor
Type Constraints¶
T in (
tensor(bfloat16),tensor(double),tensor(float),tensor(float16)):Constrain input and output types to all numeric tensors.
MeanVarianceNormalization - 9¶
Version¶
domain:
mainsince_version:
9function:
Truesupport_level:
SupportType.COMMONshape inference:
False
This version of the operator has been available since version 9.
Summary¶
A MeanVarianceNormalization Function: Perform mean variance normalization
on the input tensor X using formula: <br/> (X-EX)/sqrt(E(X-EX)^2)
Function Body¶
The function definition for this operator.
<
domain: "",
opset_import: ["" : 9]
>
MeanVarianceNormalization <axes>(X) => (Y)
{
Exponent = Constant <value: tensor = float {2}> ()
Epsilon = Constant <value: tensor = float {1e-09}> ()
X_RM = ReduceMean <axes: ints = @axes> (X)
EX_squared = Pow (X_RM, Exponent)
X_squared = Pow (X, Exponent)
E_Xsquared = ReduceMean <axes: ints = @axes> (X_squared)
Variance = Sub (E_Xsquared, EX_squared)
STD = Sqrt (Variance)
X_variance = Sub (X, X_RM)
Processed_STD = Add (STD, Epsilon)
Y = Div (X_variance, Processed_STD)
}
Attributes¶
axes - INTS (default is
[0, 2, 3]):A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance.
Inputs¶
X (heterogeneous) - T:
Input tensor
Outputs¶
Y (heterogeneous) - T:
Output tensor
Type Constraints¶
T in (
tensor(double),tensor(float),tensor(float16)):Constrain input and output types to all numeric tensors.