(l-onnx-doc-MeanVarianceNormalization)= # MeanVarianceNormalization (l-onnx-op-meanvariancenormalization-29)= ## MeanVarianceNormalization - 29 ### Version - **name**: [MeanVarianceNormalization (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#MeanVarianceNormalization) - **domain**: `main` - **since_version**: `29` - **function**: `True` - **support_level**: `SupportType.COMMON` - **shape inference**: `True` This version of the operator has been available **since version 29**. ### Summary Computes mean-variance normalization of `X`: `Y = (X - mean) / (sqrt(variance) + epsilon)`, where `mean = ReduceMean(X, axes)` and `variance = ReduceMean((X - mean)^2, axes)`. Each reduction computes the arithmetic mean over `axes` and retains the reduced dimensions with size 1 for broadcasting. An empty `axes` list reduces over all dimensions. The default `epsilon` is 1e-9 and is added after the square root. For float16 and bfloat16 inputs, `X` is converted to float32 before computation. Otherwise, computation uses the input type. `epsilon` is converted to the computation type, and the result is cast to the input type. `Y` has the same shape and element type as `X`. ### Attributes * **axes - INTS** (default is `[0, 2, 3]`): Axes over which to compute mean and variance. Values must be in [-r, r-1], where r is the rank of X; negative values count from the last dimension. An empty list reduces over all dimensions. The default is [0, 2, 3], which computes per-channel statistics for an NCHW tensor. * **epsilon - FLOAT** (default is `0.0`): Value added to the standard deviation in the denominator. ### Inputs - **X** (heterogeneous) - **T**: Input tensor ### Outputs - **Y** (heterogeneous) - **T**: Output tensor ### Type Constraints * **T** in ( `tensor(bfloat16)`, `tensor(double)`, `tensor(float)`, `tensor(float16)` ): Constrain input and output types to floating-point tensors. ### Examples #### default ```python import numpy as np import onnx node = onnx.helper.make_node( "MeanVarianceNormalization", inputs=["X"], outputs=["Y"] ) input_data = np.array( [ [ [[0.8439683], [0.5665144], [0.05836735]], [[0.02916367], [0.12964272], [0.5060197]], [[0.79538304], [0.9411346], [0.9546573]], ], [ [[0.17730942], [0.46192095], [0.26480448]], [[0.6746842], [0.01665257], [0.62473077]], [[0.9240844], [0.9722341], [0.11965699]], ], [ [[0.41356155], [0.9129373], [0.59330076]], [[0.81929934], [0.7862604], [0.11799799]], [[0.69248444], [0.54119414], [0.07513223]], ], ], dtype=np.float32, ) # Calculate expected output data data_mean = np.mean(input_data, axis=(0, 2, 3), keepdims=1) centered = input_data - data_mean std = np.sqrt(np.mean(np.square(centered), axis=(0, 2, 3), keepdims=True)) expected_output = (input_data - data_mean) / (std + 1e-9) expect(node, inputs=[input_data], outputs=[expected_output], name="test_mvn") ``` #### _epsilon ```python import numpy as np import onnx epsilon = 1.0 node = onnx.helper.make_node( "MeanVarianceNormalization", inputs=["X"], outputs=["Y"], axes=[1, -1], epsilon=epsilon, ) input_data = np.arange(24, dtype=np.float64).reshape(2, 3, 4) # Calculate expected output with custom epsilon data_mean = np.mean(input_data, axis=(1, -1), keepdims=True) centered = input_data - data_mean std = np.sqrt(np.mean(np.square(centered), axis=(1, -1), keepdims=True)) expected_output = (input_data - data_mean) / (std + epsilon) expect( node, inputs=[input_data], outputs=[expected_output], name="test_mvn_epsilon", ) ``` #### _zero_variance ```python import numpy as np import onnx node = onnx.helper.make_node( "MeanVarianceNormalization", inputs=["X"], outputs=["Y"], axes=[0], ) input_data = np.ones(2, dtype=np.float32) mean = np.mean(input_data, axis=0, keepdims=True) centered = input_data - mean expected_output = centered / ( np.sqrt(np.mean(np.square(centered), axis=0, keepdims=True)) + 1e-9 ) expect( node, inputs=[input_data], outputs=[expected_output], name="test_mvn_zero_variance", ) ``` ```{toctree} text_diff_MeanVarianceNormalization_13_29 ``` (l-onnx-op-meanvariancenormalization-13)= ## MeanVarianceNormalization - 13 ### Version - **name**: [MeanVarianceNormalization (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#MeanVarianceNormalization) - **domain**: `main` - **since_version**: `13` - **function**: `True` - **support_level**: `SupportType.COMMON` - **shape inference**: `False` This version of the operator has been available **since version 13**. ### Summary Computes mean-variance normalization of `X`: `Y = (X - mean) / (sqrt(variance) + 1e-9)`, where `mean = ReduceMean(X, axes)` and `variance = ReduceMean(X^2, axes) - mean^2`. Each reduction computes the arithmetic mean over `axes` and retains the reduced dimensions with size 1 for broadcasting. #### Function Body The function definition for this operator. ``` < domain: "", opset_import: ["" : 18] > MeanVarianceNormalization <axes>(X) => (Y) { Exponent = Constant <value: tensor = float {2}> () Epsilon = Constant <value: tensor = float {1e-09}> () axes = Constant <value_ints: ints = @axes> () X_RM = ReduceMean (X, axes) EX_squared = Pow (X_RM, Exponent) X_squared = Pow (X, Exponent) E_Xsquared = ReduceMean (X_squared, axes) Variance = Sub (E_Xsquared, EX_squared) STD = Sqrt (Variance) X_variance = Sub (X, X_RM) Processed_STD = Add (STD, Epsilon) Y = Div (X_variance, Processed_STD) } ``` ### Attributes * **axes - INTS** (default is `[0, 2, 3]`): A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance. ### Inputs - **X** (heterogeneous) - **T**: Input tensor ### Outputs - **Y** (heterogeneous) - **T**: Output tensor ### Type Constraints * **T** in ( `tensor(bfloat16)`, `tensor(double)`, `tensor(float)`, `tensor(float16)` ): Constrain input and output types to all numeric tensors. ```{toctree} text_diff_MeanVarianceNormalization_9_29 text_diff_MeanVarianceNormalization_9_13 ``` (l-onnx-op-meanvariancenormalization-9)= ## MeanVarianceNormalization - 9 ### Version - **name**: [MeanVarianceNormalization (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#MeanVarianceNormalization) - **domain**: `main` - **since_version**: `9` - **function**: `True` - **support_level**: `SupportType.COMMON` - **shape inference**: `False` This version of the operator has been available **since version 9**. ### Summary A MeanVarianceNormalization Function: Perform mean variance normalization on the input tensor X using formula: <br/> ``` (X-EX)/sqrt(E(X-EX)^2) ``` #### Function Body The function definition for this operator. ``` < domain: "", opset_import: ["" : 9] > MeanVarianceNormalization <axes>(X) => (Y) { Exponent = Constant <value: tensor = float {2}> () Epsilon = Constant <value: tensor = float {1e-09}> () X_RM = ReduceMean <axes: ints = @axes> (X) EX_squared = Pow (X_RM, Exponent) X_squared = Pow (X, Exponent) E_Xsquared = ReduceMean <axes: ints = @axes> (X_squared) Variance = Sub (E_Xsquared, EX_squared) STD = Sqrt (Variance) X_variance = Sub (X, X_RM) Processed_STD = Add (STD, Epsilon) Y = Div (X_variance, Processed_STD) } ``` ### Attributes * **axes - INTS** (default is `[0, 2, 3]`): A list of integers, along which to reduce. The default is to calculate along axes [0,2,3] for calculating mean and variance along each channel. Two variables with the same C-coordinate are associated with the same mean and variance. ### Inputs - **X** (heterogeneous) - **T**: Input tensor ### Outputs - **Y** (heterogeneous) - **T**: Output tensor ### Type Constraints * **T** in ( `tensor(double)`, `tensor(float)`, `tensor(float16)` ): Constrain input and output types to all numeric tensors.