Softmax#

Softmax - 13#

Version#

  • name: Softmax (GitHub)

  • domain: main

  • since_version: 13

  • function: True

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 13.

Summary#

The operator computes the normalized exponential values for the given input:

Softmax(input, axis) = Exp(input) / ReduceSum(Exp(input), axis=axis, keepdims=1)

The “axis” attribute indicates the dimension along which Softmax will be performed. The output tensor has the same shape and contains the Softmax values of the corresponding input.

Attributes#

  • axis - INT (default is '-1'):

    Describes the dimension Softmax will be performed on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Inputs#

  • input (heterogeneous) - T:

    The input tensor of rank >= axis.

Outputs#

  • output (heterogeneous) - T:

    The output values with the same shape as the input tensor.

Type Constraints#

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.

Softmax - 11#

Version#

  • name: Softmax (GitHub)

  • domain: main

  • since_version: 11

  • function: False

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 11.

Summary#

The operator computes the softmax (normalized exponential) values for each layer in the batch of the given input.

The input does not need to explicitly be a 2D vector; rather, it will be coerced into one. For an arbitrary n-dimensional tensor input \in [a_0, a_1, …, a_{k-1}, a_k, …, a_{n-1}] and k is the axis provided, then input will be coerced into a 2-dimensional tensor with dimensions [a_0 * … * a_{k-1}, a_k * … * a_{n-1}]. For the default case where axis=1, this means the input tensor will be coerced into a 2D tensor of dimensions [a_0, a_1 * … * a_{n-1}], where a_0 is often the batch size. In this situation, we must have a_0 = N and a_1 * … * a_{n-1} = D. Each of these dimensions must be matched correctly, or else the operator will throw errors. The output tensor has the same shape and contains the softmax values of the corresponding input.

Attributes#

  • axis - INT (default is '1'):

    Describes the axis of the inputs when coerced to 2D; defaults to one because the 0th axis most likely describes the batch_size. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Inputs#

  • input (heterogeneous) - T:

    The input tensor that’s coerced into a 2D matrix of size (NxD) as described above.

Outputs#

  • output (heterogeneous) - T:

    The output values with the same shape as input tensor (the original size without coercion).

Type Constraints#

  • T in ( tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.

Softmax - 1#

Version#

  • name: Softmax (GitHub)

  • domain: main

  • since_version: 1

  • function: False

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 1.

Summary#

The operator computes the softmax (normalized exponential) values for each layer in the batch of the given input. The input is a 2-D tensor (Tensor) of size (batch_size x input_feature_dimensions). The output tensor has the same shape and contains the softmax values of the corresponding input.

Input does not need to explicitly be a 2D vector; rather, it will be coerced into one. For an arbitrary n-dimensional tensor input \in [a_0, a_1, …, a_{k-1}, a_k, …, a_{n-1}] and k is the axis provided, then input will be coerced into a 2-dimensional tensor with dimensions [a_0 * … * a_{k-1}, a_k * … * a_{n-1}]. For the default case where axis=1, this means the input tensor will be coerced into a 2D tensor of dimensions [a_0, a_1 * … * a_{n-1}], where a_0 is often the batch size. In this situation, we must have a_0 = N and a_1 * … * a_{n-1} = D. Each of these dimensions must be matched correctly, or else the operator will throw errors.

Attributes#

  • axis - INT (default is '1'):

    Describes the axis of the inputs when coerced to 2D; defaults to one because the 0th axis most likely describes the batch_size

Inputs#

  • input (heterogeneous) - T:

    The input tensor that’s coerced into a 2D matrix of size (NxD) as described above.

Outputs#

  • output (heterogeneous) - T:

    The output values with the same shape as input tensor (the original size without coercion).

Type Constraints#

  • T in ( tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.