Swish

Swish - 24

Version

  • name: Swish (GitHub)

  • domain: main

  • since_version: 24

  • function: True

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 24.

Summary

Swish function takes one input data (Tensor) and produces one output data (Tensor) of the same shape, where \(Swish(x) = x * sigmoid(alpha * x)\).

Function Body

The function definition for this operator.

<
  domain: "",
  opset_import: ["" : 24]
>
Swish <alpha>(X) => (Y)
{
   Alpha = Constant <value_float: float = @alpha> ()
   AlphaCast = CastLike (Alpha, X)
   AlphaMulX = Mul (AlphaCast, X)
   SigmoidAlphaMulX = Sigmoid (AlphaMulX)
   Y = Mul (X, SigmoidAlphaMulX)
}

Attributes

  • alpha - FLOAT (default is '1.0'):

    Coefficient to multiply with input before sigmoid.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.