Swish

Swish - 24

Version

  • name: Swish (GitHub)

  • domain: main

  • since_version: 24

  • function: True

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 24.

Summary

Swish function takes one input data (Tensor<T>) and produces one output data (Tensor<T>) of the same shape, where \(Swish(x) = x * sigmoid(alpha * x)\).

Function Body

The function definition for this operator.

&lt;
  domain: &#34;&#34;,
  opset_import: [&#34;&#34; : 24]
&gt;
Swish &lt;alpha&gt;(X) =&gt; (Y)
{
   Alpha = Constant &lt;value_float: float = @alpha&gt; ()
   AlphaCast = CastLike (Alpha, X)
   AlphaMulX = Mul (AlphaCast, X)
   SigmoidAlphaMulX = Sigmoid (AlphaMulX)
   Y = Mul (X, SigmoidAlphaMulX)
}

Attributes

  • alpha - FLOAT (default is &#39;1.0&#39;):

    Coefficient to multiply with input before sigmoid.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.