Swish

Swish - 24

Version

  • name: Swish (GitHub)

  • domain: main

  • since_version: 24

  • function: True

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 24.

Summary

Swish function takes one input data (Tensor<T>) and produces one output data (Tensor<T>) of the same shape, where \(Swish(x) = x * sigmoid(alpha * x)\).

Function Body

The function definition for this operator.

&lt;
  domain: &#34;&#34;,
  opset_import: [&#34;&#34; : 24]
&gt;
Swish &lt;alpha&gt;(X) =&gt; (Y)
{
   Alpha = Constant &lt;value_float: float = @alpha&gt; ()
   AlphaCast = CastLike (Alpha, X)
   AlphaMulX = Mul (AlphaCast, X)
   SigmoidAlphaMulX = Sigmoid (AlphaMulX)
   Y = Mul (X, SigmoidAlphaMulX)
}

Attributes

  • alpha - FLOAT (default is 1.0):

    Coefficient to multiply with input before sigmoid.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.

Examples

default

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Swish",
    inputs=["x"],
    outputs=["y"],
    alpha=1.0,  # pass alpha as attribute
)

x = np.array([3, 4, 5], dtype=np.float32)
y = swish(x, alpha=1.0)

expect(
    node,
    inputs=[x],
    outputs=[y],
    name="test_swish",
    opset_imports=[onnx.helper.make_opsetid("", 24)],
)