Gelu

Gelu - 20

Version

  • name: Gelu (GitHub)

  • domain: main

  • since_version: 20

  • function: True

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 20.

Summary

Gelu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the gaussian error linear units function, \(y = 0.5 * x * (1 + erf(x/sqrt(2)))\) is applied to the tensor elementwise. If the attribute “approximate” is set to “tanh”, the function estimation, \(y = 0.5 * x * (1 + Tanh(sqrt(2/\pi) * (x + 0.044715 * x^3)))\) is used and applied to the tensor elementwise.

Attributes

  • approximate - STRING (default is &#39;none&#39;):

    Gelu approximation algorithm: &#34;tanh&#34;, &#34;none&#34;(default).&#34;none&#34;: do not use approximation.&#34;tanh&#34;: use tanh approximation.

Inputs

  • X (heterogeneous) - T:

    Input tensor

Outputs

  • Y (heterogeneous) - T:

    Output tensor

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

    Constrain input and output types to float tensors.