ThresholdedRelu¶

ThresholdedRelu - 22¶

Version¶

name: ThresholdedRelu (GitHub)
domain: main
since_version: 22
function: True
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 22.

Summary¶

ThresholdedRelu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the rectified linear function, y = x for x > alpha, y = 0 otherwise, is applied to the tensor elementwise.

Function Body¶

The function definition for this operator.

&lt;
  domain: &#34;&#34;,
  opset_import: [&#34;&#34; : 18]
&gt;
ThresholdedRelu &lt;alpha&gt;(X) =&gt; (Y)
{
   Alpha = Constant &lt;value_float: float = @alpha&gt; ()
   AlphaCast = CastLike (Alpha, X)
   Zero = Constant &lt;value: tensor = float {0}&gt; ()
   ZeroCast = CastLike (Zero, X)
   AlphaLessThanX = Less (AlphaCast, X)
   Y = Where (AlphaLessThanX, X, ZeroCast)
}

Attributes¶

alpha - FLOAT (default is 1.0):

Threshold value

Inputs¶

X (heterogeneous) - T:

Input tensor

Outputs¶

Y (heterogeneous) - T:

Output tensor

Type Constraints¶

T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

Constrain input and output types to float tensors.

Examples¶

default¶

import numpy as np
import onnx

alpha = 2.0
node = onnx.helper.make_node(
    "ThresholdedRelu", inputs=["x"], outputs=["y"], alpha=alpha
)

x = np.array([-1.5, 0.0, 1.2, 2.0, 2.2]).astype(np.float32)
y = np.clip(x, alpha, np.inf)  # expected output [0., 0., 0., 0., 2.2]
y[y == alpha] = 0

expect(node, inputs=[x], outputs=[y], name="test_thresholdedrelu_example")

x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, alpha, np.inf)
y[y == alpha] = 0

expect(node, inputs=[x], outputs=[y], name="test_thresholdedrelu")

_default¶

import numpy as np
import onnx

default_alpha = 1.0
node = onnx.helper.make_node("ThresholdedRelu", inputs=["x"], outputs=["y"])
x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, default_alpha, np.inf)
y[y == default_alpha] = 0

expect(node, inputs=[x], outputs=[y], name="test_thresholdedrelu_default")

ThresholdedRelu - 10 vs 22

ThresholdedRelu - 10¶