Gemm¶

Gemm - 29¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 29
function: True
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 29.

Summary¶

Generalized matrix multiplication: Y = alpha * A' * B' + beta * C. A has rank at least 1 and B has rank 2. A’ is A with its last two axes swapped when transA is non-zero; for rank-1 A, transA has no effect. B’ is B transposed when transB is non-zero. If A’ has shape (…, K) and B’ has shape (K, N), Y has shape (…, N): each vector along A’s contraction axis is multiplied by the same matrix B’. The leading dimensions of A’ are preserved, including when there are none. C must be unidirectionally broadcastable to Y. If C is omitted, it is treated as scalar zero. When beta is zero, C does not contribute to the result. This operator supports unidirectional broadcasting (tensor C should be unidirectional broadcastable to tensor A’ * B’); for more details please check Broadcasting in ONNX. This operator has optional inputs/outputs. See ONNX IR for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C.
transA - INT (default is 0):

Whether to swap the last two axes of A. Has no effect on rank-1 A.
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

Between 2 and 3 inputs.

A (heterogeneous) - T:

Input tensor A. Rank >= 1 tensor. After applying transA, its shape is (…, K).
B (heterogeneous) - T:

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (optional, heterogeneous) - T:

Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C must be unidirectionally broadcastable to the output shape (…, N).

Outputs¶

Y (heterogeneous) - T:

Output tensor of shape (…, N).

Type Constraints¶

T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16), tensor(int32), tensor(int64), tensor(uint32), tensor(uint64) ):

Constrain input and output types to float/int tensors.

Examples¶

_linear¶

import numpy as np
import onnx

for shape, suffix in [
    ((5,), "vector"),
    ((2, 3, 5), "rank3"),
    ((2, 3, 4, 5), "rank4"),
    ((0, 3, 5), "empty_batch"),
    ((2, 3, 0), "empty_features"),
]:
    for has_bias in (False, True):
        node = onnx.helper.make_node(
            "Gemm",
            inputs=["a", "b", "c"] if has_bias else ["a", "b"],
            outputs=["y"],
            transB=1,
        )
        a = np.random.ranf(shape).astype(np.float32)
        b = np.random.ranf((7, shape[-1])).astype(np.float32)
        c = np.random.ranf((7,)).astype(np.float32)
        y = gemm_reference_implementation(
            a, b, c if has_bias else None, transB=1
        )
        expect(
            node,
            inputs=[a, b, c] if has_bias else [a, b],
            outputs=[y],
            name=f"test_gemm_linear_{suffix}_{'bias' if has_bias else 'no_bias'}",
            opset_imports=[onnx.helper.make_opsetid("", 29)],
        )

_high_rank_transpose¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm",
    inputs=["a", "b", "c"],
    outputs=["y"],
    transA=1,
    transB=1,
    alpha=0.25,
    beta=0.5,
)
a = np.random.ranf((2, 5, 3)).astype(np.float32)
b = np.random.ranf((7, 5)).astype(np.float32)
c = np.random.ranf((2, 1, 7)).astype(np.float32)
y = gemm_reference_implementation(
    a, b, c, transA=1, transB=1, alpha=0.25, beta=0.5
)
expect(
    node,
    inputs=[a, b, c],
    outputs=[y],
    name="test_gemm_high_rank_transpose",
    opset_imports=[onnx.helper.make_opsetid("", 29)],
)

_default_zero_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b", "c"], outputs=["y"])
a = np.random.ranf([3, 5]).astype(np.float32)
b = np.random.ranf([5, 4]).astype(np.float32)
c = np.zeros([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_default_zero_bias")

_default_no_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b"], outputs=["y"])
a = np.random.ranf([2, 10]).astype(np.float32)
b = np.random.ranf([10, 3]).astype(np.float32)
y = gemm_reference_implementation(a, b)
expect(node, inputs=[a, b], outputs=[y], name="test_gemm_default_no_bias")

_default_scalar_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b", "c"], outputs=["y"])
a = np.random.ranf([2, 3]).astype(np.float32)
b = np.random.ranf([3, 4]).astype(np.float32)
c = np.array(3.14).astype(np.float32)
y = gemm_reference_implementation(a, b, c)
expect(
    node, inputs=[a, b, c], outputs=[y], name="test_gemm_default_scalar_bias"
)

_default_single_elem_vector_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b", "c"], outputs=["y"])
a = np.random.ranf([3, 7]).astype(np.float32)
b = np.random.ranf([7, 3]).astype(np.float32)
c = np.random.ranf([1]).astype(np.float32)
y = gemm_reference_implementation(a, b, c)
expect(
    node,
    inputs=[a, b, c],
    outputs=[y],
    name="test_gemm_default_single_elem_vector_bias",
)

_default_vector_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b", "c"], outputs=["y"])
a = np.random.ranf([2, 7]).astype(np.float32)
b = np.random.ranf([7, 4]).astype(np.float32)
c = np.random.ranf([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c)
expect(
    node, inputs=[a, b, c], outputs=[y], name="test_gemm_default_vector_bias"
)

_default_matrix_bias¶

import numpy as np
import onnx

node = onnx.helper.make_node("Gemm", inputs=["a", "b", "c"], outputs=["y"])
a = np.random.ranf([3, 6]).astype(np.float32)
b = np.random.ranf([6, 4]).astype(np.float32)
c = np.random.ranf([3, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c)
expect(
    node, inputs=[a, b, c], outputs=[y], name="test_gemm_default_matrix_bias"
)

_transposeA¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm", inputs=["a", "b", "c"], outputs=["y"], transA=1
)
a = np.random.ranf([6, 3]).astype(np.float32)
b = np.random.ranf([6, 4]).astype(np.float32)
c = np.zeros([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c, transA=1)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_transposeA")

_transposeB¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm", inputs=["a", "b", "c"], outputs=["y"], transB=1
)
a = np.random.ranf([3, 6]).astype(np.float32)
b = np.random.ranf([4, 6]).astype(np.float32)
c = np.zeros([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c, transB=1)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_transposeB")

_alpha¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm", inputs=["a", "b", "c"], outputs=["y"], alpha=0.5
)
a = np.random.ranf([3, 5]).astype(np.float32)
b = np.random.ranf([5, 4]).astype(np.float32)
c = np.zeros([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c, alpha=0.5)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_alpha")

_beta¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm", inputs=["a", "b", "c"], outputs=["y"], beta=0.5
)
a = np.random.ranf([2, 7]).astype(np.float32)
b = np.random.ranf([7, 4]).astype(np.float32)
c = np.random.ranf([1, 4]).astype(np.float32)
y = gemm_reference_implementation(a, b, c, beta=0.5)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_beta")

_all_attributes¶

import numpy as np
import onnx

node = onnx.helper.make_node(
    "Gemm",
    inputs=["a", "b", "c"],
    outputs=["y"],
    alpha=0.25,
    beta=0.35,
    transA=1,
    transB=1,
)
a = np.random.ranf([4, 3]).astype(np.float32)
b = np.random.ranf([5, 4]).astype(np.float32)
c = np.random.ranf([1, 5]).astype(np.float32)
y = gemm_reference_implementation(
    a, b, c, transA=1, transB=1, alpha=0.25, beta=0.35
)
expect(node, inputs=[a, b, c], outputs=[y], name="test_gemm_all_attributes")

Gemm - 13 vs 29

Gemm - 13¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 13
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 13.

Summary¶

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

A’ = transpose(A) if transA else A
B’ = transpose(B) if transB else B

Compute Y = alpha * A’ * B’ + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A will be transposed before doing the computation if attribute transA is non-zero, same for B and transB. This operator supports unidirectional broadcasting (tensor C should be unidirectional broadcastable to tensor A * B); for more details please check Broadcasting in ONNX. This operator has optional inputs/outputs. See ONNX IR for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C.
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

Between 2 and 3 inputs.

A (heterogeneous) - T:

Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
B (heterogeneous) - T:

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (optional, heterogeneous) - T:

Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).

Outputs¶

Y (heterogeneous) - T:

Output tensor of shape (M, N).

Type Constraints¶

T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16), tensor(int32), tensor(int64), tensor(uint32), tensor(uint64) ):

Constrain input and output types to float/int tensors.

Gemm - 11¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 11
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 11.

Summary¶

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

A’ = transpose(A) if transA else A

B’ = transpose(B) if transB else B

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C.
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

Between 2 and 3 inputs.

A (heterogeneous) - T:

Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
B (heterogeneous) - T:

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (optional, heterogeneous) - T:

Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).

Outputs¶

Y (heterogeneous) - T:

Output tensor of shape (M, N).

Type Constraints¶

T in ( tensor(double), tensor(float), tensor(float16), tensor(int32), tensor(int64), tensor(uint32), tensor(uint64) ):

Constrain input and output types to float/int tensors.

Gemm - 9¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 9
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 9.

Summary¶

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

A’ = transpose(A) if transA else A

B’ = transpose(B) if transB else B

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C.
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

A (heterogeneous) - T:

Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
B (heterogeneous) - T:

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (heterogeneous) - T:

Input tensor C. The shape of C should be unidirectional broadcastable to (M, N).

Outputs¶

Y (heterogeneous) - T:

Output tensor of shape (M, N).

Type Constraints¶

T in ( tensor(double), tensor(float), tensor(float16), tensor(int32), tensor(int64), tensor(uint32), tensor(uint64) ):

Constrain input and output types to float/int tensors.

Gemm - 7¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 7
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 7.

Summary¶

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

A’ = transpose(A) if transA else A

B’ = transpose(B) if transB else B

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C.
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

A (heterogeneous) - T:

Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
B (heterogeneous) - T:

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (heterogeneous) - T:

Input tensor C. The shape of C should be unidirectional broadcastable to (M, N).

Outputs¶

Y (heterogeneous) - T:

Output tensor of shape (M, N).

Type Constraints¶

T in ( tensor(double), tensor(float), tensor(float16) ):

Constrain input and output types to float tensors.

Gemm - 6¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 6
function: False
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 6.

Summary¶

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3 Compute Y = alpha * A * B + beta * C, where input tensor A has dimension (M X K), input tensor B has dimension (K X N), input tensor C and output tensor Y have dimension (M X N). If attribute broadcast is non-zero, input tensor C will be broadcasted to match the dimension requirement. A will be transposed before doing the computation if attribute transA is non-zero, same for B and transB.

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B, the default value is 1.0.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C, the default value is 1.0.
broadcast - INT (default is 0):

Whether C should be broadcasted
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

A (heterogeneous) - T:

Input tensor A
B (heterogeneous) - T:

Input tensor B
C (heterogeneous) - T:

Input tensor C

Outputs¶

Y (heterogeneous) - T:

Output tensor.

Type Constraints¶

T in ( tensor(double), tensor(float), tensor(float16) ):

Constrain input and output types to float tensors.

Gemm - 1¶

Version¶

name: Gemm (GitHub)
domain: main
since_version: 1
function: False
support_level: SupportType.COMMON
shape inference: False

This version of the operator has been available since version 1.

Summary¶

Attributes¶

alpha - FLOAT (default is 1.0):

Scalar multiplier for the product of input tensors A * B, the default value is 1.0.
beta - FLOAT (default is 1.0):

Scalar multiplier for input tensor C, the default value is 1.0.
broadcast - INT (default is 0):

Whether C should be broadcasted
transA - INT (default is 0):

Whether A should be transposed
transB - INT (default is 0):

Whether B should be transposed

Inputs¶

A (heterogeneous) - T:

Input tensor A
B (heterogeneous) - T:

Input tensor B
C (heterogeneous) - T:

Input tensor C, can be inplace.

Outputs¶

Y (heterogeneous) - T:

Output tensor.

Type Constraints¶

T in ( tensor(double), tensor(float), tensor(float16) ):

Constrain input and output types to float tensors.