LogCumSumExp¶

LogCumSumExp - 29¶

Version¶

name: LogCumSumExp (GitHub)
domain: main
since_version: 29
function: True
support_level: SupportType.COMMON
shape inference: True

This version of the operator has been available since version 29.

Summary¶

Computes the inclusive, forward cumulative log-sum-exp of X along axis. Let r = rank(X), where r >= 1. The attribute axis defaults to 0 and must be in [-r, r-1]. The normalized axis is a = axis if axis >= 0, and a = axis + r otherwise.

For each fixed choice of indices in the dimensions other than a, let x[0], ..., x[n-1] be the input slice along a, where n = X.shape[a]. The corresponding output slice is defined by y[j] = log(sum(exp(x[k]), k = 0, ..., j)) for 0 <= j < n. Thus, each prefix includes the current element and all preceding elements, and excludes subsequent elements. The output Y has the same shape and element type as X. If any dimension of X is zero, Y is empty.

The following rules apply independently to each prefix, in order:

If the prefix contains NaN, its output is NaN.
Otherwise, if the prefix contains positive infinity, its output is positive infinity.
Otherwise, if every element in the prefix is negative infinity, its output is negative infinity.

Negative infinity contributes zero to the sum of exponentials.

Accumulation must be numerically stable and use at least float32 precision for float16, bfloat16 and float32 inputs, and at least float64 precision for float64 inputs. Each prefix result is converted to the element type of X to form Y; this conversion must not reduce the precision of the intermediate accumulation for subsequent prefixes. The floating-point accumulation order is unspecified, and different orders may produce different rounding.

Attributes¶

axis - INT (default is 0):

Axis along which to compute inclusive prefixes, defaulting to 0. Must be in [-rank(X), rank(X)-1]. A negative value is normalized by adding rank(X).

Inputs¶

X (heterogeneous) - T:

Floating-point input tensor of rank at least one. Dimensions may be zero.

Outputs¶

Y (heterogeneous) - T:

Tensor with the same shape and element type as X, containing inclusive cumulative log-sum-exp values.

Type Constraints¶

T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ):

Input and output have the same floating-point element type.

Examples¶

_logcumsumexp¶

import numpy as np
import onnx

x = np.array([-200, -100, 100], dtype=np.float32)
node = onnx.helper.make_node("LogCumSumExp", ["X"], ["Y"])
expect(node, [x], [x.copy()], name="test_logcumsumexp_dynamic_range")

_logcumsumexp_axes¶

import numpy as np
import onnx

x = np.log(np.arange(1, 25, dtype=np.float64)).reshape(2, 3, 4)
for axis in (0, 1, 2, -1, -3):
    node = onnx.helper.make_node("LogCumSumExp", ["X"], ["Y"], axis=axis)
    y = np.log(np.cumsum(np.exp(x), axis=axis))
    expect(node, [x], [y], name=f"test_logcumsumexp_axis_{axis}")

_logcumsumexp_special_values¶

import numpy as np
import onnx

x = np.array(
    [
        [-np.inf, -np.inf, -np.inf, -np.inf],
        [-np.inf, 0, np.inf, -np.inf],
        [0, np.inf, np.inf, 1],
        [0, np.nan, np.inf, -np.inf],
        [np.inf, -np.inf, np.nan, 0],
    ],
    dtype=np.float32,
)
y = np.array(
    [
        [-np.inf, -np.inf, -np.inf, -np.inf],
        [-np.inf, 0, np.inf, np.inf],
        [0, np.inf, np.inf, np.inf],
        [0, np.nan, np.nan, np.nan],
        [np.inf, np.inf, np.nan, np.nan],
    ],
    dtype=np.float32,
)
node = onnx.helper.make_node("LogCumSumExp", ["X"], ["Y"], axis=1)
expect(node, [x], [y], name="test_logcumsumexp_special_values")

_logcumsumexp_empty¶

import numpy as np
import onnx

for shape, axis in (((2, 0, 3), 1), ((0, 3), -1)):
    x = np.empty(shape, dtype=np.float32)
    node = onnx.helper.make_node("LogCumSumExp", ["X"], ["Y"], axis=axis)
    expect(node, [x], [x.copy()], name=f"test_logcumsumexp_empty_axis_{axis}")

_logcumsumexp_low_precision¶

import numpy as np
import onnx

for dtype in (np.float16, ml_dtypes.bfloat16):
    x = np.zeros(128, dtype=dtype)
    y = np.log(np.arange(1, 129, dtype=np.float64)).astype(dtype)
    node = onnx.helper.make_node("LogCumSumExp", ["X"], ["Y"])
    expect(node, [x], [y], name=f"test_logcumsumexp_{np.dtype(dtype).name}")