(l-onnx-doc-SoftmaxCrossEntropyLoss)=

# SoftmaxCrossEntropyLoss


(l-onnx-op-softmaxcrossentropyloss-13)=

## SoftmaxCrossEntropyLoss - 13

### Version

- **name**: [SoftmaxCrossEntropyLoss (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#SoftmaxCrossEntropyLoss)
- **domain**: `main`
- **since_version**: `13`
- **function**: `True`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 13**.

### Summary

Loss function that measures the softmax cross entropy
between 'scores' and 'labels'.
This operator first computes a loss tensor whose shape is identical to the labels input.
If the input is 2-D with shape (N, C), the loss tensor may be a N-element vector L = (l_1, l_2, ..., l_N).
If the input is N-D tensor with shape (N, C, D1, D2, ..., Dk),
the loss tensor L may have (N, D1, D2, ..., Dk) as its shape and L[i,][j_1][j_2]...[j_k] denotes a scalar element in L.
After L is available, this operator can optionally do a reduction operator.

* shape(scores): (N, C) where C is the number of classes, or (N, C, D1, D2,..., Dk),
  with K >= 1 in case of K-dimensional loss.
* shape(labels): (N) where each value is 0 <= labels[i] <= C-1, or (N, D1, D2,..., Dk),
  with K >= 1 in case of K-dimensional loss.

The loss for one sample, l_i, can calculated as follows:
```
l[i][d1][d2]...[dk] = -y[i][c][d1][d2]..[dk], where i is the index of classes.
```
or
```
l[i][d1][d2]...[dk] = -y[i][c][d1][d2]..[dk] * weights[c], if 'weights' is provided.
```

loss is zero for the case when label-value equals ignore_index.
```
l[i][d1][d2]...[dk]  = 0, when labels[n][d1][d2]...[dk] = ignore_index
```

where:
```
p = Softmax(scores)
y = Log(p)
c = labels[i][d1][d2]...[dk]
```

Finally, L is optionally reduced:

* If reduction = 'none', the output is L with shape (N, D1, D2, ..., Dk).
* If reduction = 'sum', the output is scalar: Sum(L).
* If reduction = 'mean', the output is scalar: ReduceMean(L), or if weight is provided: `ReduceSum(L) / ReduceSum(W)`,
  where tensor W is of shape `(N, D1, D2, ..., Dk)` and `W[n][d1][d2]...[dk] = weights[labels[i][d1][d2]...[dk]]`.

### Attributes

* **ignore_index - INT** :

  Specifies a target value that is ignored and does not contribute to the input gradient. It's an optional value.

* **reduction - STRING** (default is `mean`):

  Type of reduction to apply to loss: none, sum, mean(default). 'none': no reduction will be applied, 'sum': the output will be summed. 'mean': the sum of the output will be divided by the number of elements in the output.

### Inputs

Between 2 and 3 inputs.

- **scores** (heterogeneous) - **T**:

  The predicted outputs with shape [batch_size, class_size], or [batch_size, class_size, D1, D2 , ..., Dk], where K is the number of dimensions.
- **labels** (heterogeneous) - **Tind**:

  The ground truth output tensor, with shape [batch_size], or [batch_size, D1, D2, ..., Dk], where K is the number of dimensions. Labels element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the label values should either be in the range [0, C) or have the value ignore_index.
- **weights** (optional, heterogeneous) - **T**:

  A manual rescaling weight given to each class. If given, it has to be a 1D Tensor assigning weight to each of the classes. Otherwise, it is treated as if having all ones.

### Outputs

Between 1 and 2 outputs.

- **output** (heterogeneous) - **T**:

  Weighted loss float Tensor. If reduction is 'none', this has the shape of [batch_size], or [batch_size, D1, D2, ..., Dk] in case of K-dimensional loss. Otherwise, it is a scalar.
- **log_prob** (optional, heterogeneous) - **T**:

  Log probability tensor. If the output of softmax is prob, its value is log(prob).

### Type Constraints

* **T** in ( `tensor(bfloat16)`, `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Constrain input and output types to float tensors.
* **Tind** in ( `tensor(int32)`, `tensor(int64)` ):

  Constrain target to integer types

### Examples

#### _softmaxcrossentropy_none

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "none"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, reduction="none")

# Check results
expect(node, inputs=[x, labels], outputs=[sce], name="test_sce_none")
```

#### _softmaxcrossentropy_none_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "none"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, reduction="none", get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_none_log_prob",
)
```

#### _softmaxcrossentropy_none_weights

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "none"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, weight=weights, reduction="none")

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[sce],
    name="test_sce_none_weights",
)
```

#### _softmaxcrossentropy_none_weights_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "none"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, weight=weights, reduction="none", get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[loss, log_prob],
    name="test_sce_none_weights_log_prob",
)
```

#### _softmaxcrossentropy_sum

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "sum"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, reduction="sum")

# Check results
expect(node, inputs=[x, labels], outputs=[sce], name="test_sce_sum")
```

#### _softmaxcrossentropy_sum_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "sum"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, reduction="sum", get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_sum_log_prob",
)
```

#### _softmaxcrossentropy_mean

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels)

# Check results
expect(node, inputs=[x, labels], outputs=[sce], name="test_sce_mean")
```

#### _softmaxcrossentropy_mean_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(x, labels, get_log_prob=True)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_mean_log_prob",
)
```

#### _softmaxcrossentropy_mean_3d

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
y = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, y)

# Check results
expect(node, inputs=[x, y], outputs=[sce], name="test_sce_mean_3d")
```

#### _softmaxcrossentropy_mean_3d_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
y = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(x, y, get_log_prob=True)

# Check results
expect(
    node,
    inputs=[x, y],
    outputs=[loss, log_prob],
    name="test_sce_mean_3d_log_prob",
)
```

#### _softmaxcrossentropy_mean_weights

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, weight=weights)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[sce],
    name="test_sce_mean_weight",
)
```

#### _softmaxcrossentropy_mean_weights_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, weight=weights, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[loss, log_prob],
    name="test_sce_mean_weight_log_prob",
)
```

#### _softmaxcrossentropy_mean_weights_ii

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(0)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
labels[0] = np.int64(0)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, weight=weights, ignore_index=ignore_index)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[sce],
    name="test_sce_mean_weight_ii",
)
```

#### _softmaxcrossentropy_mean_weights_ii_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(0)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
labels[0] = np.int64(0)
weights = np.array([0.9, 0.7, 0.8, 0.9, 0.9], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, weight=weights, ignore_index=ignore_index, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[loss, log_prob],
    name="test_sce_mean_weight_ii_log_prob",
)
```

#### _softmaxcrossentropy_mean_no_weights_ii

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
labels[0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, ignore_index=ignore_index)

# Check results
expect(
    node, inputs=[x, labels], outputs=[sce], name="test_sce_mean_no_weight_ii"
)
```

#### _softmaxcrossentropy_mean_no_weights_ii_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3,)).astype(np.int64)
labels[0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, ignore_index=ignore_index, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_mean_no_weight_ii_log_prob",
)
```

#### _softmaxcrossentropy_mean_weights_ii_3d

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(1)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)
labels[0][0] = np.int64(1)
weights = np.array([0.2, 0.3, 0.6, 0.1, 0.5], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, weight=weights, ignore_index=ignore_index)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[sce],
    name="test_sce_mean_weight_ii_3d",
)
```

#### _softmaxcrossentropy_mean_weights_ii_3d_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(1)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)
labels[0][0] = np.int64(1)
weights = np.array([0.2, 0.3, 0.6, 0.1, 0.5], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, weight=weights, ignore_index=ignore_index, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[loss, log_prob],
    name="test_sce_mean_weight_ii_3d_log_prob",
)
```

#### _softmaxcrossentropy_mean_no_weights_ii_3d

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)
labels[0][0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(x, labels, ignore_index=ignore_index)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[sce],
    name="test_sce_mean_no_weight_ii_3d",
)
```

#### _softmaxcrossentropy_mean_no_weights_ii_3d_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2)).astype(np.int64)
labels[0][0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, ignore_index=ignore_index, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_mean_no_weight_ii_3d_log_prob",
)
```

#### _softmaxcrossentropy_mean_weights_ii_4d

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2, 7).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2, 7)).astype(np.int64)
labels[0][0][0] = np.int64(2)
weights = np.array([0.2, 0.3, 0.6, 0.1, 0.5], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(
    x, labels, reduction=reduction, weight=weights, ignore_index=ignore_index
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[sce],
    name="test_sce_mean_weight_ii_4d",
)
```

#### _softmaxcrossentropy_mean_weights_ii_4d_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2, 7).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2, 7)).astype(np.int64)
labels[0][0][0] = np.int64(2)
weights = np.array([0.2, 0.3, 0.6, 0.1, 0.5], dtype=np.float32)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x,
    labels,
    reduction=reduction,
    weight=weights,
    ignore_index=ignore_index,
    get_log_prob=True,
)

# Check results
expect(
    node,
    inputs=[x, labels, weights],
    outputs=[loss, log_prob],
    name="test_sce_mean_weight_ii_4d_log_prob",
)
```

#### _softmaxcrossentropy_mean_no_weights_ii_4d

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2, 7).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2, 7)).astype(np.int64)
labels[0][0][0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
sce = softmaxcrossentropy(
    x, labels, reduction=reduction, ignore_index=ignore_index
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[sce],
    name="test_sce_mean_no_weight_ii_4d",
)
```

#### _softmaxcrossentropy_mean_no_weights_ii_4d_log_prob

```python
import numpy as np
import onnx

# Define operator attributes.
reduction = "mean"
ignore_index = np.int64(2)

# Create operator.
node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

# Define operator inputs.
np.random.seed(0)
x = np.random.rand(3, 5, 2, 7).astype(np.float32)
labels = np.random.randint(0, high=5, size=(3, 2, 7)).astype(np.int64)
labels[0][0][0] = np.int64(2)

# Compute SoftmaxCrossEntropyLoss
loss, log_prob = softmaxcrossentropy(
    x, labels, reduction=reduction, ignore_index=ignore_index, get_log_prob=True
)

# Check results
expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_mean_no_weight_ii_4d_log_prob",
)
```

#### _input_shape_is_NCd1d2d3d4d5_mean_weight

```python
import numpy as np
import onnx

reduction = "mean"

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
)

N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
labels = np.random.randint(
    0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)
).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)

sce = softmaxcrossentropy(x, labels, weight=weight, reduction=reduction)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[sce],
    name="test_sce_NCd1d2d3d4d5_mean_weight",
)
```

#### _input_shape_is_NCd1d2d3d4d5_mean_weight_log_prob

```python
import numpy as np
import onnx

reduction = "mean"

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
labels = np.random.randint(
    0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)
).astype(np.int64)
weight = np.random.rand(C).astype(np.float32)

loss, log_prob = softmaxcrossentropy(
    x, labels, weight=weight, reduction=reduction, get_log_prob=True
)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[loss, log_prob],
    name="test_sce_NCd1d2d3d4d5_mean_weight_log_prob",
)
```

#### _input_shape_is_NCd1d2d3d4d5_none_no_weight

```python
import numpy as np
import onnx

reduction = "none"

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
)

N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
labels = np.random.randint(
    0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)
).astype(np.int64)

sce = softmaxcrossentropy(x, labels, reduction=reduction)

expect(
    node,
    inputs=[x, labels],
    outputs=[sce],
    name="test_sce_NCd1d2d3d4d5_none_no_weight",
)
```

#### _input_shape_is_NCd1d2d3d4d5_none_no_weight_log_prob

```python
import numpy as np
import onnx

reduction = "none"

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
)

N, C, dim1, dim2, dim3, dim4, dim5 = 3, 5, 6, 6, 5, 3, 4
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3, dim4, dim5).astype(np.float32)
labels = np.random.randint(
    0, high=C, size=(N, dim1, dim2, dim3, dim4, dim5)
).astype(np.int64)

loss, log_prob = softmaxcrossentropy(
    x, labels, reduction=reduction, get_log_prob=True
)

expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_NCd1d2d3d4d5_none_no_weight_log_prob",
)
```

#### _input_shape_is_NCd1_mean_weight_negative_ii

```python
import numpy as np
import onnx

reduction = "mean"
ignore_index = np.int64(-1)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C, dim1 = 3, 5, 6
np.random.seed(0)
x = np.random.rand(N, C, dim1).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N, dim1)).astype(np.int64)
labels[0][0] = -1
weight = np.random.rand(C).astype(np.float32)

sce = softmaxcrossentropy(
    x, labels, weight=weight, reduction=reduction, ignore_index=ignore_index
)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[sce],
    name="test_sce_NCd1_mean_weight_negative_ii",
)
```

#### _input_shape_is_NCd1_mean_weight_negative_ii_log_prob

```python
import numpy as np
import onnx

reduction = "mean"
ignore_index = np.int64(-1)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C, dim1 = 3, 5, 6
np.random.seed(0)
x = np.random.rand(N, C, dim1).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N, dim1)).astype(np.int64)
labels[0][0] = -1
weight = np.random.rand(C).astype(np.float32)

loss, log_prob = softmaxcrossentropy(
    x,
    labels,
    weight=weight,
    reduction=reduction,
    ignore_index=ignore_index,
    get_log_prob=True,
)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[loss, log_prob],
    name="test_sce_NCd1_mean_weight_negative_ii_log_prob",
)
```

#### _input_shape_is_NCd1d2d3_none_no_weight_negative_ii

```python
import numpy as np
import onnx

reduction = "none"
ignore_index = np.int64(-5)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C, dim1, dim2, dim3 = 3, 5, 6, 6, 5
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3)).astype(
    np.int64
)
labels[0][0][0][0] = -5

sce = softmaxcrossentropy(
    x, labels, reduction=reduction, ignore_index=ignore_index
)

expect(
    node,
    inputs=[x, labels],
    outputs=[sce],
    name="test_sce_NCd1d2d3_none_no_weight_negative_ii",
)
```

#### _input_shape_is_NCd1d2d3_none_no_weight_negative_ii_log_prob

```python
import numpy as np
import onnx

reduction = "none"
ignore_index = np.int64(-5)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C, dim1, dim2, dim3 = 3, 5, 6, 6, 5
np.random.seed(0)
x = np.random.rand(N, C, dim1, dim2, dim3).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N, dim1, dim2, dim3)).astype(
    np.int64
)
labels[0][0][0][0] = -5

loss, log_prob = softmaxcrossentropy(
    x, labels, reduction=reduction, ignore_index=ignore_index, get_log_prob=True
)

expect(
    node,
    inputs=[x, labels],
    outputs=[loss, log_prob],
    name="test_sce_NCd1d2d3_none_no_weight_negative_ii_log_prob",
)
```

#### _input_shape_is_NCd1d2d3_sum_weight_high_ii

```python
import numpy as np
import onnx

reduction = "sum"
ignore_index = np.int64(10)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C = 3, 5
np.random.seed(0)
x = np.random.rand(N, C).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N)).astype(np.int64)
labels[0] = 10
weight = np.random.rand(C).astype(np.float32)

sce = softmaxcrossentropy(
    x, labels, weight=weight, reduction=reduction, ignore_index=ignore_index
)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[sce],
    name="test_sce_NCd1d2d3_sum_weight_high_ii",
)
```

#### _input_shape_is_NCd1d2d3_sum_weight_high_ii_log_prob

```python
import numpy as np
import onnx

reduction = "sum"
ignore_index = np.int64(10)

node = onnx.helper.make_node(
    "SoftmaxCrossEntropyLoss",
    inputs=["x", "y", "w"],
    outputs=["z", "log_prob"],
    reduction=reduction,
    ignore_index=ignore_index,
)

N, C = 3, 5
np.random.seed(0)
x = np.random.rand(N, C).astype(np.float32)
labels = np.random.randint(0, high=C, size=(N)).astype(np.int64)
labels[0] = 10
weight = np.random.rand(C).astype(np.float32)

loss, log_prob = softmaxcrossentropy(
    x,
    labels,
    weight=weight,
    reduction=reduction,
    ignore_index=ignore_index,
    get_log_prob=True,
)

expect(
    node,
    inputs=[x, labels, weight],
    outputs=[loss, log_prob],
    name="test_sce_NCd1d2d3_sum_weight_high_ii_log_prob",
)
```

```{toctree}
text_diff_SoftmaxCrossEntropyLoss_12_13
```

(l-onnx-op-softmaxcrossentropyloss-12)=

## SoftmaxCrossEntropyLoss - 12

### Version

- **name**: [SoftmaxCrossEntropyLoss (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#SoftmaxCrossEntropyLoss)
- **domain**: `main`
- **since_version**: `12`
- **function**: `True`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 12**.

### Summary

Loss function that measures the softmax cross entropy
between 'scores' and 'labels'.
This operator first computes a loss tensor whose shape is identical to the labels input.
If the input is 2-D with shape (N, C), the loss tensor may be a N-element vector L = (l_1, l_2, ..., l_N).
If the input is N-D tensor with shape (N, C, D1, D2, ..., Dk),
the loss tensor L may have (N, D1, D2, ..., Dk) as its shape and L[i,][j_1][j_2]...[j_k] denotes a scalar element in L.
After L is available, this operator can optionally do a reduction operator.

shape(scores): (N, C) where C is the number of classes, or (N, C, D1, D2,..., Dk),
        with K >= 1 in case of K-dimensional loss.
shape(labels): (N) where each value is 0 <= labels[i] <= C-1, or (N, D1, D2,..., Dk),
        with K >= 1 in case of K-dimensional loss.

The loss for one sample, l_i, can calculated as follows:
    l[i][d1][d2]...[dk] = -y[i][c][d1][d2]..[dk], where i is the index of classes.
or
    l[i][d1][d2]...[dk] = -y[i][c][d1][d2]..[dk] * weights[c], if 'weights' is provided.

loss is zero for the case when label-value equals ignore_index.
    l[i][d1][d2]...[dk]  = 0, when labels[n][d1][d2]...[dk] = ignore_index

where:
    p = Softmax(scores)
    y = Log(p)
    c = labels[i][d1][d2]...[dk]

Finally, L is optionally reduced:
If reduction = 'none', the output is L with shape (N, D1, D2, ..., Dk).
If reduction = 'sum', the output is scalar: Sum(L).
If reduction = 'mean', the output is scalar: ReduceMean(L), or if weight is provided: ReduceSum(L) / ReduceSum(W),
where tensor W is of shape (N, D1, D2, ..., Dk) and W[n][d1][d2]...[dk] = weights[labels[i][d1][d2]...[dk]].

### Attributes

* **ignore_index - INT** :

  Specifies a target value that is ignored and does not contribute to the input gradient. It's an optional value.

* **reduction - STRING** (default is `mean`):

  Type of reduction to apply to loss: none, sum, mean(default). 'none': no reduction will be applied, 'sum': the output will be summed. 'mean': the sum of the output will be divided by the number of elements in the output.

### Inputs

Between 2 and 3 inputs.

- **scores** (heterogeneous) - **T**:

  The predicted outputs with shape [batch_size, class_size], or [batch_size, class_size, D1, D2 , ..., Dk], where K is the number of dimensions.
- **labels** (heterogeneous) - **Tind**:

  The ground truth output tensor, with shape [batch_size], or [batch_size, D1, D2, ..., Dk], where K is the number of dimensions. Labels element value shall be in range of [0, C). If ignore_index is specified, it may have a value outside [0, C) and the label values should either be in the range [0, C) or have the value ignore_index.
- **weights** (optional, heterogeneous) - **T**:

  A manual rescaling weight given to each class. If given, it has to be a 1D Tensor assigning weight to each of the classes. Otherwise, it is treated as if having all ones.

### Outputs

Between 1 and 2 outputs.

- **output** (heterogeneous) - **T**:

  Weighted loss float Tensor. If reduction is 'none', this has the shape of [batch_size], or [batch_size, D1, D2, ..., Dk] in case of K-dimensional loss. Otherwise, it is a scalar.
- **log_prob** (optional, heterogeneous) - **T**:

  Log probability tensor. If the output of softmax is prob, its value is log(prob).

### Type Constraints

* **T** in ( `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Constrain input and output types to float tensors.
* **Tind** in ( `tensor(int32)`, `tensor(int64)` ):

  Constrain target to integer types