(l-onnx-doc-DequantizePacked)=

# DequantizePacked


(l-onnx-op-dequantizepacked-29)=

## DequantizePacked - 29

### Version

- **name**: [DequantizePacked (GitHub)](https://github.com/onnx/onnx/blob/main/docs/Operators.md#DequantizePacked)
- **domain**: `main`
- **since_version**: `29`
- **function**: `False`
- **support_level**: `SupportType.COMMON`
- **shape inference**: `True`

This version of the operator has been available
**since version 29**.

### Summary

Dequantizes the packed payload of one quantized tensor.
`x` is a one-dimensional UINT8 tensor containing exactly the tensor's bytes.
All multibyte fields are little-endian.

`format` selects the quantization format using an ONNX-defined, case-sensitive
string identifier. The `gguf:` prefix identifies the existing GGML/GGUF layouts.
Only the following fixed identifiers are supported; they accept no parameters:

| Identifier | Format | Elements | Bytes |
|---|---|---|---|
| gguf:q4_0 | Q4_0 | 32 | 18 |
| gguf:q4_1 | Q4_1 | 32 | 20 |
| gguf:q5_0 | Q5_0 | 32 | 22 |
| gguf:q5_1 | Q5_1 | 32 | 24 |
| gguf:q8_0 | Q8_0 | 32 | 34 |
| gguf:q8_1 | Q8_1 | 32 | 36 |
| gguf:q2_k | Q2_K | 256 | 84 |
| gguf:q3_k | Q3_K | 256 | 110 |
| gguf:q4_k | Q4_K | 256 | 144 |
| gguf:q5_k | Q5_K | 256 | 176 |
| gguf:q6_k | Q6_K | 256 | 210 |
| gguf:q8_k | Q8_K | 256 | 292 |
| gguf:iq2_xxs | IQ2_XXS | 256 | 66 |
| gguf:iq2_xs | IQ2_XS | 256 | 74 |
| gguf:iq3_xxs | IQ3_XXS | 256 | 98 |
| gguf:iq1_s | IQ1_S | 256 | 50 |
| gguf:iq4_nl | IQ4_NL | 32 | 18 |
| gguf:iq3_s | IQ3_S | 256 | 110 |
| gguf:iq2_s | IQ2_S | 256 | 82 |
| gguf:iq4_xs | IQ4_XS | 256 | 136 |
| gguf:iq1_m | IQ1_M | 256 | 56 |
| gguf:tq1_0 | TQ1_0 | 256 | 54 |
| gguf:tq2_0 | TQ2_0 | 256 | 66 |
| gguf:mxfp4 | MXFP4 | 32 | 17 |
| gguf:nvfp4 | NVFP4 | 64 | 36 |
| gguf:q1_0 | Q1_0 | 128 | 18 |
| gguf:q2_0 | Q2_0 | 64 | 18 |

The byte layouts, per-format decoding algorithms, and complete IQ codebooks
are defined in the in-tree
[DequantizePacked decoding specification](../repo-docs/DequantizePacked.md).
The [online specification](https://github.com/onnx/onnx/blob/19b091202765d5d444acd9d9c25a0b81eef474b5/docs/DequantizePacked.md)
is also accessible from installed schema documentation.

`shape` gives the logical output dimensions in ONNX (row-major) order.
For a GGUF descriptor `[ne0, ne1, ...]`, use `shape=[..., ne1, ne0]`.
Decoding preserves the flattened element order.
Rank must be at least one. Dimensions are nonnegative literal sizes;
a zero dimension produces an empty tensor. The last dimension must be a
multiple of the format's elements per block, even for an empty tensor.
If B is elements per block and S is bytes per block, the input length must be
`product(shape[:-1]) * (shape[-1] / B) * S`.

Blocks are decoded in input order to FLOAT using the specified
arithmetic and codebooks. The concatenated values are reshaped to `shape`
and converted once to `output_dtype`. Decoding arithmetic uses FLOAT
precision for all output types. FLOAT16 and BFLOAT16 conversion uses
round-to-nearest, ties-to-even, as in Cast.

For example, a Q4_0 block is a little-endian FLOAT16 scale `d` followed
by 16 bytes `q`. It produces `y[j]=d*((q[j]&15)-8)` and
`y[j+16]=d*((q[j]>>4)-8)` for j=0,...,15. Thus 36 bytes containing two
blocks can be decoded with `format="gguf:q4_0", shape=[2,32]`.

### Attributes

* **format - STRING** (required) :

  ONNX-defined, case-sensitive fixed quantization format identifier.

* **output_dtype - INT** (default is `1`):

  ONNX TensorProto data type of the output: FLOAT, FLOAT16, BFLOAT16, or DOUBLE.

* **shape - INTS** (required) :

  Logical output dimensions in ONNX order; rank at least one.

### Inputs

- **x** (heterogeneous) - **tensor(uint8)**:

  Consecutive quantization blocks with little-endian fields.

### Outputs

- **y** (heterogeneous) - **T**:

  Dequantized tensor with the specified shape and output_dtype.

### Type Constraints

* **T** in ( `tensor(bfloat16)`, `tensor(double)`, `tensor(float)`, `tensor(float16)` ):

  Floating-point output.

### Examples

#### _q4_0

```python
import numpy as np
import onnx

node = helper.make_node(
    "DequantizePacked", ["x"], ["y"], format="gguf:q4_0", shape=[2, 32]
)
x = np.frombuffer(
    struct.pack("<e", 0.5)
    + bytes(range(16))
    + struct.pack("<e", -2)
    + bytes([0xF0] * 16),
    dtype=np.uint8,
)
y = np.array(
    [
        list(np.arange(-8, 8) * 0.5) + [-4] * 16,
        [16] * 16 + [-14] * 16,
    ],
    dtype=np.float32,
)
expect(node, inputs=[x], outputs=[y], name="test_dequantizepacked_q4_0")
```

#### _q5_1

```python
import numpy as np
import onnx

node = helper.make_node(
    "DequantizePacked", ["x"], ["y"], format="gguf:q5_1", shape=[32]
)
x = np.frombuffer(
    struct.pack("<eeI", 0.5, -1, 0xFFFF0000) + bytes(range(16)), dtype=np.uint8
)
y = np.array(list(np.arange(16) * 0.5 - 1) + [7] * 16, dtype=np.float32)
expect(node, inputs=[x], outputs=[y], name="test_dequantizepacked_q5_1")
```

#### _iq4_nl

```python
import numpy as np
import onnx

node = helper.make_node(
    "DequantizePacked",
    ["x"],
    ["y"],
    format="gguf:iq4_nl",
    shape=[32],
    output_dtype=TensorProto.FLOAT16,
)
x = np.frombuffer(struct.pack("<e", 0.5) + bytes(range(16)), dtype=np.uint8)
y = np.array(
    [
        -127,
        -104,
        -83,
        -65,
        -49,
        -35,
        -22,
        -10,
        1,
        13,
        25,
        38,
        53,
        69,
        89,
        113,
    ]
    + [-127] * 16,
    dtype=np.float16,
) * np.float16(0.5)
expect(
    node, inputs=[x], outputs=[y], name="test_dequantizepacked_iq4_nl_float16"
)
```

#### _mxfp4

```python
import numpy as np
import onnx

node = helper.make_node(
    "DequantizePacked", ["x"], ["y"], format="gguf:mxfp4", shape=[32]
)
x = np.array([127, *list(range(16))], dtype=np.uint8)
y = np.array(
    [0, 0.5, 1, 1.5, 2, 3, 4, 6, 0, -0.5, -1, -1.5, -2, -3, -4, -6] + [0] * 16,
    dtype=np.float32,
)
expect(node, inputs=[x], outputs=[y], name="test_dequantizepacked_mxfp4")
```

#### _empty

```python
import numpy as np
import onnx

node = helper.make_node(
    "DequantizePacked", ["x"], ["y"], format="gguf:q2_k", shape=[0, 256]
)
expect(
    node,
    inputs=[np.empty(0, dtype=np.uint8)],
    outputs=[np.empty((0, 256), dtype=np.float32)],
    name="test_dequantizepacked_empty",
)
```