.. _l-onnx-op-causalconvwithstate-d27-29:

CausalConvWithState - 27 vs 29
==============================

Next section compares an older to a newer version of the same operator 
after both definition are converted into markdown text.
Green means an addition to the newer version, red means a deletion.
Anything else is unchanged.

.. raw:: html


    <div id="div_CausalConvWithState_2"></div>
    <link rel="stylesheet" type="text/css" href="../_static/diff2html.min.css" />
    <script type="text/javascript" src="../_static/diff2html-ui-slim.min.js"></script>
    <script>
    const diffString = `
    --- a/CausalConvWithState27
    +++ b/CausalConvWithState29
    @@ -1 +1 @@
    - Stateful causal 1D depthwise convolution.
    ?                 ^^
    + Stateful causal one-dimensional depthwise convolution.
    ?                 ^^^^^^^^^^^^^^^
  
    - Used by Gated DeltaNet (Qwen3.5) and Mamba (Jamba, FalconMamba) as a preprocessing step.
    - Replaces the 3-op pattern (Concat + Conv + Slice) with a single fused operation.
    + input has shape (B, C, L), weight has shape (C, 1, K), and optional
    + bias has shape (C): B is batch size, C is channel count, L is input
    + length, and K is kernel size. Let d = dilation (default 1) and
    + H = (K - 1) * d. Both K and d must be at least 1, and the effective
    + kernel size H + 1 must fit in a signed 64-bit integer.
  
    - The convolution is causal (looks only at current and past positions) and depthwise
    - (each channel is convolved independently with its own kernel).
    + Both state tensors have shape (B, C, H) and store consecutive raw samples
    + in chronological order. Form P by concatenating past_state and input
    + along axis 2; use a zero tensor of shape (B, C, H) when past_state is absent.
    + For each batch b, channel c, and position 0 &amp;lt;= t &amp;lt; L, compute
    + z[b,c,t] = sum_{j=0}^{K-1} weight[c,0,j] * P[b,c,t+j*d] + bias[c],
    + with omitted bias treated as zero. The output has shape (B, C, L) and equals
    + z for activation &amp;#39;none&amp;#39; (the default), or z / (1 + exp(-z)) elementwise
    + for &amp;#39;silu&amp;#39; and its alias &amp;#39;swish&amp;#39;.
  
    + The next state is present_state = P[:, :, L:L+H], independent of activation.
    + When L &amp;lt; H, it retains the required past samples or initial zeros.
    + Passing this state to the next call preserves convolution across chunk boundaries.
    + For K = 1, state has shape (B, C, 0). For L = 0, output has shape
    + (B, C, 0) and state is unchanged (all zeros if past_state is absent).
    - The input, weight, past_state, output, and present_state tensors are rank-3 with
    - shape (batch_size, channels, length). The optional bias input is rank-1 with
    - shape (channels). For higher-dimensional data, use Reshape nodes before and
    - after this operator to pack extra dimensions into the batch or channel axis.
    - 
    - Weight layout: (channels, 1, k) for depthwise convolution.
    - The carry state stores the last (k-1) positions for incremental decode.
    - 
    - The optional activation attribute supports fused SiLU/Swish activation.
  
      ### Attributes
  
      * **activation - STRING** (default is none):
  
        Fused activation function. One of: &amp;#39;silu&amp;#39;, &amp;#39;swish&amp;#39;, &amp;#39;none&amp;#39;. Default is &amp;#39;none&amp;#39;.
    + 
    + * **dilation - INT** (default is 1):
    + 
    +   Spacing between kernel samples. Must be &amp;gt;= 1.
  
      ### Inputs
  
      Between 2 and 4 inputs.
  
      - **input** (heterogeneous) - **T**:
  
        Input tensor with shape (batch_size, channels, length). Channels-first layout.
      - **weight** (heterogeneous) - **T**:
  
    -   Depthwise convolution kernel with shape (channels, 1, k) where k is the kernel size. The middle dim of size 1 follows the ONNX Conv weight layout (M, C/group, k1, ..., kn): since this op is always depthwise, group = channels, so C/group = 1. Keeping this layout makes the weight tensor a drop-in for a depthwise Conv(group=channels) weight, so Conv &amp;lt;-&amp;gt; CausalConvWithState rewrites require no reshape.
    +   Depthwise convolution kernel with shape (channels, 1, k), where k must be &amp;gt;= 1. Each channel is convolved independently with its corresponding kernel.
      - **bias** (optional, heterogeneous) - **T**:
  
        Optional per-channel bias with shape (channels).
      - **past_state** (optional, heterogeneous) - **T**:
  
    -   Carry state from previous step with shape (batch_size, channels, k - 1). If not provided, padding is zero.
    +   Raw chronological history with shape (batch_size, channels, (k - 1) * dilation). If not provided, history is zero.
  
      ### Outputs
  
      - **output** (heterogeneous) - **T**:
  
        Convolution output with same shape as input.
      - **present_state** (heterogeneous) - **T**:
  
    -   Updated carry state with shape (batch_size, channels, k - 1). Contains the last (k - 1) values of the effective padded/concatenated sequence along the causal axis, including any values from past_state or zero-padding when the current input is shorter than k - 1.
    +   The last (k - 1) * dilation raw samples of the concatenated history and input, with shape (batch_size, channels, (k - 1) * dilation).
  
      ### Type Constraints
  
      * **T** in ( tensor(bfloat16), tensor(float), tensor(float16) ):
  
        Constrain input and output types to float tensors.
    `;

    document.addEventListener('DOMContentLoaded', function () {
    var targetElement = document.getElementById('div_CausalConvWithState_2');
    var configuration = {
        drawFileList: true,
        fileListToggle: false,
        fileListStartVisible: false,
        fileContentToggle: false,
        matching: 'lines',
        outputFormat: 'line-by-line',
        synchronisedScroll: true,
        highlight: true,
        renderNothingWhenEmpty: false,
    };
    var diff2htmlUi = new Diff2HtmlUI(targetElement, diffString, configuration);
    diff2htmlUi.draw();
    diff2htmlUi.highlightCode();
    });
    </script>