.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/plot_benchmark_cdist.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_benchmark_cdist.py: .. _l-benchmark-cdist: Compare CDist with scipy ======================== The following example focuses on one particular operator, CDist and compares its execution time between *onnxruntime* and *scipy*. ONNX Graph with CDist +++++++++++++++++++++ `cdist `_ function computes pairwise distances. .. GENERATED FROM PYTHON SOURCE LINES 21-40 .. code-block:: Python from pprint import pprint from timeit import Timer import numpy as np from scipy.spatial.distance import cdist from tqdm import tqdm from pandas import DataFrame import onnx import onnxruntime as rt from onnxruntime import InferenceSession import skl2onnx from skl2onnx.algebra.custom_ops import OnnxCDist from skl2onnx.common.data_types import FloatTensorType X = np.ones((2, 4), dtype=np.float32) Y = np.ones((3, 4), dtype=np.float32) Y *= 2 print(cdist(X, Y, metric="euclidean")) .. rst-class:: sphx-glr-script-out .. code-block:: none [[2. 2. 2.] [2. 2. 2.]] .. GENERATED FROM PYTHON SOURCE LINES 41-42 ONNX .. GENERATED FROM PYTHON SOURCE LINES 42-48 .. code-block:: Python op = OnnxCDist("X", "Y", op_version=12, output_names=["Z"], metric="euclidean") onx = op.to_onnx({"X": X, "Y": Y}, outputs=[("Z", FloatTensorType())]) print(onx) .. rst-class:: sphx-glr-script-out .. code-block:: none ir_version: 10 producer_name: "skl2onnx" producer_version: "1.18.0" domain: "ai.onnx" model_version: 0 graph { node { input: "X" input: "Y" output: "Z" name: "CD_CDist" op_type: "CDist" attribute { name: "metric" s: "euclidean" type: STRING } domain: "com.microsoft" } name: "OnnxCDist" input { name: "X" type { tensor_type { elem_type: 1 shape { dim { } dim { dim_value: 4 } } } } } input { name: "Y" type { tensor_type { elem_type: 1 shape { dim { } dim { dim_value: 4 } } } } } output { name: "Z" type { tensor_type { elem_type: 1 } } } } opset_import { domain: "com.microsoft" version: 1 } .. GENERATED FROM PYTHON SOURCE LINES 49-54 CDist and onnxruntime +++++++++++++++++++++ We compute the output of CDist operator with onnxruntime. .. GENERATED FROM PYTHON SOURCE LINES 54-59 .. code-block:: Python sess = InferenceSession(onx.SerializeToString(), providers=["CPUExecutionProvider"]) res = sess.run(None, {"X": X, "Y": Y}) print(res) .. rst-class:: sphx-glr-script-out .. code-block:: none [array([[1.9999999, 1.9999999, 1.9999999], [1.9999999, 2. , 2. ]], dtype=float32)] .. GENERATED FROM PYTHON SOURCE LINES 60-64 Benchmark +++++++++ Let's compare onnxruntime and scipy. .. GENERATED FROM PYTHON SOURCE LINES 64-86 .. code-block:: Python def measure_time(name, stmt, context, repeat=100, number=20): tim = Timer(stmt, globals=context) res = np.array(tim.repeat(repeat=repeat, number=number)) res /= number mean = np.mean(res) dev = np.mean(res**2) dev = (dev - mean**2) ** 0.5 return dict( average=mean, deviation=dev, min_exec=np.min(res), max_exec=np.max(res), repeat=repeat, number=number, nrows=context["X"].shape[0], ncols=context["Y"].shape[1], name=name, ) .. GENERATED FROM PYTHON SOURCE LINES 87-88 scipy .. GENERATED FROM PYTHON SOURCE LINES 88-95 .. code-block:: Python time_scipy = measure_time( "scipy", "cdist(X, Y)", context={"cdist": cdist, "X": X, "Y": Y} ) pprint(time_scipy) .. rst-class:: sphx-glr-script-out .. code-block:: none {'average': np.float64(8.157823500368977e-06), 'deviation': np.float64(8.986884053687116e-06), 'max_exec': np.float64(5.3478249901672827e-05), 'min_exec': np.float64(2.6912999601336197e-06), 'name': 'scipy', 'ncols': 4, 'nrows': 2, 'number': 20, 'repeat': 100} .. GENERATED FROM PYTHON SOURCE LINES 96-97 onnxruntime .. GENERATED FROM PYTHON SOURCE LINES 97-103 .. code-block:: Python time_ort = measure_time( "ort", "sess.run(None, {'X': X, 'Y': Y})", context={"sess": sess, "X": X, "Y": Y} ) pprint(time_ort) .. rst-class:: sphx-glr-script-out .. code-block:: none {'average': np.float64(3.3001529995090104e-05), 'deviation': np.float64(3.1202860470685634e-05), 'max_exec': np.float64(0.00018513064987928373), 'min_exec': np.float64(6.747400038875639e-06), 'name': 'ort', 'ncols': 4, 'nrows': 2, 'number': 20, 'repeat': 100} .. GENERATED FROM PYTHON SOURCE LINES 104-105 Longer benchmark .. GENERATED FROM PYTHON SOURCE LINES 105-130 .. code-block:: Python metrics = [] for dim in tqdm([10, 100, 1000, 10000]): # We cannot change the number of column otherwise # we need to create a new graph. X = np.random.randn(dim, 4).astype(np.float32) Y = np.random.randn(10, 4).astype(np.float32) time_scipy = measure_time( "scipy", "cdist(X, Y)", context={"cdist": cdist, "X": X, "Y": Y} ) time_ort = measure_time( "ort", "sess.run(None, {'X': X, 'Y': Y})", context={"sess": sess, "X": X, "Y": Y}, ) metric = dict(N=dim, scipy=time_scipy["average"], ort=time_ort["average"]) metrics.append(metric) df = DataFrame(metrics) df["scipy/ort"] = df["scipy"] / df["ort"] print(df) df.plot(x="N", y=["scipy/ort"]) .. image-sg:: /auto_examples/images/sphx_glr_plot_benchmark_cdist_001.png :alt: plot benchmark cdist :srcset: /auto_examples/images/sphx_glr_plot_benchmark_cdist_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none 0%| | 0/4 [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_benchmark_cdist.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_benchmark_cdist.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_