<jupyter_start><jupyter_text>Time processing for every ONNX nodes in a graphThe following notebook show how long the runtime spends in each node of an ONNX graph.<jupyter_code>from jyquickhelper import add_notebook_menu
add_notebook_menu()
%load_ext mlprodict<jupyter_output><empty_output><jupyter_text>LogisticRegression<jupyter_code>from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y)
clr = LogisticRegression(solver='liblinear')
clr.fit(X_train, y_train)
import numpy
from mlprodict.onnxrt import to_onnx
onx = to_onnx(clr, X_test.astype(numpy.float32))
# add -l 1 if nothing shows up
%onnxview onx
from mlprodict.onnxrt import OnnxInference
import pandas
oinf = OnnxInference(onx)
res = oinf.run({'X': X_test}, node_time=True)
pandas.DataFrame(list(res[1]))
oinf.run({'X': X_test})['output_probability'][:5]<jupyter_output><empty_output><jupyter_text>More measures<jupyter_code>from mlprodict.onnxrt.validate import enumerate_validated_operator_opsets
res = list(enumerate_validated_operator_opsets(
            verbose=0, models={"LogisticRegression"}, opset_min=11,
            runtime='python', debug=False, node_time=True))
import pandas
df = pandas.DataFrame(res[0]['bench-batch'])
df['step'] = df.apply(lambda row: '{}-{}'.format(row['i'], row["name"]), axis=1)
df<jupyter_output><empty_output><jupyter_text>Following tables shows the time spent in each node, it is relative to the total time. For one observation, the runtime spends 10% of the time in ZipMap, it is only 1% with 10 observations. These proportions change due to the computing cost of each node.<jupyter_code>piv = df.pivot('step', 'N', 'time')
total = piv.sum(axis=0)
piv / total<jupyter_output><empty_output><jupyter_text>The python implementation of *ZipMap* does not change the data but wraps in into a frozen class ``ArrayZipMapDitionary`` which mocks a list of dictionaries *pandas* can ingest to create a DataFrame. The cost is a fixed cost and does not depends on the number of processed rows.<jupyter_code>from pyquickhelper.pycode.profiling import profile
bigX = numpy.random.randn(100000, X_test.shape[1]).astype(numpy.float32)
df = profile(lambda: oinf.run({'X': bigX}), as_df=True)[1]
df
%matplotlib inline
ax = df[['namefct', 'cum_tall']].head(n=15).set_index('namefct').plot(kind='bar', figsize=(8, 3), rot=30)
ax.set_title("example of a graph")
for la in ax.get_xticklabels():
    la.set_horizontalalignment('right');<jupyter_output><empty_output><jupyter_text>The class *ArrayZipMapDictionary* is fast to build but has an over head after that as it builds data when needed.<jupyter_code>res = oinf.run({'X': bigX})
prob = res['output_probability']
type(prob)
%timeit pandas.DataFrame(prob)
list_of_dict = [v.asdict() for v in prob]
%timeit pandas.DataFrame(list_of_dict)<jupyter_output>123 ms ± 15.5 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)<jupyter_text>But if you just need to do the following:<jupyter_code>%timeit pandas.DataFrame(prob).values<jupyter_output>669 ms ± 27 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)<jupyter_text>Then, you can just do that:<jupyter_code>print(prob.columns)
%timeit prob.values<jupyter_output>[0, 1, 2]
142 ns ± 30.5 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)<jupyter_text>And then:<jupyter_code>%timeit pandas.DataFrame(prob.values, columns=prob.columns)<jupyter_output>375 µs ± 28.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)<jupyter_text>We can then compare to what *onnxruntime* would do when the runtime is called indenpendly for each node:<jupyter_code>res = list(enumerate_validated_operator_opsets(
            verbose=0, models={"LogisticRegression"}, opset_min=11,
            runtime='onnxruntime2', debug=False, node_time=True))
df = pandas.DataFrame(res[0]['bench-batch'])
df['step'] = df.apply(lambda row: '{}-{}'.format(row['i'], row["name"]), axis=1)
piv = df.pivot('step', 'N', 'time')
total = piv.sum(axis=0)
piv / total<jupyter_output>The maximum opset needed by this model is only 9.
The maximum opset needed by this model is only 1.
The maximum opset needed by this model is only 9.
The maximum opset needed by this model is only 1.
The maximum opset needed by this model is only 9.
The maximum opset needed by this model is only 1.
The maximum opset needed by this model is only 9.
The maximum opset needed by this model is only 1.
{'fraction_non_alphanumeric': 0.1134861172322609, 'fraction_numerical': 0.020273248126928163, 'mean_word_length': 6.0921875, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 36, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '238963', 'license': 'MIT', 'n_tokens_mistral': 1642, 'n_tokens_neox': 1547, 'n_words': 479, 'path': '_doc/notebooks/onnx_node_time.ipynb', 'repo_name': 'Exlsunshine/mlprodict', 'size': '4538'}