<jupyter_start><jupyter_text>ONNXCommonly code is run on ONNX, and trying to utilize `fastai` as best you can can be a challenge. The beneifits of ONNX is it uses C++ so it can be a faster runtime, and recently ONNX came out with `CUDA` support as well! How hard is it to integrate? Let's make a `tabular` problem again, this time importing `ONNX`:<jupyter_code>!pip show fastinference
from fastai2.tabular.all import *
from fastinference.onnx import *
path = untar_data(URLs.ADULT_SAMPLE)
df = pd.read_csv(path/'adult.csv')
splits = RandomSplitter()(range_of(df))
cat_names = ['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race']
cont_names = ['age', 'fnlwgt', 'education-num']
procs = [Categorify, FillMissing, Normalize]
y_names = 'salary'
to = TabularPandas(df, procs=procs, cat_names=cat_names, cont_names=cont_names,
                   y_names=y_names, splits=splits)
dls = to.dataloaders()
learn = tabular_learner(dls, layers=[200,100])<jupyter_output><empty_output><jupyter_text>I've made a special `fastONNX` wrapper, which will take your `Learner` and export both your model and the `DataLoaders` so ONNX can use them via `learn.to_onnx()`<jupyter_code>learn.to_onnx('tabular')<jupyter_output><empty_output><jupyter_text>Now let's load it in:<jupyter_code>onnx_learn = fastONNX('tabular')<jupyter_output><empty_output><jupyter_text>What all can we do here? We can still do everything exactly the same minus one change: `predict``predict` requires the raw inputs, so instead *always* build a `test_dl` and pass to `get_preds` (just made it simpler to code for me). Let's run a few examples also testing the times:<jupyter_code>single_dl = onnx_learn.test_dl(df.iloc[:1])
%%timeit
preds = onnx_learn.get_preds(dl=single_dl)<jupyter_output>3.46 ms ± 5.44 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)<jupyter_text>Wow! 3.46 ms even beats our previous `predict` with 25 ms! what is it's output?<jupyter_code>name, preds = onnx_learn.get_preds(dl=single_dl)
name, preds<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11252446183953033, 'fraction_numerical': 0.011252446183953033, 'mean_word_length': 6.382671480144404, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '57532', 'license': 'MIT', 'n_tokens_mistral': 760, 'n_tokens_neox': 726, 'n_words': 219, 'path': 'fastinference/.ipynb_checkpoints/02 ONNX-checkpoint.ipynb', 'repo_name': 'ANGELLOPARR/Practical-Deep-Learning-for-Coders-2.0', 'size': '2044'}