<jupyter_start><jupyter_text>Run a Tool from Arachne CLIIn this notebook, we explain how to use Arachne CLI (i.e., `arachne.driver.cli`) for running a tool in Arachne.Here, we will be working with a tool of TVM to compile ResNet-50 v2 from the Tensorflow Keras Applications. TVM is a deep learning compiler with supporting various DNN models as its input.ResNet-50 v2 is one of the famous convolutional neural networks to classify images.  Prepare a ModelFirst, we prepare an input model by using a Tensorflow Keras API.<jupyter_code>import tensorflow as tf

model = tf.keras.applications.resnet_v2.ResNet50V2()
model.summary()
model.save("/tmp/resnet50-v2.h5")<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet50v2_weights_tf_dim_ordering_tf_kernels.h5
102875136/102869336 [==============================] - 3s 0us/step
102883328/102869336 [==============================] - 3s 0us/step
Model: "resnet50v2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
conv1_pad (ZeroPadding2D)       (None, 230, 230, 3)  0           input_1[0][0]                    
__________________________________________________________________________________________________
conv1_conv (Co[...]<jupyter_text>Apply a Tool to the Input ModelNext, let's try to execute the TVM taking the prepared model as it's input from Arachne CLI.Typically, you can start with the following command.<jupyter_code>%%bash

python -m arachne.driver.cli +tools=tvm model_file=/tmp/resnet50-v2.h5 output_path=/tmp/output.tar<jupyter_output>[2022-03-18 01:39:25,607][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
[2022-03-18 01:39:26,948][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.<jupyter_text>Deals with the Dynamic ShapeNow you can see there is a something worng because the TVM cannot deal with the negative shape value (, or the dynamic shape).TVM requires to specify the static shape for the networks that have dynamic shapes.To address this problem, we provide an option (i.e., `input_spec`) to specify the tensor specification of the input model.Users can pass a path to the YAML file that describes such information.For example, the file for this case looks like below.<jupyter_code># /tmp/resnet50-v2.yaml
inputs:
  - dtype: float32
    name: input
    shape:
    - 1
    - 224
    - 224
    - 3
outputs:
  - dtype: float32
    name: Identity
    shape:
    - 1
    - 1000<jupyter_output><empty_output><jupyter_text>Finally, you can compile it.<jupyter_code>%%bash

python -m arachne.driver.cli +tools=tvm model_file=/tmp/resnet50-v2.h5 output_path=/tmp/output.tar model_spec_file=/tmp/resnet50-v2.yaml<jupyter_output>[2022-03-18 01:40:05,686][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
[2022-03-18 01:40:06,993][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
[2022-03-18 01:40:25,341][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,346][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,351][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,355][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,360][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,364][strategy][WARNING] - conv2d NHWC layout is not optimized for x86 with autotvm.
[2022-03-18 01:40:25,368][strategy][WARNING] - conv2d NHWC layou[...]<jupyter_text>Try Tool-Specific ConfigurationsYou can configure the tool behavior by passing specific values to options.To understand what options are available, you just add `--help` to the previous command.<jupyter_code>%%bash

python -m arachne.driver.cli +tools=tvm model_file=/tmp/resnet50-v2.h5 output_path=/tmp/output.tar model_spec_file=/tmp/resnet50-v2.yaml --help<jupyter_output>cli is powered by Hydra.

== Configuration groups ==
Compose your configuration from those groups (group=option)

tools: onnx_simplifier, openvino2tf, openvino_mo, tflite_converter, tftrt, torch2onnx, torch2trt, tvm
tvm_target: dgx-1, dgx-s, jetson-nano, jetson-xavier-nx, rasp4b64


== Config ==
Override anything in the config (foo.bar=value)

input: /tmp/resnet50-v2.h5
input_spec: /tmp/resnet50-v2.yaml
output: /tmp/output.tar
tools:
  tvm:
    cpu_target: x86-64
    cpu_attr: []
    cpu_name: null
    cuda_target_device: cuda
    composite_target:
    - cpu
    target: null
    target_host: null
    desired_layout: null
    disabled_pass: null
    opt_level: 3
    export_format: tar
    cross_compiler: null
    cross_compiler_options: null


Powered by Hydra (https://hydra.cc)
Use --hydra-help to view Hydra specific help<jupyter_text>Here, we only explain a simple usage to compile for TensorRT and CUDA targets for space problem. Please refer to the API documentation for `arachne.tools` to know details.To compile for TensorRT and CUDA targets, you should set `tools.tvm.***` options appropriately like below:<jupyter_code>%%bash

python -m arachne.driver.cli +tools=tvm model_file=/tmp/resnet50-v2.h5 output_path=/tmp/output.tar model_spec_file=/tmp/resnet50-v2.yaml \
    tools.tvm.composite_target=[tensorrt,cuda]<jupyter_output>[2022-03-15 08:14:02,117][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
[2022-03-15 08:14:03,430][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
call_node: 
free_var %input_2: Tensor[(1, 224, 224, 3), float32];
%0 = (%input_2,);
%1 = call_lowered(@tvmgen_default_fused_nn_pad, %0, metadata={"relay_attrs"={__dict__={"Primitive"=1, "hash"="95b394356d414c2f"}}, "all_prim_fn_vars"=['tvmgen_default_fused_nn_pad']}) /* ty=Tensor[(1, 230, 230, 3), float32] */;
%2 = @tvmgen_default_tensorrt_main_0(%1) /* ty=Tensor[(1, 64, 112, 112), float32] */;
%3 = (%2,);
%4 = call_lowered(@tvmgen_default_fused_nn_pad_1, %3, metadata={"relay_attrs"={__dict__={"Primitive"=1, "hash"="fa69ab88c1d556e6"}}, "all_prim_fn_vars"=['tvmgen_default_fused_nn_pad_1']}) /* ty=Tensor[(1, 64, 114, 114), float32] */;
%5 = @tvmgen_default_tensorrt_main_3(%4) /* ty=(Ten[...]<jupyter_text>Pre-defined Configs for TVM TargetTo ease setup the TVM target, we provide pre-defined configurations for some devices.For example, you can pass `+tvm_target=dgx-1` for Nvidia DGX-1 instead of specifying multiple options.<jupyter_code>%%bash

python -m arachne.driver.cli +tools=tvm model_file=/tmp/resnet50-v2.h5 output_path=/tmp/output.tar model_spec_file=/tmp/resnet50-v2.yaml \
    +tvm_target=dgx-1<jupyter_output>[2022-03-15 08:19:14,651][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
[2022-03-15 08:19:16,001][tensorflow][WARNING] - No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
call_node: 
free_var %input_2: Tensor[(1, 224, 224, 3), float32];
%0 = (%input_2,);
%1 = call_lowered(@tvmgen_default_fused_nn_pad, %0, metadata={"relay_attrs"={__dict__={"Primitive"=1, "hash"="95b394356d414c2f"}}, "all_prim_fn_vars"=['tvmgen_default_fused_nn_pad']}) /* ty=Tensor[(1, 230, 230, 3), float32] */;
%2 = @tvmgen_default_tensorrt_main_0(%1) /* ty=Tensor[(1, 64, 112, 112), float32] */;
%3 = (%2,);
%4 = call_lowered(@tvmgen_default_fused_nn_pad_1, %3, metadata={"relay_attrs"={__dict__={"Primitive"=1, "hash"="fa69ab88c1d556e6"}}, "all_prim_fn_vars"=['tvmgen_default_fused_nn_pad_1']}) /* ty=Tensor[(1, 64, 114, 114), float32] */;
%5 = @tvmgen_default_tensorrt_main_3(%4) /* ty=(Ten[...]
{'fraction_non_alphanumeric': 0.13229894587232027, 'fraction_numerical': 0.0705910221485254, 'mean_word_length': 5.1055676066522055, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '109565', 'license': 'MIT', 'n_tokens_mistral': 3200, 'n_tokens_neox': 2800, 'n_words': 779, 'path': 'examples/run_cli.ipynb', 'repo_name': 'fixstars/arachne', 'size': '8443'}