Inference

#1
by loretoparisi - opened

How to run inference with ONNX runtime?

you can try our experimental code here: https://github.com/bes-dev/stable_diffusion.openvino/tree/sb/onnxruntime
But we don't recommend to use it

Hi, I am trying to convert the ONNX model to a Tensorflow model, to try to use it in tensorflow lite. But I get the following error with both vae_encoder and unet:

File "/usr/local/lib/python3.7/dist-packages/onnx_tf/backend_tf_module.py", line 99, in __call__  *
        output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node,
    File "/usr/local/lib/python3.7/dist-packages/onnx_tf/backend.py", line 347, in _onnx_node_to_tensorflow_op  *
        return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
    File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/handler.py", line 59, in handle  *
        return ver_handle(node, **kwargs)
    File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/backend/conv.py", line 15, in version_11  *
        return cls.conv(node, kwargs["tensor_dict"])
    File "/usr/local/lib/python3.7/dist-packages/onnx_tf/handlers/backend/conv_mixin.py", line 30, in conv  *
        x_rank = len(x.get_shape())

    ValueError: Cannot take the length of shape with unknown rank.

From what I have been seeing debugging the code in the case of the encoder, the input "input.240" of the convolution "Conv_465" is not an array.
Tensor("onnx_tf_prefix_Mul_464:0", dtype=float32)

I attach the code to reproduce it:

import onnx
from onnx_tf.backend import prepare

onnx_model = onnx.load("stable-diffusion-v1-4-onnx/vae_encoder.onnx")  # load onnx model
print(onnx.helper.printable_graph(onnx_model.graph))

tf_rep = prepare(onnx_model)  # prepare tf representation
tf_rep.export_graph("models/vae_encoder")  # export the model

Do you have any idea why this might be or how it could be solved? Thanks!

Sign up or log in to comment