How to use this onnx model

#1
by mkj69 - opened

Ask how to use this onnx model, and especially how to configure this splitter based on these json files?

Owner

Hi @mkj69 this model is not meant to be used. Please use the last up to date optimum version to export decoder models in a single ONNX file:

optimum-cli export onnx --model gpt2 gpt2_onnx

The prefill step needs to pass 0-length past key values (KV cache). You can inspect the exported model with Netron to understand what are the inputs/outputs.

Sign up or log in to comment