Mismatched `inputs_embeds` and `logits` shape for decoder_with_past and other decoders?
#9
by
carlesonielfa
- opened
Hi! Thanks for uploading the conversion to ONNX. I'm trying to use this on a Triton server where the merged graph does not seem to be supported. However, I can see that for the decoder_with_past the shapes are
inputs_embeds [batch_size,16,768]logits [batch_size, 16, 51289]
And for the decoder and decoder_merged exports the shapes are
inputs_embeds [batch_size, decoder_sequence_length, 768]logits [batch_size,decoder_sequence_length,51289]
Is this right? Thank you in advance!
Same error