Issue with evaluating during training using weighted layer sum
#10
by
kimedaka
- opened
Hello @sanchit-gandhi ,
When config.use_weighted_layer_sum = True, eval_pred.predictions in compute metric method in Trainer object returns two dimensional tuple/array instead of one dimensional at evaluation step during training. Consequently, it throws the following error:
setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Also, it takes more than 40 GB or GPU RAM at evaluation step during training with Whisper base model when config.use_weighted_layer_sum = True.
Is it a bug or am I mistaking something.
Regards,
K