Configuration fix to allow fine-tuning
#3
by
gbarello
- opened
Hi! As per this issue: https://github.com/huggingface/transformers/issues/22903 I believe the default config for the pix2struct decoder model is incorrect and needs to be fixed. Its easy enough to do but it would be great if these models could be updated so that they just work! This applies to all the google/pix2struct-XYZ models.
The fix is straightforward, I believe you just need to do the following and then upload the new model (with new config) in place of the current one:
model = Pix2StructForConditionalGeneration.from_pretrained("google/pix2struct-{model_version}")
model.decoder.config.is_decoder = True