RuntimeError at loss backward

#33

by franciscoliu - opened Oct 2

Discussion

franciscoliu

Oct 2

•

edited Oct 2

Dear authors,

Thanks for uploading the wonderful model. I am wondering if we can use accelerate to finetune the model using our own custom dataset? I checked the official code and it was implemented using trainer. If so, how would the customize dataloader/data collator be like? (e.g. what are contained in each batch (input_ids, attention_masks.etc) ) Here is the error I have encountered during loss backward:

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

franciscoliu changed discussion title from Fine-tune model using accelerate to RuntimeError at loss backward Oct 2

Sanyam

Meta Llama org Oct 3

Hello, we haven't looked into FT using accelerate, we have a working example for you try here if you'd like to use touchtune: https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/finetuning/finetune_vision_model.md

franciscoliu

Oct 3

Thanks, I have seen this script before. But technically, accelerate should also work for finetuning right? I have implemented an accelerate pipeline for other MLLM models (like llava, idefics) so I suppose it should also work for llama 3.2 vision model?

franciscoliu changed discussion status to closed Oct 4

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment