Can you share the steps of how mm_projecter was trained?
#5
by
solankibhargav
- opened
Hi, this almost like llava where you use vision encoder to create embeddings and then send input_embeds to llama.
Can you please share the steps to train the mm_projecter, I want to run this on llama3070b-instruct and would like to know the steps to use your existing weights or train my own module to generate the mm_projecter to run llama3-70b-instruct.