Source ModelsThe LLM component inside a VLM can be regarded as a standalone LLM, with the extra capability of understanding visual _soft prompts_. From this perspective, by fixing the vision encoder and the projection network and only focusing on the LLM component, it is straightforward to apply the methodologies detailed in Section 3 to produce a new LLM with expanded capabilities.