arxiv:2404.03118

LVLM-Intrepret: An Interpretability Tool for Large Vision-Language Models

Published on Apr 3

· Submitted by

Authors:

Abstract

In the rapidly evolving landscape of artificial intelligence, multi-modal large language models are emerging as a significant area of interest. These models, which combine various forms of data input, are becoming increasingly popular. However, understanding their internal mechanisms remains a complex task. Numerous advancements have been made in the field of explainability tools and mechanisms, yet there is still much to explore. In this work, we present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer, and assess the efficacy of the language model in grounding its output in the image. With our application, a user can systematically investigate the model and uncover system limitations, paving the way for enhancements in system capabilities. Finally, we present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.

View arXiv page View PDF Add to collection

Community

BK-Lee

Apr 5

•

edited Apr 5

Thanks for amazing work!

I have one question beyond this kind of interpretability-toward paper.

I am wondering how interpretablity-perspective research go towards for the future?

This kind of investigating what a VL model looks in an image may need to someone who wants to investigate and analyze the model. However, the recent trends show that if we feel a VL model did not work at special capabilities (e.g., detection, counting, color recognition) based on numerous benchmarks (e.g, mm-vet, mm-bench, chartqa, mathvista, and so on), then the world has collected needed data for the insufficient capabilities and generated their corresponding labels. In addition, the world has numerous visual instruction tuning, and they even struggle to scale the model up to encompass the capabilities.

On these recent backgrounds, I would like to ask the authors what the next steps for interpretability without only investigating or analyzing the model may be.