@inproceedings{
gao2024define,
title={De-fine: Decomposing and Refining Visual Programs with Auto-Feedback},
author={Minghe Gao and Juncheng Li and Hao Fei and Liang Pang and Wei Ji and Guoming Wang and Zheqi Lv and Wenqiao Zhang and Siliang Tang and Yueting Zhuang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=gShfQto5eL}
}
Visual programming, a modular and generalizable paradigm, integrates different modules and Python operators to solve various vision-language tasks. Unlike end-to-end models that need task-specific data, it advances in performing visual processing and reasoning in an unsupervised manner. Current visual programming methods generate programs in a single pass for each task where the ability to evaluate and optimize based on feedback, unfortunately, is lacking, which consequentially limits their effectiveness for complex, multi-step problems. Drawing inspiration from benders decomposition, we introduce De-fine, a training-free framework that automatically decomposes complex tasks into simpler subtasks and refines programs through auto-feedback. This model-agnostic approach can improve logical reasoning performance by integrating the strengths of multiple models. Our experiments across various visual tasks show that De-fine creates more accurate and robust programs, setting new benchmarks in the field. The anonymous project is available at https://anonymous.4open.science/r/De-fine_Program-FE15
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
Minghe Gao Juncheng Li Hao Fei Liang Pang Wei Ji Guoming Wang Zheqi Lv Wenqiao Zhang Siliang Tang Yueting Zhuang