@inproceedings{
li2024uniar,
title={Uni{AR}: A Unified model for predicting human Attention and Responses on visual content},
author={Peizhao Li and Junfeng He and Gang Li and Rachit Bhargava and Shaolei Shen and NACHIAPPAN VALLIAPPAN and Youwei Liang and Hongxiang Gu and Venky Ramachandran and Golnaz farhadi and Yang Li and Kai J Kohlhoff and Vidhya Navalpakkam},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=FjssnGuHih}
}
Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes. Yet most prior research has focused on modeling implicit and explicit human behavior in isolation; and often limited to a specific type of visual content. We propose UniAR -- a unified model of human attention and preference behavior across diverse visual content. UniAR leverages a multimodal transformer to predict subjective feedback, such as satisfaction or aesthetic quality, along with the underlying human attention or interaction heatmaps and viewing order. We train UniAR on diverse public datasets spanning natural images, webpages, and graphic designs, and achieve SOTA performance on multiple benchmarks across various image domains and behavior modeling tasks. Potential applications include providing instant feedback on the effectiveness of UIs/visual content, and enabling designers and content-creation models to optimize their creation for human-centric improvements.
UniAR: A Unified model for predicting human Attention and Responses on visual content
Peizhao Li Junfeng He Gang Li Rachit Bhargava Shaolei Shen NACHIAPPAN VALLIAPPAN Youwei Liang Hongxiang Gu Venky Ramachandran Golnaz farhadi Yang Li Kai J Kohlhoff Vidhya Navalpakkam