@inproceedings{
bi2024eagle,
title={{EAGLE}: Egocentric {AG}gregated Language-video Engine},
author={Jing Bi and Yunlong Tang and Luchuan Song and Ali Vosoughi and Nguyen Nguyen and Chenliang Xu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=mk8p2JKdu0}
}
The rapid evolution of egocentric video analysis brings new insights into understanding human activities and intentions from a first-person perspective.
Despite this progress, the fragmentation in tasks like action recognition, procedure learning, and moment retrieval, \etc, coupled with inconsistent annotations and isolated model development, hinders a holistic interpretation of video content.
In response, we introduce the EAGLE (Egocentric AGgregated Language-video Engine) model and the EAGLE-400K dataset to provide a unified framework that integrates various egocentric video understanding tasks. 
EAGLE-400K, the \textit{first} large-scale instruction-tuning dataset tailored for egocentric video, features 400K diverse samples to enhance a broad spectrum task from activity recognition to procedure knowledge learning.
Moreover, EAGLE, a strong video-based multimodal large language model (MLLM), is designed to effectively capture both spatial and temporal information.
In addition, we propose a set of evaluation metrics designed to facilitate a thorough assessment of MLLM for egocentric video understanding.
Our extensive experiments demonstrate EAGLE's superior performance over existing models, highlighting its ability to balance task-specific understanding with comprehensive video interpretation.
With EAGLE, we aim to pave the way for novel research opportunities and practical applications in real-world scenarios.
EAGLE: Egocentric AGgregated Language-video Engine
Jing Bi Yunlong Tang Luchuan Song Ali Vosoughi Nguyen Nguyen Chenliang Xu