Distracted pedestrians, like distracted drivers, are an increasingly dangerous threat and precursors to pedestrian accidents in urban communities, often resulting in grave injuries and fatalities. Mitigating such hazards to pedestrian safety requires employment of pedestrian safety systems and applications that are effective in detecting them. Designing such frameworks is possible with the availability of sophisticated mobile and wearable devices equipped with high-precision on-board sensors capable of capturing fine-grained user movements and context, especially distracted activities. However, the key technical challenge is accurate recognition of distractions with minimal resources in real-time given the computation and communication limitations of these devices. Several recently published works improve distracted pedestrian safety by leveraging on complex activity recognition frameworks using mobile and wearable sensors to detect pedestrian distractions. Their primary focus, however, was to achieve high detection accuracy, and therefore most designs are either resource intensive and unsuitable for implementation on mainstream mobile devices, or computationally slow and not useful for real-time pedestrian safety applications, or require specialized hardware and less likely to be adopted by most users. In the quest for a pedestrian safety system, we design an efficient and real-time pedestrian distraction detection technique that overcomes some of these shortcomings. We demonstrate its practicality by implementing prototypes on commercially-available mobile and wearable devices and evaluating them using data collected from participants in realistic pedestrian experiments. Using these evaluations, we show that our technique achieves a favorable balance between computational efficiency, detection accuracy and energy consumption compared to some other techniques in the literature.
The problem of detecting distracted pedestrians can be generalized as a concurrent activity recognition or CAR problem where the goal is to detect concurrent pedestrian activities of being mobile (e.g., walking, running or climbing descending stairs) and being distracted (e.g., texting, eating or reading). CAR techniques that can distinguish different combinations of elementary activities have been extensively used in the literature for complex human activity recognition. For instance, @cite_25 @cite_5 used and data, from two smartphones, one in trouser pocket and the other on the wrist, to recognize activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. @cite_9 also employed multi-sensor time series data to recognize sequential, concurrent, and generic complex activities by building a dictionary of the time series patterns (called ) to represent atomic activities.
@cite_5 @cite_9 @cite_25
2963574905 2561106390 2533503513 2138302688
Pedestrian safety continues to be a significant con- cern in urban communities with distraction being one of the main contributing factor behind serious accidents involving pedestrians. The advent of sophisticated mobile and wearable devices, equipped with high-precision on-board sensors capable of measuring fine-grained user movements and context, provides a tremendous opportunity for designing effective pedestrian safety systems and applications. Accurate recognition of pedestrian distractions in real-time given the memory, computation and com- munication limitations of these devices, however, remains a key technical challenge in the design of such systems. Earlier research efforts in this direction have primarily focused on achieving high distraction detection accuracy, resulting in techniques that are either resource intensive and unsuitable for implementation on mainstream mobile devices, or computationally slow and not real- time, or require specialized hardware and thus less likely to be adopted by most users. Our goal in this paper is to design a pedestrian distraction detection technique that overcomes some of these shortcomings (of existing techniques) and achieves a favorable balance between computational efficiency, detection accuracy, and energy consumption. Over the last decade, there have been many studies that focus on modeling driver behavior, and in particular detecting and overcoming driver distraction in an effort to reduce accidents caused by driver negligence. Such studies assume that the entire onus of avoiding accidents are on the driver alone. In this study, we adopt a different stance and study the behavior of pedestrians instead. In particular, we focus on detecting pedestrians who are engaged in secondary activities involving their cellphones and similar hand-held multimedia devices from a purely vision-based standpoint. To achieve this objective, we propose a pipeline incorporating articulated human pose estimation, and the use gradient based image features to detect the presence absence of a device in either hand of a pedestrian. Information from different streams and their dependencies on one another are encoded by a belief network. This network is then used to predict a probability score suggesting the involvement of a subject with his her device. Human activity recognition is a challenging task, especially when its background is unknown or changing, and when scale or illumination differs in each video. Approaches utilizing spatio-temporal local features have proved that they are able to cope with such difficulties, but they mainly focused on classifying short videos of simple periodic actions. In this paper, we present a new activity recognition methodology that overcomes the limitations of the previous approaches using local features. We introduce a novel matching, spatio-temporal relationship match, which is designed to measure structural similarity between sets of features extracted from two videos. Our match hierarchically considers spatio-temporal relationships among feature points, thereby enabling detection and localization of complex non-periodic activities. In contrast to previous approaches to ‘classify’ videos, our approach is designed to ‘detect and localize’ all occurring activities from continuous videos where multiple actors and pedestrians are present. We implement and test our methodology on a newly-introduced dataset containing videos of multiple interacting persons and individual pedestrians. The results confirm that our system is able to recognize complex non-periodic activities (e.g. ‘push’ and ‘hug’) from sets of spatio-temporal features even when multiple activities are present in the scene Both detection and tracking people are challenging problems, especially in complex real world scenes that commonly involve multiple people, complicated occlusions, and cluttered or even moving backgrounds. People detectors have been shown to be able to locate pedestrians even in complex street scenes, but false positives have remained frequent. The identification of particular individuals has remained challenging as well. Tracking methods are able to find a particular individual in image sequences, but are severely challenged by real-world scenarios such as crowded street scenes. In this paper, we combine the advantages of both detection and tracking in a single framework. The approximate articulation of each person is detected in every frame based on local features that model the appearance of individual body parts. Prior knowledge on possible articulations and temporal coherency within a walking cycle are modeled using a hierarchical Gaussian process latent variable model (hGPLVM). We show how the combination of these results improves hypotheses for position and articulation of each person in several subsequent frames. We present experimental results that demonstrate how this allows to detect and track multiple people in cluttered scenes with reoccurring occlusions.