With the rising popularity of social media and video sharing sites such as YouTube, people worldwide upload numerous instructional videos in diverse categories. Millions of people watch these tutorials to learn new tasks such as "making pancakes" or "changing a flat tire." Analysis of the instructional videos has drawn more attention in recent years, leading to the proposition of several tasks including step localization and action segmentation [1]}. Based on the psychological studies, it has been shown that simplifying and segmenting the video into smaller steps (sub-actions) is a more effective way to learn a new task [2]}, [3]}. For example, the task of "making pancakes" can be segmented to action steps such as "add the eggs," "pour the mixture into the pan," "heat a frying pan," and such. Many datasets are designed to study action localization and action anticipation such as EPIC-Kitchen [4]} and INRIA Instructional Videos Dataset [5]}. Both of these tasks (step localization and action segmentation) are directly related to action detection. Step localization is the task of localizing the start and endpoints of a series of steps and recognizing their labels while action segmentation is the frame-level labeling.
