TECHNICAL FIELD

The present disclosure relates to few-shot learning of repetitive human tasks.

BACKGROUND

In modern industrial manufacturing, low-cost smart sensors are commonly utilized to monitor, analyze, and improve the assembly process. In particular, body-mounted motion sensors can continuously record high-precision movement signals. Advanced machine learning systems can then be built on them to perform abnormality detection, efficiency analysis, and poka-yoke feedback etc. At each station along an assembly line, the operator needs to complete a sequence of assembling steps in a standard and timely manner. The human activity involved is often repetitive.

Few-shot learning is a task in which a classifier is adapted to accommodate new classes not seen during training, given only a few examples of each of these classes. Types of few-shot learning models include matching networks and prototypical networks. One of the issues for systems performing few-shot learning is overfitting, in which the error for a training set is driven to a small value, but new data presented to the network gives a typically-unusably large error.

SUMMARY

In one or more illustrative examples, a method for few-shot learning of repetitive human tasks is performed. Sliding window-based temporal segmentation is performed of sensor data for a plurality of cycles of a repetitive task, the plurality of cycles including a reference cycle and one or more operation cycles. Motion alignment is performed of the plurality of cycles, the motion alignment mapping portions of the plurality of cycles to corresponding portions of other of the plurality of cycles. Categories are constructed for each of the corresponding portions of the plurality of cycles according to the motion alignment. Meta-training is performed to teach a model according to data sampled from a labeled set of human motions and the categories for each of the corresponding portions, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variation between the plurality of cycles. The model is used to perform temporal segmentation on a data stream of sensor data in real time for predicting motion windows within the data stream.

In one or more illustrative examples, a system for few-shot learning of repetitive human tasks includes a memory configured to store a few-shot analysis application and motion capture data including a reference cycle and a one or more operation cycles; and a processor, operatively connected to the memory. The processor is configured to execute the few-shot analysis application to perform sliding window-based temporal segmentation of sensor data for a plurality of cycles of a repetitive task, the plurality of cycles including the reference cycle and the one or more operation cycles; perform motion alignment of the plurality of cycles, the motion alignment mapping portions of the plurality of cycles to corresponding portions of other of the plurality of cycles; construct categories for each of the corresponding portions of the plurality of cycles according to the motion alignment; perform meta-training to teach a model according to data sampled from a labeled set of human motions and the categories for each of the corresponding portions, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variation between the plurality of cycles; and use the model to perform temporal segmentation on a data stream of sensor data in real time for predicting motion windows within the data stream.

In one or more illustrative examples, a non-transitory computer readable medium comprising instructions of a few-shot analysis application that, when executed by one or more processors, cause the one or more processors to perform sliding window-based temporal segmentation of sensor data for a plurality of cycles of a repetitive task, the plurality of cycles including a reference cycle and one or more operation cycles; perform motion alignment of the plurality of cycles, the motion alignment mapping portions of the plurality of cycles to corresponding portions of other of the plurality of cycles; construct categories for each of the corresponding portions of the plurality of cycles according to the motion alignment; perform meta-training to teach a model according to data sampled from a labeled set of human motions and the categories for each of the corresponding portions, the model utilizing a bidirectional long short-term memory (LSTM) network to account for length variation between the plurality of cycles; and use the model to perform temporal segmentation on a data stream of sensor data in real time for predicting motion windows within the data stream.

