BACKGROUND

Although automation in manufacturing has been widely adopted and is pervasive, a large portion of manufacturing is still manual in nature. According to a report published by the Boston Consulting Group (BCG) in 2016, 90% of manufacturing today is manual. See e.g., “The Robotics Revolution: The next great leap in manufacturing,” The Boston Consulting Group, 2016, https://www.bcgperspectives.com/content/articles/lean-manufacturing-innovation-robotics-revolution-next-great-leap-manufacturing/. Even in an environment where automation is utilized, there is still a need to utilize human operators. Many issues arise due to the existence of this man-machine interface as humans introduce variance into these processes due to their very human nature. Thus, to reduce the variability, there is a need to conduct frequent inspections to ensure that the product and process meet the required performance metrics. The results of the inspections are analyzed and are utilized to improve metrics like quality, productivity, throughput, warranty and recall rates. Today, many techniques are utilized to conduct these inspections including having dedicated inspection stations, doubling up on human operators so that one operator can observe the other, cameras that feed their output to a central observation station, etc. While this need can be readily understood in a manufacturing scenario, the situation and needs are similar in diverse fields from safely conducting surgeries in hospitals to making burgers in fast food restaurants.

SUMMARY

Given the variability introduced by human operators, there may be a need to observe humans and machines in these man-machine systems in order to understand how tasks are being performed, the performance being measured using appropriate metrics. Additionally, there may be a further need is to then find ways to improve a process being used to deliver a product. Typically, even in state-of-the-art manufacturing floors, these two functions of observation and analysis are performed by disparate systems. An even further need may be to create a knowledge base of various manufacturing practices and to provide tools so that this knowledge base can be queried and the results can be applied to a multitude of purposes such as but not limited to improving part design, suggesting floor plans, improving recall rates, and limiting the financial impact of recalls.

To address the issues above, a feedback generation system is described that has multiple functions including, but not limited to, being able to observe and analyze the observed data and other data. In some examples, the feedback generation system includes one or more observation stations that generate observation data. Thus, observations may be accomplished by installing one or more observation stations that generate observation data. Each observation station may include one or more sensors such as, but not limited to, cameras, infrared sensors, motion trackers, position sensors, thermal sensors and audio microphones. An observation station may be stationary, mobile, worn on the head of the operator, or of some other configuration. The observation data generated by an observation station may include sensor data from the one or more sensors included in the observation station.

Furthermore, the feedback generation system comprises a computation engine. The computation engine may use the observation data from the observation stations as a training set to train a machine learning system. The observation subject may be a process performed at one or more factories, a step of the process, a task of the step of the process, an action of the task, a part used or made as part of the process, or other type of subject. Thus, in some examples, when the machine learning system is sufficiently trained, the computation engine may be able to use the trained machine learning system to identify whether an instance of the observation subject is represented in newly received sets of observation data from the same or different observation stations. As part of training the machine learning system, the observation data can be processed by one or several sub-engines, such as but not limited to, an analysis engine and a process-discovery engine. The output of these engines can be fed back locally to improve the machine learning system.

The feedback generation system can also have various architecture configurations. In some configurations, the entire architecture of the feedback generation system is placed locally. In other configurations, a distributed architecture is utilized. In a distributed architecture, some processing is done locally but some other processing is done remotely, e.g., in the cloud. The results can be made available locally or can be made available globally. In either case, depending on the system settings such as privacy settings, the results can contribute and add to a knowledge base.

A product manufactured in a factory can generally pass through three phases—pre-factory, in-factory, and post-factory. A factory, in the context of this disclosure, can include but may not be limited to manufacturing plants, operating theatres (e.g., for surgery), stores, restaurants, offices, or other locations where humans and/or machines perform routinized processes, tasks, or activities. Pre-factory activities include product design. In-factory activities include process design and personnel selection and quality measurement. Post-factory factory activities include warranty and recall. The feedback generation system described in this disclosure may address all three phases and may provide to feedback to optimize each of the phases, individually and as a whole. In contrast, companies that provide data and insights about factory processes, typically address one or perhaps two of these phases. For example, companies that provide benchmarking data typically rely on surveys that are done in the post-factory phase.

In some examples, the work output of manufacturing facilities as measured by metrics such as efficiency, throughput, recall rates or other parameters, can be improved, driven by a holistic understanding of the various factors influencing the manufacture of a product. In some examples, results of analysis can be applied locally and globally. For instance, an insurance company, not involved in the manufacture of a specific product providing warranty against defective labor, can set the rates of the product from one company based on the knowledge of product quality of the same type of product from other companies. In some situations, worker satisfaction may improve if, for example, the observation station is able to observe the human operators for parameters such as but not limited to posture and eye-strain and suggest improvements to the ergonomics of their individual environments.”

In one example, this disclosure describes a system comprising: one or more observation stations, each observation station of the one or more observation stations comprising a corresponding set of one or more sensors; and one or more physical machines that implement a computation engine configured to: receive first observation data from the one or more observation stations, the first observation data comprising sensor data collected from observing instances of an observation subject, the observation subject being in a group consisting of: a process performed at one or more factories, a step of the process, a task of the step of the process, an action of the task, and a part used or made as part of the process; train, based at least in part on the first observation data, a machine learning system; receive, after the machine learning system has been trained, second observation data from the one or more observation stations, the second observation data comprising sensor data collected from observing an additional instance of the observation subject; use the trained machine learning system to determine feedback based at least in part on the second observation data, the feedback comprising at least one of: an indication of how to improve the additional instance of the observation subject and an indication that the additional instance of the observation subject differs from a reference model; and output the feedback.

In another example, this disclosure describes a method comprising: receiving, by one or more physical machines that implement a computation engine, first observation data comprising sensor data collected from observing instances of an observation subject, the observation subject being in a group consisting of: a process performed at one or more factories, a step of the process, a task of the step of the process, an action of the task, and a part used or made as part of the process; training, by the computation engine, based at least in part on the first observation data, a machine learning system; receiving, by the computation engine, after the machine learning system has been trained, second observation data from the one or more observation stations, the second observation data comprising sensor data collected from observing an additional instance of the observation subject; using, by the computation engine, the trained machine learning system to determine feedback based at least in part on the second observation data, the feedback comprising at least one of: an indication of how to improve the additional instance of the observation subject and an indication that the additional instance of the observation subject differs from a reference model; and outputting, by the computation engine, the feedback.

In another example, this disclosure describes a computing system comprising one or more physical machines that implement a computation engine configured to: receive first data, the first data comprising at least one of: first observation data from one or more observation stations or first input data, the first observation data comprising sensor data collected from observing instances of an observation subject, the observation subject being in a group consisting of: a process performed at one or more factories, a step of the process, a task of the step of the process, an action of the task, and a part used or made as part of the process, the first input data comprising one or more of: a first process chart, first video data, first audio data, a first consumer satisfaction report, first recall data, and first insurance data; train, based at least in part on the first data, a machine learning system; receive, after the machine learning system has been trained, second data from the one or more observation stations, the second data comprising at least one of: second observation data or second input data, the second observation data comprising sensor data collected from observing an additional instance of the observation subject, the second input data comprising one or more of: a second process chart, second video data, second audio data, a second consumer satisfaction report, second recall data, and second insurance data; use the trained machine learning system to determine feedback based at least in part on the second data, the feedback comprising at least one of: an indication of how to improve the additional instance of the observation subject and an indication that the additional instance of the observation subject differs from a reference model; and output the feedback.

The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description, drawings, and claims.

