FIELD OF THE INVENTION

The present disclosure relates generally to optimizing sensor data. In particular, methods for dynamically analyzing and switching between data streams from a plurality of sensors to achieve optimal performance in a target application are described.

DESCRIPTION OF RELATED ART

Most computer technology devices today are equipped with multiple visual sensors. Such sensors can include CMOS and CCD devices that are sensitive to visual light (e.g. RGB), thermal, infrared, or ultraviolet, and may be capable of depth sensing or range finding. These array of sensors are useful for a variety of purposes, such as gesture recognition or 3D visualization and/or mapping. However, each type of sensor has particular limitations. For example, IR sensors may wash out easily in bright light, while RGB sensors can, depending on size and quality, have difficulty providing a clean image in low light conditions. The ability of a device to provide particular functionality can be dependent upon the sensors used.

Known methods of providing sensor data to applications may not be entirely satisfactory for the range of applications in which they are employed. For example, existing systems typically utilize a single sensor, and limit optimization of the data to techniques such as gain boosting and/or noise reduction. Such singular approaches may remain limited by the fundamentals of the sensor technology employed.

Alternately, the use of multiple sensors to accomplish the same task may impose high demands for resources on a given system. For example, while in principle it may be possible to fully process both an RGB data stream and a near infrared (NIR) data stream in parallel for some task such as gesture detection, the computational loads, power consumption, heat dissipation, etc. associated with processing two data streams end-to-end may be undesirable for at least certain applications.

BRIEF SUMMARY OF THE INVENTION

The embodiments are directed to a variety of systems, apparatuses, methods, and paradigms for sensor switching.

In one embodiment of the present invention, a machine implemented method is provided that includes disposing a processor in communication with sensors, each sensor being adapted to provide a primary data stream. The method includes instantiating a sensor controller and an operating system on the processor. The method also includes instantiating a data stream analyzer on the processor, the data stream analyzer including a data stream standard. The method further includes instantiating an input detector, an input executor, and a data stream standard on the processor. The method includes communicating the primary data streams from the sensors to the sensor controller, communicating the primary data streams from the sensor controller to the operating system, and communicating the primary data streams from the operating system to the data stream analyzer. The method also includes comparing the primary data streams against the data stream standard with the data stream analyzer, selecting with the data stream analyzer a secondary data stream most closely matching the data stream standard from among the primary data streams, and communicating the secondary data stream from the data stream analyzer to the input detector. The method further includes detecting with the input detector an input in the secondary data stream, communicating the input from the input detector to the input executor, and executing a control command with the input executor corresponding with the input. The method also includes iterating: communicating the primary data streams from the sensors to the sensor controller to the operating system to the data stream analyzer, comparing the primary data streams against the data stream standard, selecting the secondary data stream, communicating the secondary data stream from the data stream analyzer to the input detector, detecting the input in the secondary data stream, communicating the input from the input detector to the input executor, and executing with the input executor the control command corresponding with the input, wherein a primary data stream selected as the secondary data stream in a first iteration differs from a primary data stream selected as the secondary data stream in a second iteration.

In another embodiment of the present invention, a machine implemented method is provided that includes disposing a processor in communication with sensors, each sensor being adapted to provide a primary data stream, and disposing the processor in communication with at least one data stream recipient. The method includes establishing in the processor a data stream standard, communicating the primary data streams from the sensors to the processor, comparing the primary data streams against the data stream standard with the processor, and selecting with the processor a secondary data stream most closely matching the data stream standard from among the primary data streams. The method further includes communicating the secondary data stream from the processor to the data stream recipient.

The method may include iterating comparing the primary data streams against the data stream standard with the processor, and iterating selecting with the processor the secondary data stream most closely matching the data stream standard from among the primary data streams, wherein a primary data stream selected as the secondary data stream in a first iteration differs from a primary data stream selected as the secondary data stream in a second iteration.

The data stream standard may be specific to the data stream recipient. The data stream standard may be specific to a task for the data stream recipient.

The data stream standard may include an environment factor. The environment factor may include a minimum light level, a maximum light level, a maximum infrared light level, and/or a maximum ultraviolet light level.

The data stream standard may include a content factor. The content factor may include a minimum dynamic range, a maximum noise, a minimum input level, and/or a maximum input level.

The data stream standard may include a function factor. The function factor may include input detection, input confidence, input step detection, and/or input step confidence.

The sensors may include an RGB image sensor, a gray scale image sensor, a NIR image sensor, a TIR image sensor, a NIR time-of-flight depth sensor, an RGB stereo sensor, a grayscale stereo sensor, a NIR stereo sensor, and/or an ultrasonic depth sensor.

Comparing the primary data streams may include assigning a quality value to each of the primary data streams with respect to the data stream standard, and selecting the secondary data stream includes selecting one of the primary data streams exhibiting a best quality value. The quality value may be at least two dimensional.

In another embodiment of the present invention, a machine implemented method is provided that includes disposing a processor in communication with sensors, each sensor being adapted to provide a primary data stream, and disposing the processor in communication with at least one data stream recipient. The method includes establishing in the processor a data stream standard, communicating the primary data streams from the sensors to the processor, comparing the primary data streams against the data stream standard with the processor, and synthesizing with the processor a secondary data stream more closely matching the data stream standard than the primary data stream from at least one of the primary data streams. The method further includes communicating the secondary data stream from the processor to the data stream recipient.

In another embodiment of the present invention, an apparatus is provided that includes a processor and sensors in communication with the processor, each sensor being adapted to provide a primary data stream. The apparatus includes a sensor controller instantiated on the processor and adapted to control the sensors and adapted to receive the primary data streams therefrom, an operating system instantiated on the processor and adapted to receive the primary data streams from the sensor controller, and a data stream analyzer instantiated on the processor and adapted to receive the primary data streams from the operating system, the data stream analyzer including a data stream standard and being further adapted to compare the primary data streams against the data stream standard and to select a secondary data stream most closely matching the data stream standard from among the primary data streams. The apparatus also includes an input detector instantiated on the processor and adapted to receive the secondary data stream from the data stream analyzer and to identify an input in the secondary data stream, and an input executor instantiated on the processor and adapted to receive the input from the input detector and to execute a control command corresponding with the input.

In another embodiment of the present invention, an apparatus is provided that includes a processor and sensors in communication with the processor, each the sensor being adapted to provide a primary data stream. The apparatus includes a data stream analyzer instantiated on the processor and adapted to receive the primary data streams from the sensors, the data stream analyzer including a data stream standard and being further adapted to compare the primary data streams against the data stream standard and to select a secondary data stream most closely matching the data stream standard from among the primary data streams. The apparatus further includes a data stream recipient in communication with the processor and adapted to receive the secondary data stream from the data stream analyzer.

The data stream standard may include an environment factor, a content factor, and/or a function factor.

The sensors may include an RGB image sensor, a gray scale image sensor, a NIR image sensor, a TIR image sensor, a NIR time-of-flight depth sensor, an RGB stereo sensor, a grayscale stereo sensor, a NIR stereo sensor, and/or an ultrasonic depth sensor.

The data stream recipient may include a gesture input detector adapted to identify an input in the secondary data stream.

The apparatus may include an input executor adapted to receive the input from the gesture input detector and to execute a control command corresponding with the input.

In another embodiment of the present invention, an apparatus is provided that includes a processor and sensors in communication with the processor, each the sensor being adapted to provide a primary data stream. The apparatus includes a data stream analyzer instantiated on the processor and adapted to receive the primary data streams from the sensors, the data stream analyzer including a data stream standard and being further adapted to compare the primary data streams against the data stream standard and to synthesize a secondary data stream more closely matching the data stream standard than the primary data streams from at least one of the primary data streams. The apparatus further includes a data stream recipient in communication with the processor and adapted to receive the secondary data stream from the data stream analyzer.

In another embodiment of the present invention, an apparatus is provided that includes means for disposing a processor in communication with a plurality of sensors, each the sensor being adapted to provide a primary data stream, means for disposing the processor in communication with at least one data stream recipient, and means for establishing in the processor a data stream standard. The apparatus includes means for communicating the primary data streams from the sensors to the processor, means for comparing the primary data streams against the data stream standard with the processor, and means for selecting with the processor a secondary data stream most closely matching the data stream standard from among the primary data streams. The apparatus further includes means for communicating the secondary data stream from the processor to the data stream recipient.

