BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image capturing apparatus configured to perform image processing of obtaining a feature value from image data obtained by image capturing and to transmit the obtained image data to the outside, and also relates to a system including the image capturing apparatus.

Description of the Related Art

In recent years, internet of things (IoT) has spread also in the field of factory automation (FA). For example, a production device management system including a data management server that collects information from components of a production device is known. In a production device management system of this kind, for example, the data management server analyzes data collected by a method such as deep learning, and there is an attempt to use the result of analysis for prediction maintenance of predicting abnormality of the production device and to analyze the cause or tendency of a trouble when the trouble occurs. For such analysis, image information of a surveillance camera and an image inspection apparatus is useful, and a configuration of collecting image data obtained by these apparatuses by a data management server and using the image data for the management of the production device has been proposed in, for example, Japanese Patent Laid-Open No. 2016-1768 and Japanese Patent Laid-Open No. 2004-333446 shown below.

However, there is a problem that image information occupies a large communication bandwidth when being communicated through a network. In IoT big data processing of recent years, the amount of communicated data has increased dramatically. Therefore, building a wired or wireless network interface (network I/F) that connects to a data management server or the like has become difficult. For example, the increase in the amount of communicated data causes various problems such as increase in the cost and size of an apparatus accompanied by increase in the amount of wiring, and, in the case of wireless communication, insufficiency of number of channels.

Meanwhile, technology has advanced immensely in these years, and an image capturing apparatus such as a so-called smart camera that not only simply performs image capturing but also has a processing capability of performing image processing or image analysis on image data obtained by image capturing has appeared. For example, Japanese Patent Laid-Open No. 2013-158889 shown below discloses a production system in which a robot is controlled by directly transmitting information based on image processing on captured image data from a smart camera that captures an image of an operation area of the robot to a robot control apparatus or the like.

A smart camera of this kind can possible use a built-in I/F capable of performing high-speed communication of image data via near-field high-speed communication without being affected by external disturbance, at low power consumption, and easily. In addition, a network between a smart camera and control apparatus of a production system can be realized by a general local area network: general LAN or wireless LAN that is low-cost such as 100 BASE or 1000 BASE because it suffices as long as a result of image processing not including a large amount of information is transmitted. In addition, the amount of data transmitted from the camera is reduced greatly, and thus the communication load on a management server that performs high-load processing such as big data processing is reduced.

In this manner, in the case where a smart camera is used for production management, the production cost and network cost of each device can be reduced, and there is a possibility that it becomes easier to reduce the overall size and weight of a production management system. In addition, part of processing conventionally performed by the management server can be performed by the smart camera, and there is a possibility that advanced production management, in which production devices and workpieces of many kinds should be quickly identified, can be realized.

However, in the production system including the smart camera described above, the following problem arises in the case of building a production device management system.

A smart camera generates processed information obtained by image processing, a result of analysis or diagnosis based on image processing, or the like. Therefore, although a trouble that has occurred during a normal operation of the production system or a trouble within an expected range can be addressed, the information lacks many elements as information for analyzing an unexpected trouble. For example, there is a case where the information obtained by image capturing by the camera includes information other than a processed image of a workpiece and a defect of the image processing or the cause of the defect can be revealed by a person just looking at the entirety of the image in the time of occurrence of the trouble. In contrast, in the case where only an image processing result is provided, it is difficult to identify the defect of processing and it is also very difficult to estimate the cause of the detect. In addition, for image processing performed in a camera, image processing parameters for adjusting the image processing are needed, and generally, these parameters need to be adjusted in accordance with the temporal change of lighting and an image capturing apparatus, and sudden change in the color of the workpiece. For this adjustment, for example, an original captured image, in other words, a raw captured image that has not been processed or compressed is needed in some cases.

In addition, there is a case where more detailed information is needed when defect or abnormality of a workpiece from a result of image processing during a normal operation of the production system. In such a case, a raw captured image that has not been processed or compressed is sometimes needed.

In such a circumstance as described above, an original or raw captured image obtained by capturing an image of a production device or a workpiece by a smart camera is sometimes needed for, for example, addressing an unexpected trouble, adjusting the camera, or addressing defect or abnormality of a workpiece that has been detected. Therefore, ideally, a configuration in which not only a processing result is transmitted from a smart camera but also an original or raw captured image obtained by the smart camera is logged in a storage device such as a hard disk disposed in a network is desired. However, the size of original or raw image data obtained by the smart camera as described above is large, and there is a case where all pieces of captured image data are transmitted is not practical. For example, in the case where the overall network of the production management system is accelerated in order to let the smart camera communicate a large amount of image data at a high speed, there is a possibility that the above-described merit of using a smart camera is cancelled.

In order to solve the problem described above, reducing the size of the image data or the size of communicated image data can be considered. For example, a method of cutting out a needed part of image data as in Japanese Patent Laid-Open No. 2016-1768 and a configuration of providing an image compression portion such as a joint photographic experts group encoder: JPEG encoder, in other words, a JPEG engine, in a smart camera can be considered. However, in the case of trying to achieve a high compression rate by an image compression method such as JPEG, the image data is irreversibly compressed. In the case where irreversible compression is performed, it is impossible to restore image data before compression from JPEG image data obtained by compression. Therefore, in a system of transmitting irreversibly-compressed image data from a camera to a server, the same data as a non-compressed RAW format image used in the camera cannot be restored from image data remaining in a management server or the like. Therefore, in some cases, it is difficult to finely adjust, for example, image parameters described above.

In addition, although cutting out image data can reduce the amount of information of the image data, the size of the cut out image data is still large compared with the case of performing image compression, and thus cutting out the image data does not reduce the network cost greatly. In addition, in the case where a part needed for target processing does not remain in the cut out image region, the adjustment of image parameters and analysis described above are almost impossible.

Further, a decompressing process is required if an image data is compressed via method such as JPEG, and this decompressing can cause a problem in the response of the system. For example, accompanied by progress of IoT, the amount of data transmitted from each component of a production system to a management server is dramatically increasing, and thus transmitting a large chunk of information such as image data can greatly affect the data response of an IoT communication I/F.

That is, image data, which is large data, needs to be communicated by using a large packet. However, a large packet occupies a communication resource while the packet is transmitted. If the communication resource is occupied by the large image data, communication of other information cannot be performed, and, aa a result, there arises a problem that the control system cannot respond immediately while the packet is communicated even in the case where information with a higher priority has been generated. This problem is called decrease in data response. To be noted, although the data response can be secured by transmitting the image data divided into small packets, in this case, the communication efficiency of the image data decreases greatly, and another problem such as decrease in the performance of the production system that utilizes the image arises.

A production device management system including production device components and utilizing wireless communication is a typical example that cause the problem described above. In wireless communication, the number of channels that can be simultaneously used for communication is limited because communication frequencies are limited. Therefore, in wireless communication, another communication path cannot be secured by assigning a different channel in accordance with priority as necessary. This is because, if all channels are assigned to one production device system, interference occurs between the wireless communication and wireless communication of another production device that is adjacent, and the data response is further decreased. In addition, in a production system of this kind, a device that is convenient when used via wireless communication, requires quick data response, and has a higher priority of processing, such as an emergency stop panel, is disposed. In a production system including such a device as this emergency stop panel, a configuration of transmitting an image captured by a smart camera to an image recording management system via the same network as the network used in the device is not preferred because this configuration degrades the response of another device.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, an image capturing apparatus includes an image capturing portion configured to capture an image of a target object, a controller configured to perform image processing of obtaining a feature value from image data obtained by the image capturing portion, and a storage device configured to accumulate the image data. In an image capturing period of capturing an image of the target object by the image capturing portion, the controller transmits a result of the image processing to a data management server while accumulating the image data obtained by the image capturing portion in the storage device. In an image capturing suspension period of not performing image capturing of the target object by the image capturing portion, the controller transmits the image data accumulated in the storage device to the data management server.

According to another aspect of the present invention, a system includes an image capturing apparatus having an image forming portion configured to capture an image of a target object, a controller configured to perform image processing of obtaining a feature value from image data obtained by the image capturing portion, and a storage device configured to accumulate the image data and a data management server configured to communicate with the image capturing apparatus. In an image capturing period of capturing an image of the target object by the image capturing portion, the controller transmits a result of the image processing to the data management server while accumulating the image data obtained by the image capturing portion in the storage device. In an image capturing suspension period of not performing image capturing of the target object by the image capturing portion, the controller transmits the image data accumulated in the storage device to the data management server.

According to another aspect of the present invention, a method of producing a product using a production apparatus includes a step of capturing an image of the product to be produced by the production apparatus by an image capturing apparatus, accumulating image data obtained by the image capturing apparatus in a storage device of the image capturing apparatus, performing image processing on the image data by a controller of the image capturing apparatus, and transmitting a result of the image processing to a data management server, the image capturing apparatus having an image capturing portion, the controller configured to perform image processing on the image data, and the storage device configured to accumulate the image data. The method also includes a step of producing the product by the production apparatus while monitoring a production state of the product by a production control apparatus on a basis of the result of image processing transmitted to the data management server, the image processing transmitted to the data management server having been performed on image data obtained by the image capturing apparatus in an image capturing period of capturing an image of the product by the image capturing apparatus. Further the method includes a step of transmitting the image data stored in the storage device to the data management server from the image capturing apparatus in an image capturing suspension period of not performing image capturing of the product by the image capturing apparatus.

Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

