PRIORITY

This application claims priority to Taiwan Patent Application No. 109132599 filed on Sep. 21, 2020, which is hereby incorporated by reference in its entirety.

FIELD

The present invention relates to an image labeling apparatus, method, and non-transitory computer readable storage medium thereof. Specifically, the present invention relates to a semi-automatic image labeling apparatus, method, and non-transitory computer readable storage medium thereof using clustering technology.

BACKGROUND

Deep learning is a branch of machine learning, which has been widely used in various fields to perform various tasks (e.g., classification and identification) in recent years. Deep learning models adopted in most applications have to undergo supervised learning in advance (that is, they have to be trained with labeled data in advance), and whether a deep learning model can accomplish tasks effectively (e.g., providing high-accuracy classification and providing high-accuracy identification) depends on whether huge amount of labeled data has been provided to let the deep learning model learn the key for distinguishment in the learning stage/training stage. Therefore, providing huge amount of labeled data is an inevitable technical problem in the technical field of deep learning.

Practically, it is difficult to obtain data in some fields (e.g., images of production in manufacturing industry), let alone obtain labeled data in these fields. Although data in some fields can be obtained easily, the cost of manually labeling these data one by one is often unaffordable. Taking textile industry as an example, if a deep learning model that can distinguish various defective and non-defective cloth segments is desired, it is relied on having experts to label the defective images obtained by automatic optical inspection machine(s) and then training the deep learning model by these labeled images. Nevertheless, defective images generated by automatic optical inspection machine(s) in the textile industry are of huge amount and categories of defects (such as broken ends, stop marks, weft crackiness) and fake-defects (such as dirt, creases, cotton coverage) are numerous, which make the task of having experts to examine these defective images one by one and then label them become time-consuming. Furthermore, the textile products are developing towards the trend of little batch and diverse species, which makes images of cloth segments become more diverse. As a result, it is difficult to prepare a fully automatic labeling system that can cope with all kinds of products.

Accordingly, reducing the cost of labeling data by users (such as experts) is an urgent and unavoidable task for technologies related to deep learning.

SUMMARY

An objective of herein is to provide an image labeling apparatus. The image labeling apparatus in certain embodiments may comprise a transceiving interface, a storage, and a processor, wherein the processor is electrically connected to the transceiving interface and the storage. The storage stores a plurality of to-be-processed images. The processor generates an image feature for each of the to-be-processed images and classifies the image features into a plurality of groups. The processor performs the following operations on each of the groups to generate at least one representative feature of each of the groups: (a) selecting the image feature that is closest to a center of the group from the image features of the group as one of the at least one representative feature of the group and as a compared feature, (b) selecting the image feature that is farthest to the compared feature from the image features of the group that have not been selected as a candidate feature, wherein a similarity degree is between the candidate feature and each of the at least one representative feature, and (c) comparing at least one of the at least one similarity degree with a first threshold. When the at least one similarity degree is all smaller than the first threshold, the processor assigns the candidate feature as one of the at least one representative feature of the group and as the compared feature of the next iteration and performs the operation (b) and the operation (c) again.

The groups are classified into a first portion and a second portion by a second threshold, wherein a number of the at least one representative feature of each group of the second portion is greater than the second threshold. The processor classifies each group of the second portion into a plurality of refined groups and assigns the groups of the first portion and the refined groups of the second portion as a plurality of to-be-labeled groups. The transceiving interface receives a plurality of labeling instructions, wherein each of the labeling instructions corresponds to one of the to-be-labeled groups. The processor labels the to-be-processed images of each of the to-be-labeled groups according to the corresponding labeling instruction.

Another objective is to provide an image labeling method, which is for use in an electronic computing apparatus. The electronic computing apparatus stores a plurality of to-be-processed images, and the image labeling method of certain embodiments may comprise the following steps (a) to (g). The step (a) is to generate an image feature for each of the to-be-processed images. The step (b) is to classify the image features into a plurality of groups. The step (c) is to perform the following steps on each of the groups to generate at least one representative feature of each of the groups: (c1) selecting the image feature that is closest to a center of the group from the image features of the group as one of the at least one representative feature of the group and as a compared feature, (c2) selecting the image feature that is farthest to the compared feature from the image features of the group that have not been selected as a candidate feature, wherein a similarity degree is between the candidate feature and each representative feature of the group, and (c3) comparing at least one of the at least one similarity degree with a first threshold. When the at least one similarity degree is all smaller than the first threshold, the candidate feature is assigned as one of the at least one representative feature of the group and as the compared feature of the next iteration and the step (c2) and the step (c3) are performed again.

The groups are classified into a first portion and a second portion by a second threshold, wherein a number of the at least one representative feature of each group of the second portion is greater than the second threshold. The step (d) is to classify each group of the second portion into a plurality of refined groups. The step (e) is to assign the groups of the first portion and the refined groups of the second portion as a plurality of to-be-labeled groups. The step (f) is to receive a plurality of labeling instructions, wherein each of the labeling instructions corresponds to one of the to-be-labeled groups. The step (g) is to label the to-be-processed images of each of the to-be-labeled groups according to the corresponding labeling instruction.

Yet another objective is to provide a non-transitory computer readable storage medium, which stores a computer program comprising a plurality of codes. After an electronic computing apparatus loads the computer program, the electronic computing apparatus executes the codes of the computer program to perform the aforesaid image labeling method.

The image labeling technology (including at least the apparatus, method, and non-transitory computer readable storage medium) provided herein achieves semi-automatic labeling of a plurality of to-be-processed images by adopting clustering technology. Specifically, the image labeling technology provided by the present invention generates an image feature for each of the to-be-processed images, classifies the image features into a plurality of groups, and generates at least one representative feature for each of the groups. According to the image labeling technology provided herein, if a group has a plurality of representative features, the similarity degree between any representative feature in the group and the rest representative feature(s) in the group is smaller than a first threshold. In other words, if a group has a plurality of representative features, the representative features are diverse enough from each other.

If the number of the representative features in a group is greater than a second threshold, it means that the group is not properly classified so that the group is mixed with image features belonging to different categories or/and images having different characteristics. Therefore, if the number of the representative features in a group is greater than a second threshold, the image labeling technology provided by the present invention will further divide the group and may further determine at least one representative feature for each group after the division. The image labeling technology provided by the present invention may repeat the above operations until the number of the representative features of all groups is not greater than the second threshold (i.e., all groups have been properly classified).

The image labeling technology provided herein will consider the groups obtained by the above processing as to-be-labeled groups. Since the to-be-labeled groups have been properly classified, all the to-be-processed images in the same to-be-labeled group can be given the same label. For example, for a to-be-labeled group, the user may decide the label of all the to-be-processed images of the to-be-labeled group according to one or several to-be-processed images of the to-be-labeled group (e.g., the to-be-processed image(s) corresponding to the representative feature(s)) and input a corresponding labeling instruction. The image labeling technology provided by the present invention then labels the to-be-processed images corresponding to the to-be-labeled group according to the corresponding labeling instruction. With the above operations/steps, the image labeling technology provided by the present invention can achieve semi-automatic labeling of a plurality of to-be-processed images and thereby greatly reduce the cost of labeling data by users.

The detailed technology and exemplary embodiments implemented for the subject invention are described in the following paragraphs accompanying the appended drawings for a person having ordinary skill in the art to well appreciate the features of the claimed invention.

