-48-Dishes Recognition System Based on Deep Learning

: In recent years, food image recognition has emerged as a research hotspot in the field of computer vision. This paper proposes a dish recognition system based on image recognition and deep learning image processing and recognition, which can replace manual dish recognition. Because the food image is complex and dynamic, the system employs a Convolutional Neural Network (CNN). The dish detection algorithm has been completed using CNN. Multiple sets of comparative experiments were designed for the data before and after the improvement, and the ionfluence of the improvement points on the test data results was analyzed.


Introduction
With the rapid expansion of the mobile Internet's scale, sensing technology and artificial intelligence have gradually matured and made significant progress. In the last two years, the payment method in the cafeterias represented by college cafeterias has remained inefficient. This type of canteen has a high volume of people in a short period of time, is noisy and chaotic, does not print receipts, and has management difficulties. It is prone to long checkout lines and erroneous checkout links [1].
At the same time, a dinner plate is frequently required at the checkout. The scene of checking out with a card or mobile phone in one hand, the sound of the dinner plate falling on the floor one after the other during the peak period of the meal. In China, several "unmanned supermarkets" have emerged, improving people's shopping efficiency and eliminating long lines at checkout. With rising labor costs and a trend toward faster-paced living, it is clear that the service models of "self-service" and "machine instead of labor" will replace traditional service methods on a large scale.

Principles of Convolutional Neural Networks
Convolutional Neural Network (CNN) is a feed-forward neural network. Artificial neurons can respond to surrounding units and can perform large-scale image processing [2]. Convolutional neural networks include convolutional layers and pooling layers. Convolutional neural networks are MLPs (multilayer perceptrons) inspired by biological thinking. They have different category levels, and the working methods and functions of each layer are also different.

CNN Network Hierarchy
The CNN network has a total of 5 hierarchies as follows: •Input layer

Convolutional Layer
Local perception: In the process of the human brain recognizing a picture, it does not recognize the entire picture at once, but first perceives each feature in the picture locally, and then performs comprehensive operations on the parts at a higher level to obtain global information [3].
Through the local perception characteristics, the calculation parameters of the model are greatly reduced. But there are still many parameters just like this. This has the weight sharing mechanism: for each layer, the weights corresponding to all neurons should be equal, for example, the parameter vector of a neuron is [w1,w2,...,w100], Then other neurons in the same layer are also [w1,w2,...,w100], which is weight sharing.
Why do we need weight sharing? The connection parameters of neurons in the same layer are only related to the feature extraction and have nothing to do with the specific location, so it can be guaranteed that the connections of all locations in the same layer are weight-shared. For example, the first layer of the hidden layer is generally used for edge detection, the second layer is to combine the edge curves learned in the first layer to obtain some features, such as angle, line shape, etc.; the third layer will learn more complex features, such as: "Eyes, eyebrows," etc. For the same layer, they extract features in the same way. The neurons in the third layer are all used to extract the features of the "eye", so the parameters that need to be calculated are the same.

Incentive Layer
The so-called excitation is a nonlinear mapping of the output result of the convolutional layer.
If the activation function is not used (in fact, the activation function is equivalent to f(x)=x), in this case, the output of each layer is the linear function of the input of the previous layer. It is easy to conclude that no matter how many neural network layers there are, the output is a linear combination of inputs. The effect is the same as without hidden layers. This is the most primitive perception [4].
Commonly used excitation functions are: Suggestion from the incentive layer: ReLU first, because the iteration speed is fast, but the effect may not increase. If ReLU fails, consider using Leaky ReLU or Maxout. In this case, the general situation can be solved. The Tanh function has better results in text and audio processing.
The structure of DNN is not fixed. A general neural network includes an input layer, a hidden layer, and an output layer. A DNN structure has only one input layer, one output layer, and there are hidden layers between the input layer and the output layer. Each layer of the neural network has several neurons. The neurons between the layers are connected, the neurons in the layer are not connected, and the neurons of the next layer are connected to all the neurons of the previous layer.
A neural network with more hidden layers is called a deep neural network (the number of network layers in DNN does not include the input layer). The expressive power of a deep neural network is stronger than that of a shallow network. A neural network with only one hidden layer can fit Any function, but it requires many, many neurons.
 Advantages: Since DNN can fit almost any function, the nonlinear fitting ability of DNN is very strong. Often deep and narrow networks need to be more resource-efficient.
 Disadvantages: DNN is not easy to train, requires a lot of data, and a lot of skills to train a deep network.

Sensor
DNN can also be called multi-layer perceptron (MLP). The network structure of DNN is too complex and the number of neurons is too large. To facilitate the explanation, we design the simplest DNN network structure-perceptron.  Output: y=f(w * x+b)y=f(w * x+b)

Data Collection&Data Augmentation
Comparing to some classical data set including ILSVRC or CIFAR-10/100, we only need to consider a relatively small set of 6 types of cuisines. Basically as the data set for a neural network enlarged, the test performance will be more accurate. Here, we create a small data set that contains 54000 pictures to verify that our algorithm works as expected, which is a good start point to test the codes and make an efficient classifier for dishes. By using web spider and paragraphing on our own, we derive 300 pictures for every 6 cuisines and assign a value as their labels( ). Then, we rotate each picture to a certain angle to gain 9000 pictures per dish. A Gaussian filter is also employed to make the pictures smooth. To avoid aliasing, we preserve the RGB channels while resizing the pictures to 32*32 pixels. Last, we transfer the data to numpy.array with a shape (pic_numbers,32,32,3) and divide the data into train set and test set (ratio = 4:1). The label according to each single picture is transfered to the one-hot-encoding format to compare with the result of classifying later on..

The Architecture
After data collection, we define the model to extract features from those numerical data. Our model contains 6 layers, with 4 convolutional and 2 fully-connected. The first convolutional layer filters the 32*32*3 pictures with 32 kernels of 3*3*3. The other 3 convolutional layers all have the same input and Briefly describe our model as above, it is plain to see that our model has a relatively single size as a dishes classifier only need to distinguish 6 cuisines. In fact, we have tried pictures ranging from 16*16 to 256*256 and found that there will be a prompt of test accuracy from 16 to 32, with negligible gains thereafter. We also notice that the training time is only 2-3 minutes on the 54000*32*32*3 data set, but the time is about 10 times longer when it comes to the same pictures of128*128. So we employ the current model, which is inexpensive and competitive when put into commercial use, since the kinds of cuisines might change frequently and the data will be enlarged. Along with the small imput size, the convolutional layers also have small-scale stride as 1 pixel and kernels. In this way, depth is increased for better performance.
We also employ some techniques to solve the problem of overfitting which will be introduced below.

Activation Function
A standard model to activate an output of an neuron is the function f(x) = tanh(x)or f(x) = (1 + e −x ) −1 . However, the gradient of this function lies between (0 , 0.25), which indicates the gradient of the loss function has the multiply of these minor gradients by 4 times and becomes negligible to back propagation. The chain rule also implies that the different gradient on each layer will lead to a huge discrepancy on different depth. To solve this problem called vanishing gradient, we tried the ReLU function: f(x) = max(0, x) and finally adopted the LeakyReLU: f(x) = { ax, x < 0 x, x ≥ 0 . These functions are one-side saturating, which is better for feature extraction. To simplify, when kernels distinguish features from inputs, a high value represents high level of matching and a low value describes obscure, while a negative value is meaningless. They should be undiscriminating to those mismatching. In spite of this, we still introduce a coordinate to the negative values to prevent dying. The neurons of negative values are part of back propagation and if we set their gradients to 0, the neuron chains containing them will no more able to update.
Using LeakyReLU is fast and accurate. It is 6-7 times faster to use LeakyReLU than tanh(x) unints in our model. Also,when we changed the activation function ReLU, the accuracy will drop by an average 0.5%.

Dropout
When training with a small dataset, overfitting occurs. To prevent overfitting，introducing different classifiers and using the average is a good approach, since they will counterbalance the errors to some degree. Dropout is a method to train different models while less time-consuming than training traditionally. Instead, it hides each neuron in fully-connected layers with equal probability p=0.4 . When a neuron is dropped, it no longer acts in propagation. Dropout only appears at training, and the weights will be divided by (1-p) at test. This method also reduces co-adaptations of neurons. With dropout, two neurons may not occur simultaneously, and each neuron have to learn more robust feature, which exists at other model with different dropout pattern. All models share weights and update one after another.

Training and Testing
We train our model with a batch size of 64, and the data set will be shuffled after certain times of iteration. To initialize, the weights in each layer obey Gaussian distribution with mean=0 and standard deviation of 0.1. Also, the biases are all of constant 1 in every 6 layers. We add regularization to our loss function to prevent overfitting. To specify, the loss function is: Where loss is the mean of softmax cross entropy between logits and labels, γ is the regularization coordinate, W fci and b fci are weights and biases in the ith fully-connected layer.
Once the loss function is derived, we adapt adamoptimizer to update the weights, which is robust and outperforms other stochastic optimizers. The updating rule is:  Where α is the stepsize, β 1 and β 2 are exponential decay rates, ϵ is 1e − 8, and gt = ∇ θft(θt−1). To compare several optimizers, we record the accuracy and iteration steps in fig1. However, compare to the 100% accurate performance on training data, the accuracy of test is considered as weak--less than 85%. By reducing the iteration step to 1400, the model has approximately 94% accuracy of test data but with great fluctuation on both train and test data, since the steps to converge is different at each time. So we use the average accuracy of last 10 batches and set the threshold to 98%.