A New Framework for Video-based Frequent Iris Movement Analysis towards Anomaly Observer Detection

: This paper suggested a new framework for detecting abnormal behavior, specifically based on frequent iris movements. It contributed to a decision whereas an individual is dubious or unsuspected from a video. One of the key components of questionable observer detection is to detect some specific suspicious activity. According to the writer, various areas of the body movement and human behaviors may be an indicator of suspicious behavior. In this research, we considered the movement of human eyes to identify suspicious activity. This working field is also a significant aspect of machine vision and artificial intelligence, and a big part of the understanding of human behavior. The system framework comprises three parts to monitor suspicious video activities. First, we used the Multi-task Cascaded Convolutional Networks (MTCNN) classifier to detect eyes. Second, we observe irises from eye representations with the use of Circular Hough Transformation (CHT). Finally, we calculated the average distance of iris movement from eye images using a new morphological method called TRM using some properties of the iris movement. We have observed a particular phenomenon of frequent iris movement. Hence, we are making a case of someone being an abnormal person and referring it to a suspicious observer. To vouch for our work, we created our data set with 100 videos where 30 individuals volunteered to validate this research. Each video comprises 200 frames with a duration of seconds. We’ve reached an accuracy of 94% on detecting a frequent iris movement. Rather the goal is to minimize people’s burdens so they can focus on a small range of cases for investigation in more depth. This research’s sole purpose is to indicate a person’s anomalous behavior on the basis of frequent iris movement. Our research outstrips much of the current literature on abnormal iris movement and dubious investigator identification.

In this study, we suggested a technique that could suggest irregular activity and point to unusual human actions by repeated movement of a person's iris. Maximum accused people seek to prevent touch with their eyes and attempt to perceive the environment or circumstances while using the side of their eyes. Such people typically switch their iris vertical or horizontal more frequently than a normal person [20]. When a questionably suspect observer tries to hide his or her goal or remorse, he or she may most of the time stop eye contact, which may be a good sign for a dubious observer. Our research main objective is to help a problematic observer detector from occurring and also a suspicious individual is identified before the offense is committed. The intelligent mechanism that can identify dubious observers; the intelligent machine can track and identify all the questionable observers within its range.

Related Works
Several research papers in the areas of identification of human activity, behavioral understanding, and messy scenario analysis have been reported that apply to video-based anomaly detection clearly or indirectly [25][26][27].
Facial expression and human action recognition are also relevant to this study since any particular facial expression and actions can suggest suspicious conduct. These works are also entirely based on computer vision [23,29].
Noureddine [30] proposed a robust attribute recognition method for frontal face images based on a circular Hough transformation. The edges in the input image, using edge detection techniques, can be detected in the first case. The second is to establish the identifying points of rings from which the symbol of iris is identified. For 396 individuals with 24 frames per human, the proposed algorithm is tested for particular situations in the Caltech Database. 379 pictures of 396 photographs were easily seen, with a performance rate of 95 percent. Significant gains are achieved in a variety of color models. Poor experiments were found for observable objects where Gaussian noise was greater than 0.1.
Depending on deep convolutional network techniques, Zhang [31] has adopted three phases of deep convolutional networks that can predict the gross-to-fine position of the face and landmarks in a large way. A new analysis has shown that in this field, deep learning techniques can produce tremendous results. The author has proposed CNNs for eye detection consisting of three phases: the Proposal Network (P-Net), the Refinement Network (R-Net), and the Output Network (O-Net). Experimental results find that these methods exceed the state-of-the-art methods over multiple demanding tests while maintaining real-time efficiency.
Xiang et al. [32] model both uncommon and natural behaviors. The whole system conserves both the regular and unusual structures. Both normal and abnormal behavioral classes are amended as soon as one of those categorizes the detected behavioral examples. To determine whether a behavioral instance is irregular, the Likelihood Ratio Test (LRT) measures the chance of normal or irregular activity. All dysfunctional and regular behavior trends will lead to their equivalent (e.g. abnormality is usual). This system relies on the weight of each norm and the weight of the two thresholds (i.e. the lowest weight for the normal form of conduct and the highest weight for the abnormal behavior). This work examines more systematically the use of contexts. The study, unfortunately, since the developers have argued that the thresholds are not immune, using such thresholds is one of the system's disadvantages. And if the weight of the action is weighted to one, naturally, the weights decrease with the rise in the number in compartment classes. The threshold values also need to be adjusted accordingly.
Although some approaches [33,34] suggest that contextual information can be used to further enhance system performance, the use of contextual information is still limited in suspicious behavioral detection. Situation information may be used to classify system awareness into awareness groups. In the same context, every other category reflects information gained. The system performance can only be enhanced by using a knowledge culture that is relevant to the current context. The result of failing to consider contextual information may not be that the technique could detect subjective suspicious behavior. Situational suspicious behavior is characterized as behavior that is perceived to be questionable only in a specific context.
From the above discussion it can be said that the respected authors have worked on computer vision regarding human behavior detection which gave us the motivation to start our work in the first place. Hence we have invented our new morphological (TRM) method for the detection of abnormal and anomalous activity based on specific behavioral action is the first of its kind in the field of computer vision.

Proposed Research Methodology
In this article, we presented a new paradigm for the method of identifying a person's irregular behavior from the iris movement. In this step, we detected eyes using the MTCNN method and detected iris using the Circular Hough Transformation (CHT) system. Later, using the morphological model (TRM), we measured the irregular and frequent

A. Data Collection
Our dataset has been generated and comprises 100 videos. Such videos were taken from volunteers. Light requirements were correct, and then all videos were taken from a single phone to show voluntary facial vision. There were also people with unusual eye disorders. Each video has about 150-200 frames and is 6-10 seconds long. Also, every video was taken using only a 12-megapixel cell phone with a resolution of 720x1080 pixels. The distance from the object to the human being is about 1 to 3 meters. Growing the gap is going to make the machine unstable, so we took all the videos from a closer range. Fig.3 illustrates sample images of our dataset.

B. Data Preprocessing
Data preprocessing is a very important component of image analysis. Every other frame in each video has a resolution of 720 x 1080p. The histogram was used to evaluate the range of the lightness of every bottle. Alternatively, noise control is conducted with the help of certain morphological operations: closing, erosion, and opening. We have created an image pyramid, an input for the detection of eyes using MTCNN. And after that, we re-dimension the pyramid at a higher size.

C. Eye Detection using MTCNN
Once the frames have indeed been preprocessed, the input images in the MTCNN are used to identify the eyes. It has to go through three phases [31]: 1) Stage-I: This phase is known as the Proposal Network (P-Net) which receives the openings of the candidate and the vectors of its boundary box. So we use bounding box regression vectors that are estimated to control the participants. Non-maximum suppression (NMS) may then be used to merge strongly reflected participants.
2) Stage-II: The next CNN, dubbed the Refine Network (R-Net), also removes a significant portion of the undesirable contestants, advocates boundary box regression and incorporates NMS members.
3) Stage-III: This step is the same as the previous phase, except that the mask seems to be better defined at this step. The circle is drawn to highlight and, eventually, to differentiate the eyes from the face.

D. Iris Detection using Circular Hough Transformation
After having successfully defined the eyes of our study, we planned to detect the true border of the iris and finally detect the iris. This is accomplished by the Canny Edge Detection and Hough Transformation method. The key stages for iris detection: median filter and gamma correction implementation and applying CED & CHT.
1) Median Filter: This median filtering before processing is used to eliminate noise from images. Here, the median is determined by combining all pixel values from the screen to the quantitative series then switching the pixel to the middle (median) pixel value. The example of applying a median filter to an eye image from our dataset to reduce image noise is given in Fig.4. 2) Gamma Correction: As our system does not capture and display the luminescence properly, the gamma correction is being used to correct the pixel fluorescence. It can also contribute to low contrast if the gamma value is too high or too small. An example of applying gamma correction to an eye image from our data set to rectify the luminescence of the image is illustrated in Fig.5. 3) Performing CED and CHT: Initially, edges are detected using Canny Edge Eye Image Detection, which has been analyzed and extracted from our dataset. The iris is determined from the eye images after the Hough Transformation process has been applied.
In our proposed approach, the Canny Edge Detection algorithm was used to categorize edges and at the same time suppressing noise. A Gaussian filter has been used to eliminate noise and unwanted bits.
After the execution of the CED, we applied the Circular Hough Transformation, which further allowed circular objects to be ejected from the image, even if the circle is incomplete. The Hough transformation can be defined as a transformation of the X, Y-plane space parameter. Statistically, the circle equation in x, y-plane is given by Where the circular radius is r and the center of the circle is a, b. The parametric method of the sphere is given by When the circle's coordinates are x, y, the circle's radius is r, and t is the parameter that defines the angle of the circle at the center of the circle. The circle is shaped at the point of the bottom in the middle of the point, taking the optimal distance into account. Miscellaneous circles may cause false centers to be observed in the image of the eye. Across the whole phase of effective iris detection, we identified that the middle point of the first circle was between all the other center points of the circles created for the edge points [35]. Throughout this process, we regarded parameters such as the Hough parameter, canny parameter, Maximum radius, and Maximum distance, where a certain value of the parameters is adjusted to locate the iris with the accuracy of the eye pictures.

E. TRM Model to Calculate the Average Distance of Iris Center Point and Frequent Iris Movement Detection
Upon detection of the iris from eye images, we noted the change in the x, y coordinates of the iris center point for consecutive frames in the video. Then, finally, a new morphological model (TRM Method) was used to detect the anomaly and abnormal iris movement from one eye. As a result, we can indicate and detect abnormal human behavior through frequent iris movements which try to avoid eye contact eventually. This research will divide the unusual movement of iris from the normal movement of the iris by checking the significant change in the distance from iris centroid. Initially, we took the center of the iris to measure the distance from the initial position of the iris to the position of the next frame. Then check all the frames of more than 50 videos. We have calculated the average change in distance for iris centroid point from the initial center point. The average center coordinates distance (Center X, Center Y) of the iris is detected by using the formula: Where, = Total number of frames, = Current frame x coordinate value, = Next frame x coordinate value, The x coordinate of the average distance of the center point is determined by dividing the sum of the distance crossed by the x coordinates of the iris by the total number of frames. In the same way, the y coordinate of the average distance of the center point is determined by dividing the sum of the distance crossed by the y coordinates of the iris by the total number of frames. By using this average distance from the iris center point, we observed our detection in the next step.
In all successive frames, we recorded the range of motion of the center point of the iris. Calculating the average distance value of the motion of the center point of the iris, we set the threshold value by testing our research with 50 videos. After viewing these 50 videos effectively, we found that when a person attempts to prevent visual contact or changes the iris rapidly from right to left, or vice versa, the x-coordinate range will fluctuate significantly from frame to frame, and the average distance of iris center point's x coordinate will be greater than that of a person who does not try to avoid eye contact or shifts iris frequently. In the same way when a person attempts to prevent visual contact or changes the iris rapidly from right to left, or vice versa, the y-coordinate range will fluctuate significantly from frame to frame, and the average distance of iris center point's y coordinate will be greater than that of a person who does not try to avoid eye contact or shifts iris frequently. Therefore, when measuring the average distance value of the iris center point of the video, we detected irregular and unusual iris motions. This work will group the behavior of avoiding eye contact and significant movement of the iris if the average distance value of the iris center point of the video is greater than the threshold value that we have assigned.   Fig.7 shows the average distance of iris center point movement of (x, y) coordinates in our dataset for a nonavoiding and normal iris movement case. It represents the variation of average distance of iris center point of both x and y coordinates for the case of non-frequent iris movement.
From Fig.7 we observe that the x coordinates of the average distance of the iris center point in consecutive videos vary from 2 to 15 with a range of 13 and the y coordinates of the average distance of the iris center point in consecutive videos vary from 1 to 9 with a range of 8.
From these two above instances, discrepancy in the distribution of the average distance of the iris center point coordinates variance can be seen. In avoiding cases and frequent iris movement cases, the range is much higher than that of non-avoiding cases and normal iris movement cases. Upon carefully reviewing the 50 videos, which involve both regular iris movement and non-frequent iris motion images, a threshold value is set for the average distance of both x and y coordinates of the iris center point. Taking into consideration the threshold value, a video is identified as "frequent iris movement" and "avoid case" if the average distance value of the iris center point for either x or y of the video coordinates remains higher than the threshold value. On the contrary, a video is classified as "normal iris movement" and "non-avoiding case" if the average distance value of the iris center point for either x or y of the video coordinates remains lower than the threshold value. This threshold value was used in our further experiments to successfully identify suspicious and normal human behavior on the basis of frequent iris movement criterion.

Experimental Results and Discussion
To explain our proposed framework, a comparative study of our proposed unusual and irregular observer identification program with other implementations is seen after our dataset is used as a test data. Our analysis is performed in two steps. 100 frames per video were considered to be monitoring the outcome of abnormal behavioral detection in the first experiment, while in the second experiment, 150-200 frames per video were taken into account for observing the outcome of abnormal behavioral detection. Our analysis is performed in three major phases. By using MTCNN, we have attained 98 percent precision of eye detection where our dataset has been used. The outcome of the previous phase is used to identify the iris of the eye pictures. 73% accuracy is achieved for our dataset when CHT is used to detect iris. In the last phase of abnormal behavior detection using frequent iris movements, we achieved 94% accuracy by using the TRM model and our dataset. This significant result has been achieved in terms of frequent iris movement and TRM model. Later, our results have been compared with other established research that concentrated on accuracy, consistency, and recall. Finally, our work is combined in the relevant field of object detection with existing work. Parts A to D give a detailed review of our study outcomes and discussion which has been reflected in Fig. 8.

A. Experimental Set Up
At first, the movements of iris were detected and observed from the eye images that we found after the eyes were detected using our dataset. Then the TRM method was used to find out the abnormal behavior in the iris movement. Every one of the implementations has been performed with OpenCV on the python platform. Fig.9 displays the reference images from our dataset where both irises from the images have been successfully determined.
After detecting both iris from eye images the movements of iris were identified by observing the coordinates of iris center points for consecutive frames in a video. Here, one iris was considered for further process for the simplification of our model and implementation. Besides, because a person's both iris move in the same manner always while the movement of the iris, one iris is taken to find out the abnormal behavior of an observer. Iris movement is plotted in Fig.10 to show the variance between iris center points for our dataset. It elucidates that iris are continuously moving from its initial point and it will lead us to our morphological model to detect abnormal and suspicious frequent iris movement.  Then using our morphological model (TRM) the average distance of iris center point for each video was calculated and a threshold was set and while testing the rest of our dataset we observed frequent iris movements of different persons who were trying to avoid eye contact whereas some normal iris movements are detected where none were trying to avoid eye contact and were normal. Fig.11. Frequent movement of one iris in successive frames for a person who wishes to escape eye contact In Fig.11 it is observed that an individual tries to prevent eye contact and one iris is constantly moving which points to an abnormal and suspicious characteristic of humans.

B. Dataset
Due to the uncertainty of anomalies, the context in which the incidents occur, the lack of appropriate sizes of the abnormal ground truth training data, and other factors relevant to changes in other environments and lighting conditions, the detection of abnormal characteristics from a video are very difficult. Since we do not find any appropriate datasets available particularly to verify our research, we have generated our dataset. Hardly any accessible dataset contains videos that often allow participants to transfer iris from left to right or from up or down or conversely. We generated a dataset called "TRM Iris Dataset" for this reason which includes a total of 100 videos with about 18265 frames. The participants frequently switch their irises in 44 videos, and 56 videos either don't switch or, in a variety of words, are in a constant position. For the first 100 (2-4 seconds) and 200 (6-10 seconds) frames of each video, we conducted our experiment.

C. Experimental Results
Based on the threshold value we inspected the abnormal and suspicious behavior after the evaluation of frequent iris movement and avoiding eye contact by using the TRM method and TRM Iris Dataset.  In Fig.12 we have illustrated the average distance of iris center point (x, y) fluctuations for frequent iris movement cases as a histogram which eventually indicates to the suspicious observer who tries to evade contact with the eyes.
In this case, since x and y coordinates are higher than the threshold for the average iris center point for most videos, the videos were described in our system as "frequent iris movement" and "eye contact avoidance". Similarly, In Fig.13 we have illustrated the average distance of iris center point (x, y) fluctuations for non-frequent iris movement cases as a histogram which eventually indicates to the normal observer who does not try to evade contact with the eyes.
In this case, the videos are referenced as "non-frequent iris movement" and "non-avoiding eye contact" in our system, since x and y coordinates are less than the threshold of the average iris center point for most videos.
By observing the above two cases, a feasible threshold was taken for further experiments and evaluation tests. The experiments were conducted in two parts. Firstly, the confusion matrix of abnormal and suspicious iris movement was calculated by considering the 100 frames (2-4 seconds) from each video from the TRM Iris Dataset. It was drawn in Table 1. Here, the confusion matrix was generated using 50 videos where the first 2-4 seconds were counted from each video. We can observe here, for the first 100 frames or the first two to four seconds, that our model identified 18 videos containing frequent iris movements and 26 videos containing normal iris movements. The accuracy rate was 88%. Where the precision was 81.82% and recall was 90%. The specificity rate was 86.67% and finally, the F-1 Score was 85.71%. However, the error rate of this experiment was 0.12.
Additionally, the first experiment could point out a suspicious or normal video with lower accuracy. Secondly, we have calculated the confusion matrix of abnormal and suspicious iris movement by considering the 150-200 frames (6-10 seconds) from each video from TRM Iris Dataset. It was drawn in Table 3. Here, the confusion matrix was also generated using 50 videos where the first 6-10 seconds were counted from each video. Table 3. reveals that, in the first 150-200 frames or the first 6-10 seconds, our proposed model identified 19 videos containing recurrent iris movements and 28 videos containing non frequent iris motions correctly which is significantly better than the one based on the first 100 frames per video. Here the True positive rate and True negative rate are 19 and 28 consecutively. And the False Positive and False Negative rate are 2 and 1 one after another. From Table 4. We observed that the accuracy rate was 94%, precision was 90.48% and recall was 95%. The specificity rate was 93.33% and finally, the F-1 Score was 92.68%. In addition, the error rate of the second experiment was 0.06.

D. Analysis and Discussion
In this segment, we compared our research with previous works on dubious observer detection. In Table 5. We discussed the confusion matrix of the previous morphological method and the TRM method. Here we can see our recall rate is higher than the existing morphological method whereas the precision rate is lower than the previous morphological method. The F-1 score is identical. But our accuracy rate is slightly higher than the existing method.
A comparison between our two experiments to detect abnormal and frequent iris movement for 100 frames and 150-200 frames has been illustrated in Fig.14. This figure perfectly describes the significant change in result in terms of identifying abnormal and frequent iris movement for our two different experiments which were considered by taking the first 100 frames per video and the first 150-200 frames per video consecutively. In Fig.15 a detailed comparison has been observed between our TRM method with the previous morphological method in terms of identifying frequent iris movement. After evaluating the outcome for our two experiments we can come to the conclusion that the result for 150-200 frames is better than the experiment for 100 frames. And it indicates to detect someone's behavior as abnormal and frequent iris movement we should consider 6-10 seconds of video. So, we need to observe a particular person for about 6-10 seconds to get the desired output.
Finally, the contrast of our findings with known works was demonstrated in Fig.16. In Fig.16, the first two labels reflect the real-time identification of aggressive crowd activity dependent on the Assembled database and the Hockey database, where the accuracy was 81.3 percent and 82.9 percent successively for the Assembled database and the Hockey database [24]. The labels UMN and WEB constitute 96 percent of the UMN dataset and 73 percent of the web data of work conducted by Mehran and others [36]. They developed a new approach for detecting and evaluating odd behaviors in crowd videos based on the concept of visual flow and using the social force model. The mark Motion Detector represents Yasin et al.'s accuracy in the research [37]. Their human skepticism and irrational motion have been categorized based on instant invariants with an accuracy of 87.6 percent on their own data collection. The segmentation mark of Face Track identifies Barr et al's attempts to identify suspicious observers on the ND-QO-Flip with a precision of 96 percent using their face track grouping method [22].
The Fig.16 mark for the Public Transportation Field shows the precision of the identification work of Elhamod et al. The BEHAVE CAVIAR and PETS 2006 data collection has achieved an accuracy of 66 percent [38]. The second last bar represents accuracy [20] with a data set of its own. They also reached an accuracy of 93 percent. The experiment was carried out with the first 150 frames of their dataset. Finally, the last bar represents a new morphological TRM method for the identification of behavioral abnormalities based on frequent iris movements where 94 percent accuracy was achieved. The experiment was carried out on the basis of 150-200 frames.
Evaluating all the results of the existing works, we can conclude that we have successfully detected iris from the eye whether it is a suspicious person's iris or not based on specific criteria. In this regard, we have used our own generated method called TRM and achieved an accuracy of 94%.

Conclusion
In terms of identifying a suspect individual, the regular monitoring of iris activity is one of the most significant ways of identifying abnormal human behavior. The detection of the dubious observer has several important connections with law enforcement personnel. Not only can fraud be stopped and identified in shopping centres or elsewhere, but even violence or events in public spaces can be avoided. Saving human lives and a wide variety of goods and resources will also be possible. Presently, no work has been conducted using TRM method, and so this article will pave a new way to detect dubious actions of a person. This research described the methodology and techniques used for the recognition of activities, and also their data set and accuracy. In comparison, this work identified two basic concepts for the definition of an abnormal observer to reflect one of the features of a questionable observer, while some other odd acts and actions may categorize one individual as a doubtful observer. Frequent iris movements and avoiding eye contact have been concentrated here as unusual and suspicious actions. We have successfully achieved an accuracy of 94 percent because of these specific actions. The creation of detailed datasets in various circumstances to validate the proposed context is one of the most challenging tasks in this research. The datasets used in this research include a different environment form, lighting, facial expression, and eye movement for testing the proposed structure. Taking into account the vision-based tracking function, all sensor-based response identification limitations were resolved. This research has not only introduced a new and significant method (TRM), but has also constructed a new data set for the