Overview of the application of computer vision technology in fish farming

The issues that are currently identified in Russia during the implementation of Digital Agriculture project are considered. The main issues that need to be addressed in development of modern digital technologies in the fish farming industry using natural and artificial reservoirs are highlighted. Aqua engineering trends and scientific works of a number of teams that conduct research and use the capabilities of deep machine learning, are analyzed. Particular attention was paid to specific tasks and research results that solve applied problems in the field of aquaculture and fish farming. Conclusions are made about the prospects for implementing these objectives in Russia. The conclusions of scientific teams and new tasks set as a result of scientific experiments are considered. The main directions in the area of commercial fish farming that need active adaptation of computer vision to deal with applied problems, are identified. Questions of efficiency in introduction of neural networks of deep learning are raised, and also conclusions are drawn on introduction of the term "selectivity" to determine the relation of a data set received by a digital method, referred to quantity of the same data which would be received at their collection by means of non-digital technologies.


Introduction
With the growth of computing power, libraries for software development, popularity of digital technologies and at the same time simplification of methods of mastering programming languages, there was a call for deep digitalization of the most specific areas of national economy. For example, the team of authors of this article has been actively developing digital solutions for the problems of fish farming in the Tyumen region [1,2]. However, this article is of a review nature, since a detailed analysis of foreign practices at the "junction" of IT and fish farming is interesting, first of all, because it will make it possible to form a spread of areas that are relevant for fish farming abroad. Certainly, it is necessary to make a reservation that problems and solutions of foreign fish farming do not always correspond to the problems and solutions of Russian fish farming, however ignoring the given experience would be at least ineffective. The team of authors has analyzed a number of articles  and summed up the applicability of each solution to the identified problem in fish farming in the Tyumen region. In this regard, the logic of the presentation will follow the principle "information given in the article -summing up -applicability for fish farming in the Tyumen region".

Computer vision for fish feeding
In the first part of the article, the team of authors solves the problem of determining spatial feed distribution in the sea cage. It should be especially noted that it was the specifics of feed distribution by rotary spreaders were taken into account, which was highlighted separately in the article.
The emphasis is made to the following: 1. Experimental confirmation of the operability of the installation for measuring the degree of distribution of fish feed pellets.
2. Check of the possibility to distribute fish feed using the unmanned aerial vehicle (UAV).
3. Evaluations of the efficiency of pellets distribution using computer vision technology (by quantitative analysis of disturbances on the water surface).
Within the framework of the review, the last stated emphasis is of particular interest. The fish feeding procedure was recorded by an UAV, then the images were analyzed with the help of computer vision technology of disturbances appeared on the surface ("peak" of pixels that are brighter than pixels in the immediate neighborhood). As a result of the analysis of that activity from the UAV video records, the model of pellets distribution by rotary spreaders was received. The method adequacy for analyzing the feed pellets distribution using computer vision was confirmed experimentally, since the results obtained were confirmed by duplicating the results obtained through "manual" analysis.
As a result, the team of authors proved that with the help of computer vision, it is possible not only to make an actual assessment of the efficiency of feed distribution by various spreaders, but also to forecast distribution efficiency. Also, the provision was made to the further generation of "digital twins" for spreaders and for testing of simulation models which can take into account all kinds of spreader structures, pellet weights, etc. Unfortunately, this technology is not applicable to fish farming of the Tyumen Region, since the area of cages in the region is much smaller (due to specifics of river and lake fish farming). However, the digital part of the technology is capable of assessment of the extent of feed spreading in the area, but may also count fish by splashes. Therefore, the developed technology after some modification can be quite useful for the region as an automatic counting technology.

Computer vision for measuring fish parameters
A large list of articles analyzed by the team of authors was related to the development of technology for automatic measurement of fish parameters, based on computer vision technology. The availability of such a significant number indicates, first of all, to the demand of such application of computer vision technology in fish farming. For example, the scientific team of V. Puig-Pons et al., (2019) carried out a study related to testing 2D computer vision technique for measuring a fish parameter such as mass. For the object of study, they took exclusively "jade perch" which moves freely in a closed water reservoir. The study was divided into two stages, the first of which was to make the program accumulate the dependence of the fish shape (as I understand it also included a parameter of its size) on its weight, the second stage was to compare the actual weight of fish with the weight predicted by the computer program. The resulted determination coefficient (R) was 0.99 and a relative error of 6 ± 3 per cent compared to the value measured by the weighing scale, so computer vision technology has proven its adequacy when used in that way. By the way, only 120 images were needed to train the program.
A similar study was carried out by the team of Mohammadmehdi et al., (2018), but the difference with the previously mentioned work was in the use of infrared reflection, since the geometry of the fish back was used for learning, which allowed (unlike the previous team) to go beyond a particular fish species. The image library used 295 farmed sea basses of different species and sizes. The mathematical model was optimized due to ten-fold cross-validation of model prototypes (about ten in total). As a result of the study, a viable, maximally approximate and trained model for distinguishing the mass of fish by its shape features was obtained. The determination coefficient (R) is lower than in previous studies and is equal to 0.868, which is most likely due to the diversity of fish species. At this point, we understand that such technology will be ineffective if used in the "wilderness" without prior observation and development of a library assigned to a flock of fish being measured. But the authors state that their development is applicable to forecasting in all conditions. It would be interesting to test this technology in the conditions of river fish farming, because the degree of water transparency is obviously different and, as a consequence, it is complicates the ability of computer vision technology to analyze the parameters of fish. Such a solution can be adapted to the realities of fish farming in the Tyumen region. However, the technology needs to be improved for better analysis in conditions of complicated visibility and water contamination by large objects.
In the following article the development of computer vision library was improved by auxiliary library of acoustic vision. The result is a simultaneous convergence of technologies (acoustic locator and computer vision) enhanced by machine learning algorithms. The authors' team solved the problem of estimating the mass of tuna during transfers. The libraries were developed with side-scan sonar that operates at a frequency of 200 kHz, and stereo cameras that were positioned to the surface to register the ventral aspect of the fish. Such use undoubtedly reduced the determination coefficient, but the task was also to measure the total mass of all tunas rather than a specific specimen.
The results showed that the system provides automatic counting of tuna with an error below 10%, reaching about 1% error in the best configuration.
This study confirmed the improved efficiency of the results when combining computer vision with the analytics of an auxiliary database (in this case, the acoustic one). It would be very interesting to compare the effectiveness of a number of hybrid technologies (computer vision -acoustics, computer vision -splash analysis, etc.) and to compare which of the combinations of technologies give the most accurate result.

Computer vision for monitoring fish behavior
Researches of various types using machine learning have been conducted for a relatively long time.
The scientific team at Saberioon et al., (2016) conducted a study aimed at developing the tracking of multiple fish in three-dimensional space using sensors.
The value of image-based tracking is to provide automatic remote tracking and analysis of fish movements without human presence. In the direction of fish farming, observation and analysis of fish behavior has been carried out for a long time, for example, the scientific teams of Spampinato et al., (2012) recorded fish behavior using an underwater camera to understand how individual species behave during typhoons. Currently, the number of parameters that can be tracked using machine learning is quite large. For example, Kato et al. (2004) developed a system to quantify the behavior of individual fish, such as speed, distance that an individual can travel, and a map of individual movements. The use of machine learning for research in this area has been going on for a long time. The most common technical problem encountered by researchers in the study of fish behavior through machine learning is overlapping of several fish images during movement. Such effect occurs when several fish are moving simultaneously, or when some individuals come in close proximity to the other. This crossing can be displayed when recording in one plane, at the same time, while recording in another plane, it is visible that that the individuals are at some distance from each other.
There are several types of crossing while recording. Generally, there are two main types of such crossings: when two fish individuals swim so close that they appear as one long fish in the image, and the second type is when the trajectories intersect and the two fish are perceived as T-shaped and X-shaped images (Dolado et al., 2014). Various machine learning algorithms are used to recognize inaccurate data.
The research team considered the possibility of using special sensors permitting to receive three-dimensional image. Such sensors generate a three-dimensional geometric field to obtain depth information from objects of interest. This makes it possible to develop an affordable system for tracking and studying multiple fish in real time. Here we again encounter the use of a number of databases, which gives us an understanding of synergy of different source databases affecting the efficiency of the result. The technology is applicable to fish farming in Tyumen Region and can be used with no modifications.
The research team conducted laboratory tests using sensors for tracking and precise measurement of the three-dimensional trajectory of several fish, even in the conditions of their trajectories constantly and repeatedly crossed.
The developed and tested system in the aquarium has shown good potential, and has led to conclusions about the prospects of its application for monitoring and research of fish behavior in groups, rather than just to consider their individual behavior. Such system enhances the possibilities to study fish behavior by setting and studying more parameters, such as speed and interpersonal distances.
Other research teams faced similar difficulties.
In the next article the team of authors conducting laboratory tests of the installation using computer vision focused on processing a large number of quality images obtained from high-speed cameras. The goal set by scientists was not only to study the behavioral patterns of individuals of the fish in research, but also the duration and stability of image processing. Experimental observations made it possible to draw conclusions and recommendations regarding the necessary characteristics of the number of photos per second required for detailed analysis. The developed proprietary algorithm allowed to form further tasks connected with automatic determination of fish movement and body position, as well as spatial position. Implementation of research using computer vision has been actively used by specialists from various countries over the past decade. But unfortunately, the use of computer vision in fish farming in Russia has not become widespread, despite the fact that the industry has set applied tasks that can be solved only with the help of deep-learning neural networks.

3.3.Computer vision to study the nets integrity
For aquaculture enterprises, the integrity and safety of fishing nets is of great importance. The team of the article's authors Yun-Peng et al., (2020) is conducting a study using deep machine learning technology for research in this field.
The development of small-water aquaculture, namely coastal aquaculture, is becoming an important area of development and breeding of marine fish in China. This helps to achieve sustainable development of fish farming. A section of the allocated water resource is fenced off for use by special nets. This is done to keep the farmed fish inside a confined space, to track its population, growth, nutrition and to conduct appropriate monitoring.
However, damage to the prepared cage (Heidi et al., 2009;Hoy et al., 2012), namely holes in the cage, allows fish to enter the main water reservoir and therefore leads to losses for the aquaculture industry and has serious consequences for the ecosystem as a whole. Detection of net cage damage consists in determination of the mesh gaps between each other. Currently, there are several methods to conduct the netting integrity check, and one such method is regular netting examination by professional divers. This process is very time-consuming and not very efficient, and also presents a high safety risk. Therefore, detection of netting damage is a key problem in aquaculture development related to fish farming.
The team of authors suggests using two ways to detect holes in fencing nets. The research demonstrates the results of experiment of the netting examination with a special robot that takes high-resolution images. Further, the authors suggest two ways to process the obtained high-resolution images.
One method is based on the neural network training to recognize nodes in netting, and thus a rule for the location of node points is developed. The second method suggested by the scientists is based on recognizing holes in the netting by a special algorithm. Testing both methods to detect the netting holes has led to the conclusion that one method is mode advantageous than the other. For example, the method of hole detection in the netting can be used to detect the broken areas even when the netting is partially contaminated. This method also makes it possible to estimate how much biologically contaminated the netting is and to identify the extent of damage.

Conclusions
Summing up the analyzed articles, a number of common factors can be identified: 1. Computer vision technology when used for an applied problem in fish farming is always "backed up" by additional technology and/or a database. It can be concluded that the teams of authors, by doing so, increased not the efficiency of the technology, but the level of its selectivity. In other words, the digital technology was double-checked in more proven ways. This, in its turn, suggests that there is a lack of trust in such methods for solving applied problems in fish farming. 2. The extent of computer vision application to fish farming is increasing and shows a positive growth dynamics, but it is still a very narrowly specialized thing. Therefore, in the future it will be necessary to develop methods of adaptation of already existing computer vision solutions to the conditions of fish farms of different regions.