Deep Learning Techniques for In-Crop Weed Identification: A Review

Weeds are a significant threat to the agricultural productivity and the environment. The increasing demand for sustainable agriculture has driven innovations in accurate weed control technologies aimed at reducing the reliance on herbicides. With the great success of deep learning in various vision tasks, many promising image-based weed detection algorithms have been developed. This paper reviews recent developments of deep learning techniques in the field of image-based weed detection. The review begins with an introduction to the fundamentals of deep learning related to weed detection. Next, recent progresses on deep weed detection are reviewed with the discussion of the research materials including public weed datasets. Finally, the challenges of developing practically deployable weed detection methods are summarized, together with the discussions of the opportunities for future research.We hope that this review will provide a timely survey of the field and attract more researchers to address this inter-disciplinary research problem.


Introduction
Due to their competition with crops for water, nutrients and sunlight, weeds are a significant threat to agricultural productivity [1,2]. Weed control in conservation cropping systems is reliant on the use of herbicides due to the lack of suitable alternative weed control options that do not interfere with the minimum tillage, residue retention principles, and thus benefits of these systems. To alleviate this threat, precise weed management becomes an increasingly important approach that is implemented through site-specific weed control (SSWC) techniques to identify and target weeds with treatments such as herbicides, lasers, electrical weeding and targeted tillage [3].
While in-crop SSWC creates important opportunities for improved production, the major challenge for these systems is the development of robust weed identification algorithms that perform accurately and consistently in complex and highly variable crop production environments [4]. The majority of previous methods have been based on conventional computer vision and machine learning techniques in both colour and multispectral imagery (e.g., [5]). These approaches often follow a pipeline in which hand-crafted image features play a primary role. As a result, developing such pipelines is labour intensive and images are required to be captured within well-defined conditions.  In summary, this review aims to: 1) investigate deep learning techniques related to weed control; 2) summarize current deep learning-based weed identification research including architectures, research materials and evaluation methods; 3) identify further challenges and improvements for future research with deep learning-based solutions.
The remainder of the review is organized as follows. In Section 2, deep learning architectures related to weed control are introduced. Section 3 provides a discussion of the existing deep learning based weed detection studies. In addition, public datasets and evaluation metrics for benchmarking are summarised.
Section 4 discusses the challenges for weed detection and potential deep learning solutions. Finally, Section 5 concludes this review.

Overview of Deep Learning Techniques
In this section, the theory of deep learning techniques is introduced including the deep learning building blocks and architectures relevant for weed detection.

Machine Learning
Machine learning (ML) algorithms are a class of algorithms that 'learn' to perform a specific task given sample data (i.e., training data). These algorithms are not explicitly programmed with rules or instructions to fulfil the task. In general, a set of samples for a ML task D = {(x i , y i )} can be obtained, where x i ∈ R p is the observed features describing the characteristics of the i-th sample and y i is the associated output. In general, for (x, y) ∈ D, it can be costly and time-consuming to obtain y, whilst x is convenient to collect. Therefore, it is expected to learn a model f Θ (x) that maps input values to the target variablê y as close as possible, where Θ is a set containing parameters of the model.
Optimization methods can be used to find the best set of model parameters, Θ, to minimize the difference between the predicted outputŷ = fΘ(x) and the ground truth y. In regard to the form of y, machine learning problems can be differentiated as classification problems if the value of y is categorical, or regression problems if the value of y is continuous.
In the past decades, various machine learning models have been proposed (e.g. support vector machines [6]). However, these methods require carefully devised hand-crafted features. Thanks to the recent growth in computational capacity and the availability of a large amount of training data, deep learning automatically integrates feature extraction and modelling together, and gains promising performance for many tasks. Fig. 1 provides an illustration of the difference between conventional machine learning vs. deep learning. In the following subsections, the details of deep learning are introduced.

Neural Networks
The model f Θ in machine learning can be chosen as a neural network (NN) [7], which contains an interconnected group of nodes (i.e., artificial neurons) inspired and simplified by the biological neural networks in animal brains.
The most well-known neural network architectures are the multi-layered perceptrons (MLPs), shown in Fig. 3 (a). This architecture organizes nodes into groups of layers and connects nodes between neighbouring layers. In detail, the computations of the k-th layer can be written as: where x (k) ∈ R p (k) is the input of the k-th layer which can be viewed as p (k) nodes of the neural network; W (k) ∈ R p (k) ×p (k−1) with the bias vector b (k) ∈ R p (k) represents a linear transform of the input signal which introduces full connectivity between the (k − 1)-th layer and the k-th layer; σ is an activation function which introduces a non-linearity to the output, allowing complex representations. In particular, x (0) is the input feature of a sample in D.
The layer defined in Eq. (1) can also be referred to as a fully connected (FC) layer. By stacking multiple FC layers, neural networks are able to formulate more complex representations of the input. To obtain predictions, computations are conducted from the first (input) layer to the last (output) layer, which is known as a forward propagation stage. To optimize the parameters Θ = {W (k) , b (k) } of a neural network, a backward propagation stage updates the parameters in an inverse order. Recently, more mechanisms and architectures have been proposed for constructing deeper neural networks, of which the ones related to weed identification are reviewed in the rest of this section.

Convolution Neural Networks
Inspired by the biological processes of animal visual cortex, convolution neural networks (CNNs) reduce the challenges of training deep models for visual tasks [8]. Convolution layers are the key components of CNNs, as illustrated in Fig. 3 (b). It involve partial connections compared with fully connected layers of MLPs, where each node focuses on a local region of the input. In detail, C (k) } to represent a series of C (k) convolution filters and the computations of the k-th convolution layer can be written as: where * represents a convolution operator and σ is an activation function; is the input feature map containing C (k−1) channels and the output feature map C (k) ) containing C (k) channels. A convolution layer can be viewed as a special case of FC layers with a sparse weight matrix.
Convolution layers often reduce the input spatial size but increase the number of channels. For some applications, recovery from a deep representation to the original input size is required. For this purpose, deconvolution (transpose convolution) operations were used. Readers can refer to [9] for more details.

Graph Neural Networks
Whereas most neural networks were designed for processing vectorized data, there are a wide range of applications involving non-vectorized data. Graph neural networks (GNNs) were devised for graph inputs. One commonly used GNN is graph convolution neural networks (GCNNs), which is a generalisation of the conventional CNNs by involving adjacency patterns [10]. In detail, a particular form to compute graph convolutions in k-th layer can be written as: where X (k) ∈ R n×p (k) represents the vertex features of n vertices in a graph, A ∈ R n×n is an adjacency matrix to illustrate the relations between vertices, W (k) ∈ R p (k−1) ×p (k) contains trainable weights and b (k) is a bias vector. Instead of using pre-defined adjacency matrix, graph attention network (GAT) estimated edge weights of the adjacency in line with the vertex features [11]. Recently, various methods were proposed to focus on some graph characteristics, which could not be captured by GCNNs (e.g. longest circle in [12]).

Deep Learning Architectures
Following the above discussed neural networks for deep learning, various deep learning architectures can be constructed in line with different target applications. In terms of weed detection tasks, four categories of deep neural network architectures are summarized, including image classification, object detection, semantic segmentation, and instance segmentation.

Image Classification
Image classification tasks focus on predicting the category (e.g. weed species) of the object in an input image. Input images can be viewed as spatially organized data, and many CNN based architectures have been proposed for classi-fying them into a specific class or category. AlexNet, which consists of 5 convolution layers and 3 fully connected layers was first adopted for large scale image classification [13]. Convolution layers (potentially with other mechanisms) were used to formulate deep representations from input images and FC layers were further used to generate output vectors in line with the categories involved.
VGG [14] further introduced convolution filters with a 3 × 3 perception field to substitute each convolution filter with a large perception field to learn a deeper representation and reduce the computational costs. InceptionNet was further proposed to introduce filters of multiple sizes at the same level to characterize the salient regions in an image, which can have extremely large variations in size [15]. With the growing depth of the CNN architectures, short-cuts between the layers alleviate the gradient vanishing issues [16], such as ResNet and DenseNet.
NASNet [17] was obtained by architecture search algorithms.

Object Detection
Object detection aims to identify the positions and the classes of the objects contained in an image. Generally, various CNN architectures for image classification can be used as backbones to learn deep representations and specific output layers can be introduced to obtain object-level annotations including positions and categories. For example, R-CNN and its improvements such as Faster R-CNN [18] were proposed with a two-stage scheme where the first stage generates regional proposals and the second stage predicts the positions and labels for those proposals. One-stage methods were also explored to perform object detection with less latency. For example, YOLO [19] (You Only Look Once) was proposed by treating object detection as a regression problem, of which the output is a feature map containing the positions and labels of each pre-defined grid cell. Single shot multi-box detector (SSD) [20] introduced feature maps of multiple scales and prior anchor boxes of different ratios. As class imbalance is one of the critical challenges for one-stage object detection, Reti-naNet with focal loss was proposed [21]. Note that pre-defined anchor boxes played an important role for most of the above mentioned methods. To avoid the significant computational costs to compute such anchor boxes, anchor-free methods were also investigated in [22].

Semantic Segmentation
Semantic segmentation focuses on the pixel-wise (dense) predictions of an image, by which the category of each pixel is identified. In general, semantic segmentation uses fully convolution networks (FCNs), which were first explored in [23]. These studies often involved an encoder-decoder scheme: the encoder formulates a latent representation of an image through convolutions and the decoder focuses on upsampling of the latent representation to the original image size for dense predictions. By increasing the capacity of the decoder, U-Net [24] was proposed with promising performance for medical images. SegNet [25] additionally involved pooling indices for its decoder, compared with the general encoder-decoder models to use the pooled values only, to perform non-linear upsampling to keep the boundary information. Pyramid scene parsing network (PSPNet) and different-region-based context aggregation by a pyramid pooling module [26] exploited the capability of global context as a superior framework for pixel-level predictions. Instead of following a conventional encoder-decoder scheme, DeepLab models adopted atrous convolutions to reduce the downsampling operations with a large reception field [27].

Instance Segmentation
Instance segmentation aims to output both the class and class instance information for individual pixels. Instance segmentation methods were initially devised in a two-stage manner by focusing on two separated tasks: object detection and semantic segmentation. For example, Mask R-CNN [28] was proposed using a top-down design, which first conducts object detection task to locate the bounding boxes of each instance and next within each bounding box undertakes semantic segmentation. Bottom-up methods were also investigated, which first conduct semantic segmentation and use clustering or metric learning to obtain different instances (e.g. [29]). Two-stage methods require accurate results from each stage and the computation cost of two-stage methods could be expensive. Therefore, single shot methods were explored. Anchor-based methods YOLACT [30] introduced two parallelized tasks to an existing one-stage object detection model including a dictionary of non-local prototype masks over the entire image and predicting a set of linear combination coefficients per instance.
An anchor-free method fully convolutional instance-aware semantic segmentation (FCIS) was proposed based on FCNs by introducing position-sensitive inside/outside score maps [31]. PolarMask [32] conducted instance center classification and dense distance regression in a polar coordinate, which is a much simpler and flexible framework. Very recently, BlendMask [33] inspired by FCIS introduced a blender module to effectively combine instance-level information and semantic information with low-level fine-granularity.

Deep Learning for Weed Identification
In this section, deep learning based weed identification studies are summarized covering four 4 approaches: image classification, object detection, semantic segmentation and instance segmentation. Before reviewing those approaches, research data, data augmentation and evaluation metrics used in these studies are reviewed first to provide a context in the field.

Weed Data
Weed data is the foundation for developing and benchmarking weed identification methods, and sensing technologies will determine what weed data can be acquired and what weed management practice can be developed [34]. While various sensing techniques like ultrasound , LiDAR (Light Detection And Ranging) and optoelectronic sensors were used for simple differentiation between weeds and crops, image-based weed identification has gained increasing interests due to the advances of various imaging techniques.
Multispectral imaging captures light energy within specific wavelength ranges or bands of the electromagnetic spectrum, which can capture information beyond visible wavelength [35]. For example, hyperspectral images consist of many contiguous and narrow bands; near infrared (NIR) imaging uses a subset of the infrared band, as the pigment in plant leaves, chlorophyll, strongly absorbs red and blue visible light and reflects near infrared. Driven by the low cost RGB cameras and the significant progresses in computer vision, RGB images have been increasingly used (e.g. [4]). In addition, some studies involved depth images (the distance between the image plane and each pixel) [36,37].
The availability of rich public datasets in the field plays a key role in facilitating the development of new algorithms specific to weed identification tasks.
In recent years, a number of in-crop weed datasets have been made public, as shown in Fig. 4, and will be reviewed in the rest of this section.  (https://github.com/zhangchuanyin/weed-datasets).
Whilst these datasets provide useful imagery and annotation data for benchmarking, there is a lack of consistency and details in metadata reporting standards and contextual information. An understanding of weed species, beyond a simple awareness of the difference to crops, is important in creating opportunities to deliver specific weed control treatments. For example, contextual understanding of crop growth stages, presence/absence of stubble will assist in developing algorithms capable of handling variability across different conditions.

Data Augmentation
Due to the laborious nature of developing annotated datasets within weed control contexts, existing datasets are often not large enough and do not reflect sufficient diversity in conditions. A significant risk for deep learning using small datasets is overfitting, where the model performs well on the training set but performs poorly when being deployed in the fields. To address this issue, various data augmentation strategies were adopted to enlarge the size and the quality of the training sets such as random cropping, rotation, flipping, color space transformation, noise injection, image mixing, random erasing and generative approaches. Readers can refer to [49] for more details.

Evaluation Metrics
As a binary classification model generally outputs continuous predictions, a threshold is required to judge the predicted labels: if the score is beyond the threshold, the corresponding sample is predicted as a positive case; otherwise the sample is predicted as a negative case. By varying the threshold, trade-offs among some metrics can be made. A receiver operating characteristic curve (ROC curve) illustrates the diagnostic ability of a binary classification model by plotting the sensitivity against the 1-specificity at various threshold settings.
A precision-recall curve (PR curve) shows plots the precision against the recall.
A large area under these curves (AUC) often indicates a model of high quality.
For multi-class classification, most of the these metrics can be computed class by class and the mean of these metrics can be used. In segmentation tasks, a sample can be viewed as a pixel. The metrics such as (mean) accuracy (mAcc), recall, precision and F-score discussed above can be derived in a similar manner. By organizing the pixels belonging to the same class as regions, the concepts such as mAP and mIoU can be derived as well.

Weed Identification Methods
Existing studies on weed identification can be organized into four categories in terms of the approaches they used: weed image classification, weed object detection, weed object segmentation or weed instance segmentation. Tables 1,    [53].
Recently, a few fine-grained architectures were explored to improve weed image classification performance. By introducing graph-based image representation, a graph weeds net achieved the state-of-the-art accuracy 98.1% on the DeepWeeds dataset, which formulated global and fine-grained weed characteristics with GCNs [54]. Another study also investigated the graph mechanisms [45], in which GCN-ResNet-101 was proposed and the accuracy values varied from 96.5% to 98.9% on 4 public RGB datasets.
Deep unsupervised learning was explored in a recent weed study [72], which Besides RGB images, multispectral imaging has also been investigated. Different sizes of multispectral images with a CNN involving 4 convolution layers were explored [35]. By varying the input size from 125 × 125 to 500 × 500, the accuracy varied from 86.3% to 94.7% on the UNSW Hyperspectral Weed Dataset. FCNN-SPLBP combined CNN and superpixel based local binary pattern feature extraction [50], which was evaluated on two public datasets: an accuracy 89.7% for 100 × 100 pixel images on the UNSW Hyperspectral Weed Dataset and an accuracy 96.4% on the sugar beet/weed dataset.

Weed Object Detection
Existing weed object detection methods are mainly based on generic object detection methods. In [60], DetectNet was used with an mIoU 64.0 and an F-score 60.3 on an in-house dataset containing 1,427 RGB images. Another study [57] with DetectNet achieved F-scores 99.8% and 100.0% for two different environments in detecting a weed species, namely Poa annua. In [59], Faster R-CNN achieved an mIoU 84.0% and an F-score 67.0% in detecting waterhemp (Amaranthus tuberculatus), Palmer amaranthus (Amaranthus palmeri ), common lambsquarters (Chenopodiam album), velvetleaf (Abutilon theophrasti ), and foxtail species such as yellow and green foxtails on an in-house dataset containing 450 augmented RGB images. YOLOv3 was used to detect broadleaves, sedges and grasses [61] and achieved an F-score 95.0%.

Weed Semantic Segmentation
An intuitive approach for weed semantic segmentation is based on a twostage scheme. Two CNNs were devised in [62], namely sNet and cNet, in which the first stage generated segmented objects and the second stage predicted the class of each object [62]. The method were applied to multispectral images to identify the regions of crops, weeds and soil with a mean accuracy 92.0% and an mAP 97.4%. Another study adopted a conventional algorithm using HSV-colour room vegetation index method for segmentation and a CNN for classification [65]. It achieved an F-score 98.6 and a mean accuracy 97.9% on an in-house RGB image dataset containing carrots, weeds and soil.
FCNs were investigated in pursuit of end-to-end solutions which treat the segmentation and classification within one neural network. In [68,69], the last FC layer of a VGG-16 was replaced as a deconvolutional layer. The modified VGG-16 was evaluated on two RGB datasets: one to segment maize, weeds and soil and an mIoU 84.0% and a mean accuracy 95.4% was achieved; on another to segment equipment, soil, stump, weeds, grass, radish and unknown categories, a mean accuracy 79.0% was achieved. A FCN with a DenseNet backbone [63] was evaluated to identify crops, weeds and soil in multispectral images and achieved F-Scores 86.6% and 92.4% on two datasets collected from sugar beet fields in two different locations. Two FCN-8s were trained to segment RGB images: the first one recognized grass, clover, weeds and soil, and the latter one recognized fine-grained clover species including white clover and red clover [41]. It achieved an mIoU 55.0 on its proposed GrassClover dataset.
In addition to using simple FCNs, recent studies tended to explore FCNs with additional mechanisms that are beneficial for segmentation tasks. In [38], a SegNet with VGG-16 backbone achieved a mean accuracy 64.1% on a carrotweed dataset containing RGB images of carrots, weeds and soil. In [44], a public multispectral dataset -Sugar beet/Weed -was proposed to identify crops, weeds and soil, and a SegNet with a VGG-16 backbone achieved an F-Score 80.0 and an AUC 78.0 by evaluating the crops and the weeds predictions within a binary pixel-wise classification scheme. In [70], a SegNet with a ResNet-50 backbone was adopted to identify canola and weeds in RGB images using a pre-processing step to remove backgrounds, which achieved an mIoU 82.9 and a mean accuracy 99.5%. A Bonnet framework [73] to segment sunflower, weed and soil in RGB images achieved an mIoU 0.8 [66]. A customized U-Net with different data augmentation strategies was investigated for the CWFID dataset [36], which achieved an F-score 83.4. A VGG-UNet was evaluated for the Sugar beet/Weed dataset and achieved a mean accuracy 95.0% [64]. DeepLab-v3 was evaluated on Sugarbeet2016 containing multispectral and depth data and an in-house RGB oilseed dataset, which achieved mIoU values of 87.1 and 88.9, respectively [37].
Lightweight models aiming for efficient segmentation were explored. In [67], light models were mixed together with the guidance of a large and accurate model Inception-V3. On the CWFID dataset, compared to the Inception-V3 with an accuracy 93.9% and 0.12 fps during the inference, 4 mixed lightweight models achieved an accuracy 90.3% with 1.83 fps. A customized CNN using a ResNet-10 backbone [39] with side outputs and short connections for multi-scale feature fusion achieved an F-score 98.0 and an mIoU 95.9 on the CWF-788.

Weed Instance Segmentation
There have been a few attempts at deploying instance segmentation algorithms for weed identification. A recent study adopted Mask R-CNN for field RGB images of two crop species and four weed species [74]. Further explorations for this approach are needed by considering the perspectives such as the improvements on different growth stage and small size plants.

Discussions
In this section, based on the current weed management studies and the recent developments of deep learning techniques, we discuss the challenges and opportunities for further advancing weed identification research from the following aspects: fine-grained learning, real-time inference, explainability, weaklysupervised / unsupervised learning, and incremental learning.

Fine-grained Learning
As reviewed in Section 3.4, most of the existing weed identification methods were based on general deep architectures ignoring the challenge caused by the strong similarities of crops and weed species. Recently, 3 major categories of fine-grained deep methods were explored to address this challenge.
Patch-based methods based on the fact that the fine-grained details often occur at a local-level. With the patterns collected from each region, fused or aggregated methods can be used to compute the final outputs. For example, regional CNN based features can be collected according to the key points of human poses for fine-grained action recognition [75].
High-order pooling based methods were introduced to address finegrained tasks as well, which did not require explicit patch proposals (e.g. [76]).
In particular, for a given convolutional feature map X ∈ R c×wh , the bilinear pooling can be computed by X X; the trilinear pooling can be computed by (X X)X. The pooled vector can be used as the input of the subsequent layer of the network. The relations between the high-order pooling methods and the patch-based methods can be explained from the perspective of the attention [77]. Both of these methods result in focusing on the critical regions to collect efficient deep representations for their associated tasks.
Regularization based methods are based on that the intra-class difference could be higher than the inter-class difference for fine-grained modelling, to introduce regularization terms for loss and drive the optimization to focus on learning fine-grained patterns. For example, in [78], pair-wise confusion and entropic confusion was introduced to construct its loss function.
Such fine-grained deep models provide a great opportunity to advance weed identification by taking the domain knowledge into account. For example, a weed can be decomposed into meaningful regions, such as leaves and stems.
In our recent work, a patch-based GNN [54] was proposed towards fine-grained weed classification, which achieved an accuracy 98.1% on the DeepWeeds dataset, compared with the accuracy 95.3% of DenseNet.

Real-time Inference
While most of the weed identification studies demonstrated promising performance using deep learning techniques, these deep networks often contain a huge number of parameters. It leads to three major issues in regard to efficiency, memory consumption, and power consumption for deployment. Intuitively, as indicated in [79], lightweight models (e.g. MobileNet [80]) can be devised by using mechanisms such as parameter pruning, low-rank factorization, transferred/compact convolutional filters. In particular, using a Google inference latency is around 180ms on an Nvidia Jeston TX2 [39].
In addition to devising lightweight architectures, methods such as quantization and knowledge distillation are devised for any existing models with less  [53]. It was slightly lower than a general DenseNet but obtained a very fast average inference latency 1.539ms. Knowledge distillation follows a similar way in which human beings learn, which contains one or more large pre-trained teacher models and a small student model. It aims to obtain an efficient student model which mimics and performs comparably to the the teacher models. A distillation loss penalizes the difference between the outputs from the teacher and the student models. A weed identification study followed this scheme to obtain a few lightweight models for semantic segmentation [67]. Mixing these lightweight models achieved an accuracy 90.0% and the inference latency between 934ms to 546ms by using an Nvidia GeForce Titan X graphics card.

Weakly-supervised & Unsupervised Learning
As manually collected supervision information for weed dataset can be re-

Explainable Learning
Deep learning shows a black-box nature, since it is difficult to understand and interpret the relations between the inputs and the outputs. However, explainability is of great importance for building trust between models and users to eventually facilitate the model adoption. As summarized in [84], there are three major approaches in pursuit of the explainability of deep learning: 1) visualization methods identify the most important parts of an input, which highly influence the results; 2) model distillation involves conventional machine learning models, most of which have clear statistical explanations and indications, to mimic the behavior of trained deep models; 3) intrinsic methods integrate mechanisms (e.g., the attention mechanisms).
Explainable learning has been seldom investigated for deep learning based weed identification, although it has the potentials to provide further insights.
Recently, graph weeds net was proposed with its graph mechanism, which treats the regions of an input image as graph vertices, to analyze the critical regions [54]. As it usually takes more efforts for object detection or segmentation than image classification, such explainable learning approach also provides an opportunity to take less effort to focus on the critical objects within an image. Furthermore, the critical regions are obtained without regional annotations, which can be viewed as a weakly-supervised learning requiring less human efforts.

Incremental Learning
Most of the existing weed identification methods assume that a trained network will only deal with fixed target species, which are available during the Recently, AgroAVNET explored the chance for incremental learning on the plant seedling dataset [52] and the accuracy achieved 91.35 for 12 species, compared to 93.64 from a general re-training. It followed a very straightforward way without fully exploiting the incremental learning, which froze the convolution layers trained by the original dataset and re-trained the FC layers only.

Large Scale Dataset
Large scale datasets are essential for developing high performance and robust deep learning models. For example, ImageNet [13], which contains 15 million labeled images belonging to roughly 22,000 categories, has played a significant role in advancing deep learning based vision tasks. However, as summarized in Section 3.1, most of the existing weed datasets contain images of a small number of classes. In addition, those images were collected under limited scenarios, such as one growth stage and one light condition. This has limited the development of advanced methods applicable to a large variety of fields and prevents the translation towards commercial adoption. Therefore, constructing large scale datasets with diverse and complex conditions in the context of practical deployment can be highly demanded.

Conclusion
In this paper, we reviewed the recent progresses in the field of deep learning based weed identification and discussed the challenges and opportunities for future research. After introducing the fundamentals of deep learning techniques, we provided a systematical review from three aspects: research data, evaluation, and weed identification methods. There have been more than 10 public datasets collected through different modalities and many weed identification methods have been reported across different research disciplines due to the inter-disciplinary nature of this topic. It is also noticed that most existing weed identification methods were proposed by using the architectures developed for generic deep learning problems. Finally, we discussed the challenges and opportunities in terms of 5 different learning techniques and large scale dataset.
Overall, deep learning based weed identification has gained increasing interest from different research communities and we feel that large scale datasets are strongly needed to bring this research direction to a new level.