The increasing demand for allergen-free and reduced-allergen foods has led to an investigation into the potential use of O-free wheat, a low-allergy wheat cultivar, in cookie production. This study focused on assessing the gluten composition of O-free flour and comparing its suitability for cookie making in comparison to flours with varying gluten strengths. Several analyses were conducted, including gluten composition, solvent retention capacity (SRC), thermal and pasting properties, dough-mixing characteristics, and cookie-making performance. The gluten composition of O-free flour by SDS-PAGE confirmed the absence of ω-gliadins and the reduced levels of low-molecular-weight glutenins and γ-gliadins. The SRC values of O-free flour fell between the flours with weak and medium gluten strengths. While thermal and pasting properties showed significant differences in sucrose solution but not across flour types, indicating similar starch structures, mixograms displayed distinct variations influenced by both sucrose solution and flour type, highlighting the importance of gluten quality and composition. Cookies made with O-free flour demonstrated similarities to those produced with weak gluten flour, known for their favorable cookie characteristics. This study emphasizes the significant influence of flour gluten composition on cookie-making performance and advocates for the adoption of O-free flour in the development of allergy-friendly cookies. Food allergies are hypersensitivity reactions caused by an abnormal immune response to specific foods . The preGlu-D1-encoded high-molecular-weight glutenin subunits resulted in poor gluten quality but improved cookie quality [Glu-D1 locus positively influenced cookie quality [In cookie production, sucrose, a primary ingredient, suppresses gluten development during mixing and delays starch gelatinization and pasting during baking, resulting in cookies with large diameters and small stack heights ,9,10,11. quality ; subunit quality . Sulfur-containing amino acid residues in proteins play a crucial role in the formation of disulfide bonds through free thiols during gluten development. Gliadins and glutenins, excluding ω-gliadin, contain these sulfur amino acid residues, which impact gluten development and strength, ultimately leading to reduced cookie spread ,19. HoweWhen comparing O-free wheat to its two parent lines, Keumkang and Olgeuru, a deficiency of ω-5-gliadin was identified through RNA sequencing analysis, revealing a simple deletion in the chromosome. This finding was further corroborated by the similarity analysis using the expressed sequence tags (ESTs) for gliadins and glutenins. The wheat also exhibited a reduced low-molecular-weight glutenin subunit . HoweverTherefore, this study aims to analyze the quality characteristics of low-allergy O-free wheat flour by comparing its gluten protein composition, SRC, pasting, thermal properties, and dough-mixing properties with commercial flours of varying gluten strengths. It also evaluates the quality of cookies made with O-free flour.The flour used in this study was milled from a low-allergy wheat cultivar, O-free, supplied by the National Institute of Crop Science . For a comparative study, three commercial flours with varied gluten strengths  were purchased from a local market: the flour with strong gluten strength , the flour with medium gluten strength , and the flour with weak gluten strength .The moisture and ash contents of the wheat flour samples were measured according to Method 44-15.02 and 08-01,01, respectively . For moiThe protein contents of the wheat flour samples were measured according to Method 46-30.01 . The flog for 15 min. Gliadin was then extracted from the remaining pellet by employing 5 mL of 70% ethanol for a duration of 12 h, followed by centrifugation under identical conditions. Glutenin, on the other hand, was extracted from the pellet after gliadin extraction, using a solution of 50% 1-propanol and 1% DTT for two hours, followed by centrifugation.Following the method outlined by Lookhart and Bean , albuminv/v) ratio. A running buffer consisting of 10× Tris/Glycine/SDS Buffer was diluted tenfold and employed. Each extract (10 μL) was loaded onto 12% TGX Stain-Free Precast Gels, and electrophoresis was conducted at 70 V for 10 min, followed by 100 V for 90 min. Upon completion of electrophoresis, the staining solution  was applied for 30 min, followed by the destaining solution (40% methanol and 10% acetic acid) for 1 h. The extraction and electrophoresis experiments were conducted in triplicate.For the electrophoresis of the gliadin and glutenin extracts, a 4× Laemmli sample buffer containing 0.4 M DTT was combined with the protein extract in a 2:1 (w/w) lactic acid, 5% (w/w) sodium carbonate, and 50% (w/w) sucrose solutions were prepared, and 25 g of each solution was added to each tube containing the flours. The tubes were then shaken for 20 min at 5 min intervals to disperse and hydrate the flour and centrifuged at 1000× g for 15 min . After centrifugation, the supernatant in each tube was discarded, and the pellet weight was measured to determine SRC percentages using the AACCI method [To evaluate flour quality, the SRC was determined according to Method 56-11.02 . Four 50I method .w/w) and flour were mixed in a 1:1 (w/w) ratio, and approximately 40 mg of the mixture was placed in an empty aluminum pan and sealed. A pan containing the sample was placed in a DSC instrument  and heated at a rate of 10 °C/min from 10 to 140 °C. An empty aluminum pan was used as a reference, and the Pyris Software  was used to calculate gelatinization temperatures and enthalpies.Differential scanning calorimeter was used to analyze the gelatinization properties of starch in the flour with water, according to the method described by Kweon et al. . Water ow/w), and 3.5 g of the flour was added and thoroughly mixed using a plastic paddle. The RVA was run according to standard 1 profile . Pasting parameters such as peak, breakdown, final, and setback viscosities; peak time; and peak temperature were calculated using RVA software (Thermocline for Windows ver. 2.5).The pasting properties of starch in the flours were investigated using a RVA  based on the method described by Kweon et al. . The RVAw/w) was added. The Mixograph machine was run for 10 min, and the peak height and time of the graph were measured. Each sample was tested in duplicate.A mixograph test was conducted using a 10 g Mixograph machine . Each flour sample (10 g) was placed in a mixograph bowl, and 6.0–7.0 g of water or 6.5 g of pre-dissolved sucrose solution , a* (redness), and b* (yellowness) values using a chromometer , and the average value was obtained from four measurements. Cookie firmness was determined using a Texture Analyzer . was measured using a Texture Analyzer. The measurement conditions were as follows: Mode, measure force in compression; pre-test speed, 1.0 mm/s; test speed, 0.5 mm/s; probe, 2 mm cylinder probe; penetration distance, 10 mm. The measurement was repeated three times, and the average cookie firmness was calculated from these three measurements.p < 0.05, using SPSS 22.0  program.All experiments, except for moisture, protein and ash content, were conducted in triplicate unless otherwise specified. All data were obtained through multiple measurements at least three times. The difference between the mean values of the samples was analyzed using analysis of variance with Tukey’s HSD test at a significance level of p < 0.05), suggesting the lowest extraction rate during milling [The physicochemical properties of the flour samples are shown in  milling .The protein content of the flour samples ranged from 8.50% to 13.98%. The flour with strong gluten strength (S) had the highest protein content, while the flour with weak gluten strength (W) had the lowest protein content. This is consistent with the relationship between flour protein and gluten strength, where flours milled from hard wheat (strong gluten) have higher protein content, and flours milled from soft wheat (weak gluten) have lower protein content . The proThe SDS-PAGE gel electrophoresis of glutenin and gliadin extracted from the flour samples was presented in Importantly, no clear bands in the molecular weight range of 50–75 kDa were observed in the gluten extract, but approximately three bands appeared in the gliadin extract of S, M, and W flours, except for O-free flour. These results indicated the absence of ω-gliadins in O-free flour, which is consistent with findings from ω-gliadin-free wheat reported by Waga and Skoczowski . The banAdditionally, the bands in O-free flour in the range of MW 37–50 kDa appeared significantly different from those of the other flours. Gliadin extract exhibited much thicker bands than glutenin extract, which could be attributed to differences in γ-gliadins and low-molecular-weight glutenin subunits (LMW-GS) . In partThese results clearly demonstrate variations in the amount of glutenin and gliadin in the flours. Overall, the SDS-PAGE gel electrophoresis results confirm the differences in the gluten protein composition of O-free flour compared to flours with varying protein contents and gluten strengths . Therefop < 0.05). This result was consistent with a study on the SRC of O-free flour conducted by Moon et al. [Water SRC values for the flours with strong, medium, and weak gluten strengths were 72.9%, 64.8%, and 51.4%, respectively, while the O-free flour had a water SRC value of 55.4%. The flour with strong gluten strength had the highest water SRC value, indicating a potentially high amount of damaged starch generated during the milling of hard wheat. Water SRC values are associated with water absorption , suggestLactic acid SRC is related to flour gluten strength . The SRCSodium carbonate SRC is associated with damaged starch in wheat flour . The SRCSucrose SRC is associated with the contribution of arabinoxylans in wheat flour . The SRCGaines  reportedw/w) sucrose solution are presented in p < 0.05). This result is consistent with previous studies on the effect of sucrose solution on starch gelatinization [The DSC results for the flour samples in both water and pre-dissolved 50%  sucrose solutions are presented in The RVA results for the flour samples in both water and pre-dissolved 50%  sucrose solutions are depicted in Mixograms of the flours in water and pre-dissolved 50%  results of the flour samples . DespiteThe cookie firmness followed the order: flour with weak gluten strength > O-free flour > flour with medium gluten strength > flour with strong gluten strength. Generally, cookies made from wheat flour with low protein content are known to exhibit high firmness and excellent crispness due to high moisture baked out . This suThis study explores the potential of using O-free wheat, a low-allergy wheat cultivar, in cookie production to meet the demand for allergen-free and reduced-allergen foods. The research highlights the unique gluten composition of O-free flour, showcasing its absence of ω-gliadins and reduced levels of low-molecular-weight glutenins and γ-gliadins. The flour’s solvent retention capacity (SRC) places it between weak and medium gluten flours, emphasizing its distinctive quality. While thermal and pasting properties show similarities, mixograms reveal variations, indicating the critical role of gluten quality and composition in cookie dough characteristics. Cookies made with O-free flour closely resemble those produced with weak gluten flour, known for their appealing cookie attributes. In conclusion, this study supports using O-free flour for allergy-friendly cookies, providing a safe and tasty option for consumers with dietary needs.


Azadirachta indica, belongs to the Meliaceae family, and its use in the treatment of medical disorders from ancient times to the present in the traditional medical practices of Asia, Africa and the Middle East is well-documented. Neem oil, extracted from the seeds of the fruit, is widely used, with promising medicinal benefits. Azadiradione, a principal antioxidant component of the seeds of A. indica, is known to reduce oxidative stress and has anti-inflammatory effects. To directly measure the antioxidant ability of neem oil, we used Rotating Ring Disk Electrode (RRDE) hydrodynamic voltammetry to quantify how it can scavenge superoxide radical anions. The results of these experiments show that neem oil is approximately 26 times stronger than other natural products, such as olive oil, propolis and black seed oil, which were previously measured using this method. Next, computational Density Functional Theory (DFT) methods were used to arrive at a mechanism for the scavenging of superoxide radical anions with azadiradione. Our work indicates that azadiradione is an effective antioxidant and, according to our DFT study, its scavenging of the superoxide radical anion occurs through a reaction mechanism in which azadiradione mimics the antioxidant action of superoxide dismutase (SOD). In this mechanism, analogous to the SOD enzymatic reaction, azadiradione is regenerated, along with the production of two products: hydrogen peroxide and molecular oxygen. This antioxidant process provides an explanation for azadiradione’s more general and protective biochemical effects.The neem tree,  Azadirachta indica, belongs to the Meliaceae family of plants, which are found in tropical and subtropical regions of the world. Its use in the treatment of medical disorders from ancient times to the present is well-documented. In fact, extremely ancient medicinal practices from the Indian subcontinent, such as Indian Siddha medicine, highlight the importance of the neem tree as a medicinal plant [The neem tree, al plant ,2. Evideal plant , and a nal plant ,5,6,7,8.al plant ,10,11.Citrus family (Rutaceae). An important review describing the chemistry and biological properties of the large number of Meliaceae limonoids was published in 2011 [The neem tree is a rich source of plant secondary metabolites, including many terpenes that have promising biological activities, as in the biosynthesis of important compounds such as steroids and sterols ,13,14. L in 2011 . Of inte in 2011  against  in 2011 ,17, whil in 2011 ,19,20. N in 2011 . Gedunin in 2011 . The ROS in 2011 . Gedunin in 2011 . In summA. indica (neem) oil significantly reduced oxidative stress and lipid peroxidation, leading to mice liver damage when exposed to the mycotoxin Ochratoxin A [2O2-induced lipid peroxidation and DNA damage were attributed to their ability to inhibit various free radicals [The impressive biomedical consequences of using neem tree parts are often accompanied by descriptions of antioxidant and anti-inflammatory activities ,26,27,28atoxin A . The antradicals . The composition/concentration of neem tree limonoids was shown to vary according to different factors including ripening . AnotherThis compound, azadiradione, also has important biological actions such as reducing oxidative stress and anti-inflammatory effects ,28,33,34Indeed, azadiradione has a triterpenoid structure similar to that of another biologically active antioxidant plant species previously studied by us, celastrol . Consequ2/N2 (35%/65%) for five minutes to establish the required dissolved molecular oxygen level. The rotation setting used for the rotation of the Au/Au disk electrode was fixed at 1000 rpm, and the potential sweep was applied to the disk from 0.2 V to −1.2 V and then reversed to 0.2 V, while the potential of the ring electrode was held invariable at 0.0 V. The disk voltage sweep rate was positioned at 25 mV/s. The molecular oxygen reduction peak (Reaction 1) was detected around −0.6 V at the disk electrode. Meanwhile, the oxidation (Reaction 2) occurred at the ring electrode. An initial blank solution consisting of bubbled O2, the electrolyte TBMB and DMSO alone (in the absence of neem oil), was run, and the ratio of the ring/disk current was defined as the “efficiency”. Next, a neem oil antioxidant aliquot was provided, as indicated in A commercial sample of 100% cold-pressed neem oil was used as received . For the experiment, the electrochemical cell containing a solution of 0.1 M dried tetrabutylammonium bromide, TBAB , dissolved in 50 mL dimethyl sulfoxide, DMSO, anhydrous, ≥99.9% , was bubbled with dry OReaction 1: Reduction of molecular oxygen occurring at the disk electrodeReverse Reaction 2: Oxidation of superoxide radicals at the ring electrode3, a program in the Biovia package . This program utilizes Density Functional Theory (DFT) to calculate energy, geometry, and frequencies, implemented in Materials Studio 7.0 [−6 electron/Å3. No solvent effects were included in these calculations. The convergence criteria applied during geometry optimization were 2.72 × 10−4 eV for energy and 0.054 eV/Å for force.Calculations were run using Dmoludio 7.0 . We empludio 7.0 . Correlaudio 7.0 . All ele2 can capture an electron from the working disk electrode to form the anionic superoxide radical, O2 + e− → O2•−, as in Reaction (1). From the RRDE graph in The Rotating Ring Disk Electrode (RRDE) method is an electroanalytic technique related to standard cyclic voltammetry. It is effective in rapidly detecting stable species during electrochemical reactions and has the advantages of a high sensitivity and low cost. The antioxidant capability of neem oil for the superoxide radical was studied using the RRDE method. Essentially, the superoxide radical is generated in a voltaic cell using anhydrous DMSO as a solvent, together with an electrolyte, and by bubbling a controlled amount of oxygen gas, allowing for solution saturation. The superoxide radical is obtained at a sufficiently negative potential so that the OThe clear-cut parameter defining antioxidant superoxide scavenging ability is the slope of the collection efficiency graph, which is shown in 26 times stronger, −2.271. Among these studied oils, extra virgin olive oil is well-known to have a low concentration of polyphenols, from 50 to 1000 mg/kg [The previously examined oils have slopes of approximately −0.08. Neem oil has a slope approximately 00 mg/kg , since i2 moiety, separated from the remaining azadiradione neutral radical by 1.624 Å, as shown in 2 was positioned 2.60 Å from an additional proton, making the whole radical system charged (+1), from two protons plus the reacted superoxide. Upon DFT optimization, H2O2 formed and detached from the organic moiety, 1.572 Å, showing the neighboring carbonyl (C-O bond of 1.254 Å), which was very similar to the carbonyl at the opposite end, as in The molecular structure of azadiradione was DFT-energy-minimized using starting coordinates from its crystal structure . After a2, as shown in 2 was rejected from the system at 3.791 Å, and H2O2, already formed in 2, H2O2 and reformed azadiradione, which was ready for the additional scavenging of superoxide. The related Reaction (3) is the same as that of superoxide dismutase and fully described in reaction for each step. For instance, the capture of the proton by the azadiradione O(carbonyl) has ΔGreaction of −552.3 kcal/mol , while the second superoxide is a reducing reagent, as it becomes oxidized on releasing its unpaired electron.We can conclude that by incorporating a proton, the azadiradione carbonyl, the C=O bond distance of which is 1.243 Å, lengthens to 1.365 Å, which is a typical single C-O bond length, and so a C-O-H moiety is established . Next, a 1.250 Å . Howevercentroid , longer Other tetracyclic terpenoids found in neem oil may also contribute to superoxide scavenging, among them azadirone, epoxyazadiradione, gedunin, nimbolide and zafaral , shown iThe antioxidant properties of neem oil compounds have been explored in the literature. For instance, nimbolide exhibited a concentration-dependent anti-radical scavenging activity and is a potent antioxidative agent . MoreoveThe results of our data, taken together, indicate that azadiradione, a component of neem fruit, is an effective antioxidant, and a mechanism for its scavenging of the superoxide radical anion has been proposed. 26 times stronger than other natural products, such as extra virgin olive oil [2●−) into molecular oxygen and hydrogen peroxide through metal ion redox chemistry, as described in Reaction (3) [A. indica extract, protocatechuic acid (−)− epicatechin and gallic acid, has been described [In this study, we have shown that azadiradione is a strong antioxidant, along with neem oil. Additionally, these electrochemistry experiments showed that neem oil is approximately live oil , propolilive oil  and blaclive oil , which wlive oil . SOD aretion (3) . In the escribed . directly by interacting with the superoxide radical anion to produce hydrogen peroxide and the oxygen molecule. Because SOD provides a defense against oxidative stress under physiological and disease conditions, the finding of natural products that can mimic SOD activity is of interest for their therapeutic potential. Thus far, most SOD mimics have been metal complexes [We also observed a different correlation: the neem oil component azadiradione can mimic SOD activity omplexes , and the2O2, while the second superoxide releases its unpaired electron to the antioxidant. Here, these SOD mimics carry out the dismutation of superoxide in this manner: overall, there are two superoxide anions reacting, the first capturing two H atoms and forming HThe results of this work add to our earlier studies, which demonstrated that some organic natural products such as isoflavones , galangiin situ. Our results confirm that the RRDE cyclic voltammetry experiment has several advantages over traditional methods for measuring antioxidant activity . Other methods used to generate the superoxide radical consist of measuring a product after superoxide consumption, in which adding an antioxidant will decrease the concentration of such a product, e.g., the superoxide concentration is measured indirectly. One of these methods is an enzymatic reaction using xanthine dehydrogenase , while a


To minimize dependency on the availability of data labels, some WiFi-CSI based-gesture recognition solutions utilize an unsupervised representation learning phase prior to fine-tuning downstream task classifiers. In this case, however, the overall performance of the solution is negatively affected by domain factors present in the WiFi-CSI data used by the pre-training models. To reduce this negative effect, we propose an integration of the adversarial domain classifier in the pre-training phase. We consider this as an effective step towards automatic domain discovery during pre-training. We also experiment with multi-class and label versions of domain classification to improve situations, in which integrating a multi-class and single label-based domain classifier during pre-training fails to reduce the negative impact domain factors have on overall solution performance. For our extensive random and leave-out domain factor cross-validation experiments, we utilise (i) an end-to-end and unsupervised representation learning baseline, (ii) integration of both single- and multi-label domain classification, and (iii) so-called domain-aware versions of the aformentioned unsupervised representation learning baseline in (i) with two different datasets, i.e., Widar3 and SignFi. We also consider an input sample type that generalizes, in terms of overall solution performance, to both aforementioned datasets. Experiment results with the Widar3 dataset indicate that multi-label domain classification reduces domain shift in position (1.2% mean metric improvement and 0.5% variance increase) and orientation (0.4% mean metric improvement and 1.0% variance decrease) in domain factor leave-out cross-validation experiments. The results also indicate that domain shift reduction, when considering single- or multi-label domain classification during pre-training, is negatively impacted when a large proportion of negative view combinations contain views that originate from different domains within a substantial amount of mini-batches considered during pre-training. This is caused by the view contrastive loss repelling the aforementioned negative view combinations, eventually causing more domain shift in the intermediate feature space of the overall solution. Gesture recognition is important for enabling a variety of Human Computer Interaction (HCI) applications such as intuitive clinical device interaction during surgery, vehicle infotainment systems, and automatic sign language translation ,2. RecenGesture recognition with WiFi-CSI data can be conducted using deep learning, most of which use supervised learning ,5,6, whiA problem with standard unsupervised pre-training approaches is that structural information learned from unlabeled data is influenced by domain factors present during data collection. WiFi-CSI domain factor examples include sensing environment, object/person characteristics, and position/orientation with respect to the sensing system . Domain The influence of domain factors can be mitigated in a downstream task classifier by introducing adversarial domain classification in a deep neural network used in the unsupervised pre-training phase”. (ii) “In situations where the downstream task classifier performance does not improve, or marginally improves, due to the multi-class and single label domain classifier used during pre-training failing to reduce the negative performance impact of domain factors, a multi-class multi-label domain classifier can be introduced instead. The multi-class multi-label domain classifier, during pre-training, does cause the eventual downstream task classifier performance to improve because the negative performance impact of domain factors is reduced. The multi-class multi-label domain classifier exploits the positive and negative view combination matrix created in the Normalized Temperature-scaled cross-entropy (NT-Xent) loss function during pre-training”. Our unsupervised pre-training approach is inspired by the contrastive unsupervised pre-training approach proposed by Lau et al.  [Integration of adversarial domain classification in the pre-training phase of a self-supervised contrastive deep learning-based approach.Introduction of a multi-class multi-label domain classification loss capable of improving adversarial domain classification performance in situations, in which standard multi-class and single label domain classification loss functions fail.Demonstrate that adversarial domain classification integration is negatively impacted by a large proportion of negative view combinations with views that originate from different domains within a substantial amount of positive view combination mini-batches considered during pre-training.Present clear future research directions for domain generalization in the area of unsupervised representation learning with WiFi-CSI data.In this paper, we address the influence of domain factors in the pre-training phase and propose to integrate a domain-independent feature extractor learned by means of an adversarial domain classification  in it. E et al.  . It is, n-shot (where n is number of samples per class) and m-way  classification tasks based on a representation distance comparison. This is helpful in situations in which the downstream dataset only contains a few samples per class [Unsupervised representation techniques for WiFi CSI data are either based on few-shot learning  or contrer class . In contn-shot gesture classification, Hu et al. [For u et al.  used Mobu et al.  convolutu et al.  used a SContrastive learning, during pre-training, tries to learn structural information in earlier feature extractor representations that are relevant for a variety of downstream tasks by comparing positive against negative input sample combinations. Combination creation can utilize task labels to group together input samples belonging to similar/different classes. However, this is not required since positive combinations can also be created based on an augmented and unaugmented version of an input sample or subsections of an input sample. Combinations of the first element of a positive combination and any other element are assumed to be negative. A downside to contrastive learning is that it always has to fine-tune a downstream task classifier using a low labeled data volume prior to the obtained structural information during pre-training being useful for a specific task. Fine-tuning a downstream classifier works well when the downstream dataset contains a medium to high number of samples  per class  . During Lau et al.  considerDomain-independent learning by means of an adversarial domain classification steers model training towards giving priority to feature commonness across domain factors . During Current adversarial domain classification ,13,14 teJiang et al.  proposedOur proposed pipeline integrating adversarial domain classification in the pre-training phase is illustrated in A denotes the transceiver link antenna and K sub-carrier. The reason for calling A the transceiver link antenna is, when indexing the transmitter and receiver dimensions, for the downlink or uplink sampled WiFi-CSI data, the data measured at the antenna quantifies the state of a specific transmitter/receiver link (we bastardize this in the term ‘transceiver’ in this case). During the experiments we considered two datasets: a Widar3 data subset and the SignFi dataset [Data pre-processing takes WiFi-CSI data in the  dataset ,6. Consi dataset ,6 and a  dataset . More in dataset ,6,9.A axis to 2 such that the first antenna matrix contains far greater amplitudes than the second antenna matrix [WiFi-CSI data in the Widar3 subset is initially phase-unwrapped one-dimensional-wise across the time and sub-carrier dimensions in subsequent order. When a subsequent phase value (measured in radians) has an absolute difference of at least a matrix . Amplitua matrix . Reductia matrix . Therefoa matrix  filter wn denotes the dataset index, WiFi-CSI data in the SignFi dataset is also initially phase-unwrapped one-dimensional-wise across the time and sub-carrier dimensions in subsequent order. We removed phase value STOs and FTOs by the offset term addition to be found in Equation . Symbol Equation . Symbolsfound in . By emplfound in . The filA. In the pre-training phase, when considering domain classification, a single combination is always made under similar domain factors. Let us assume the mini-batch size of 10. The mini-batch then has 10 positive view combinations, of which views belonging to a specific combination run through the network in a series .A of size 12 (6 tranceiver links · 2 antennas) in two dimension groups of size 6. During the fine-tuning phase, to prevent antenna transceiver link dimension mismatches, predictions resulting from the dimension groups made in a series were averaged to produce a final prediction. In the case of using the SignFi dataset as input dataset, a positive view combination denotes pre-processed input samples for two repetitions of the same class, while a negative view combination denotes samples belonging to either different or similar classes. The reason for this decision is that the antenna transceiver link dimension A in the SignFi dataset is 3 . In the fine-tuning phase, input data were provided as is without creating view combinations. The view combination creation step for the SignFi dataset, when mapped over training subset indices, assumes the available knowledge regarding the training subset indices, where samples denoting a similar class are stored for a specific training subset index. Views in a combination are fed separately in a series into the feature extractor.While using the Widar3 data subset as an input dataset, positive view combinations were made by splitting the antenna transceiver link dimension T denotes time dimension, A denotes transceiver link antenna dimension. The input modality denotes a single view. As explained previously, views are fed in a series into the feature extractor. The feature extractor illustrated in Feature extraction involves one input modality Feature extractor hyperparameter details can be found in When looking at Interesting to note is that the average pooling operators consider a dynamic pooling stride value in the time dimension that is dependent on the input time dimension size. Dynamic values can be considered because the average pooling operator has no learnable parameters. The dynamic size prevents Hyperparameters mentioned in P is the projection vector size. The projection network is a MultiLayer Perceptron (MLP) with 3 feed-forward layers, which use 450, 300 and 150 neurons, respectively. The first two layers include a ReLU activation function. The last layer linearly maps its input to the projection space (does not consider an activation function). All layers do not consider bias addition. Learnable parameter initialization is based on random samples taken from a variance scaled uniform distribution with a scale factor of 1/3 and `fan_in’ mode.The pre-training phase maps view combinations to a new embedding space, i.e., producing intermediate representation projections Our pipeline, when considering standard cross-entropy domain classification, also predicts domains under which view combinations were measured. This means that it returns the discrete probability distribution When considering multi-label domain classification, our pipeline predicts multiple domains, under which view combinations were measured. This means that it returns Bernoulli logits ,30 per cThe projection network and domain discriminator outputs are pre-trained with an overall adversarial loss presented in Equation . Subtrac ignored . Loss de ignored . For a c ignored . The rea ignored  is that  ignored . Symbol Standard domain discrimination loss rsion of  multi-cl(0 or 1) . Symbol n output , approxiEquation ) is thatWe manually defined both the projection and domain discrimination network parameters. For doing so, we were inspired by the projection network parameters presented in  and discAfter pre-training, the feature extractor network parameters were transferred to a supervised down-stream classifier. The classifier contains a gesture classification network instead of projection networks and a domain discriminator. When fine-tuning the classifier, the feature extractor network parameters were frozen and only the gesture classifier parameters were fine-tuned. Based on the type of view combination creation, a set of gesture classes was predicted either as is or the predictions per transceiver link antenna dimension group made in series were averaged to produce a final set of predicted gesture classes. The gesture classifier, in the fine-tuning phase, considers a multi-class cross-entropy loss (see Equation ). SymbolThe gesture classifier is a MLP with two feed-forward layers. The first layer contains 300 neurons and uses a ReLU activation function. The last layer uses a SoftMax activation function. The number of neurons is based on the number of classes in the data subset(s) considered per experiment  and lab data subset of 5520 samples (276 sign gestures·20 repetitions). During user leave-out cross validation, a downlink lab data subset for users 1–5 was considered. This data subset contains 7500 samples (150 sign gestures·5 users·10 repetitions).From the original Widar3 dataset, we constructed a subset including all repetitions of the following 6/22 gestures: push&pull, sweep, clap, slide, draw-O , and draw-zigzag . Regarding domain factors, we used data from environment 1 (classroom), all torso locations and face orientations, and user ids 10, 12, 13, 14, 15, and 16. Users vary in their age, weight, and height in ranges of 22–28 years, 45–87 kg, and 155–186 cm, respectively. The total size of this subset dataset is We compared the performance of our proposed pipeline, i.e., standard (ADV-S) and multi-label domain classification (ADV-M1) in the pre-training phase, with several other pipelines. Some pipelines consider the use of domain labels in a slightly different way compared to the adversarial domain classification during pre-training. When considering the multi-label domain classification in the pre-training phase, we also consider a loss (see Equation ) versionine and   is illusAn overview of the computation resources that we have used and average required wall-clock time per epoch per pipeline can be found in This baseline only considers the pre-processing, feature extractor, and gesture classifier blocks presented in This baseline considers similar blocks inside the pipeline presented in Equation . All othThis pipeline uses the same blocks as the pipeline presented in ENT loss . An overEquation . All othEquation . When coLooking at Equation , it can This pipeline considers similar blocks inside the pipeline presented in Equation . In ordeUsing the Widar3 dataset, views were first uncombined and shuffled. The reason for this is that when views originate from the same sample by transceiver link antenna dimension splitting, the class and domain for each view is always the same. Afterwards, we applied two grouping functions. The first grouping function groups views based on a similar task label. Therefore, views belonging to different domains may or may not end up inside a view combination. The second grouping function groups view combinations inside a mini-batch based on a key function For both the Widar3 and Signfi datasets, in every square 4-element Alternative versions of the DAN-F and DAN-B pipelines which are trained in a semi-supervised way are also considered. The pipelines trained in a semi-supervised way are compared against DAN-F and DAN-B to investigate the potential domain shift showing up in the output representations of the gesture classifier architecture layers during fine-tuning the DAN-F and DAN-B pipelines. This situation seems unlikely to occur when the feature extractor is already taught to prioritize extracting domain independent features during pre-training. However, we argue that any pipeline and training process  involving learnable neural network parameters that are not constrained during a training process via a domain shift mitigation technique should be put under test to rule out potential domain shift influences showing up during the inferencing stage. The pipelines are called ALTernating domain-aware Filter NTXENT (ALT-F) and ALTernating domain-aware Batch NTXENT (ALT-B). An architecture overview compatible to both pipelines is illustrated in During training iterations within a specific epoch, view combinations originating from a large pre-training dataset are given to the projection network pathway. Subsequently, samples originating from a much smaller dataset normally considered for fine-tuning are given to the gesture classifier pathway. Gradient updates for the respective pathways during each training iteration happen in sequence based on separate decoupled NTXENT loss Equation ). All otBy alternating the pre-training and fine-tuning processes at the same time, we argue that the potential performance drops on a held out test set, caused by an introduced domain shift during fine-tuning in the gesture classifier output representations, can be alleviated. Performance drop alleviation is caused by the domain mitigation constraint the pre-training pathway introduces when alternating pre-training and fine-tuning. In the case of no alternation and leaving the fine-tuning as a subsequent training process altogether, a domain shift may or may not show up due to no domain shift mitigation constraint being present.All pipelines mentioned in Dataset splitting prior to pre-training was based on leaving out specific domain factors. This means that if 5 factor types exist for a specific factor, 20% will be placed into the test subset and 80% in the training subset. The Widar3 training subset is further split into a 17% validation subset and 83% training subsubset. Splitting was based on randomly leaving out specific domains . For SignFi, validation/training subset splitting happens randomly based on stratified sampling with task labels and a 25/75% split due to the low number of domain factors being present in each dataset.Pre-training was performed by means of the Stochastic Gradient Descent (SGD) optimizer with extra regularization by means of weight decay and momentum. Additional regularization by means of early stopping after loss convergence or indications of overfitting was not considered due to overall noisy pre-training loss progression. The reason why the ADAptive with Momentum (ADAM) optimizer was not considered during the pre-training phase is that the loss function during experimentation constantly converged at a high loss value. A likely reason for this premature convergence is the inability of the learning rate scaling mechanism and standard ADAM hyperparameters that normally take care of reducing loss value oscillation around local minima to allow the loss value to surpass local minima boundaries. Epochs, batch size, learning rate, weight decay, Nesterov momentum, NT-Xent temperature, and random seed to introduce pseudo-randomness in random operations’ hyperparameters were the same for all domain factor leave-out cross-validation experiments, i.e., 100, 32, 0.0005, 0.000001, 0.95, 0.1, and 42, respectively (when applicable). The pipelines were trained for the complete number of epochs. Whenever a loss value obtained with the validation subset improved upon the last best predecessor, a new model parameter checkpoint was created. The best checkpoint at the end was loaded for fine-tuning and testing.Epochs, batch size, learning rate, NT-Xent temperature, and random seed hyperparameter values used during pre-training were determined based on manual hyperparameter tuning. Weight decay and Nesterov momentum values used during pre-training were determined based on automatic hyperparameter tuning with the Bayesian optimization tuning algorithm . We consThe fine-tune training and validation subsets were sampled from the pre-train training and validation subsets directly. This means no training and validation subset data mixing can occur when switching to the fine-tune phase. Data mixing can occur when you sample the fine-tune training and validation subsets directly from the original data(sub)sets. The pre-train training and validation subsets are randomly divided based on stratified sampling with task labels. The pre-train and validation division ratios are 75%/25% and 50%/50%, respectively. Subsequently, the 25% division from the pre-taining training subset and second 50% division from the pre-train validation subset are used as fine-tune training and fine-tune validation subsets.Fine-tuning happened with the ADAM optimizer. The learning rate and batch size throughout all fine-tuning phases were set to 0.0001 and 16, respectively. The learning rate and batch size values were determined based on manual hyperparameter value tuning. Epoch number and random seed were the same as those during the pre-training phase. Weight transferring happened by copying weights to the fine-tuning model and setting all layers inside the feature extractors see  to the iAll training subsets, during pre-training and fine-tuning in all domain factor leave-out cross-validation experiments, were augmented on-the-fly after sampling. Augmentation was applied to partly alleviate sub-optimal performance metric test set results caused by the small size of the considered input datasets (Widar3 subset dataset and SignFi dataset). We considered a limited number of augmentations such as no augmentation, Gaussian noise overlay, time permutation , and random sample mixing with the considered input sample. We did not perform multiple augmentations in sequence. Every augmentation has a 25% chance to be picked (augmentation method sampling is based on picking a number between 1 and 4 from a uniform distribution with seed 42). The augmentations were directly taken from research studies involving Doppler Frequency Spectogram (DFS) and three-dimensional acceleration signal augmentation conducted by Zhang et al. and Um et al. ,39. AddiAfter fine-tuning, classification performance metric results were obtained with the specifically sampled test data subset. Performance metrics considered include accuracy (A), precision (P), recall (R), F1 score (F), and Kappa score (CK).The STD pipeline is trained end-to-end in a supervised way. It therefore does not consider separate pre-train and fine-tune phases. The STD pipeline uses similar data sampling procedures as the earlier discussed pre-training phase to obtain training/validation/test subsets. This is possible due to the Independent and Identically Distributed (IID) nature of the considered datasets with regards to both task and domain labels. The considered optimizer and hyperparameters, including hyperparameter values, were the same as for the earlier discussed fine-tuning phase . End-to-end training in a supervised way does not consider the view combination creation during pre-processing.The pre-train loss uses a domain loss weight (see Equation ). The coThe ALT-F and ALT-B pipelines, during training iterations within a specific epoch, are first given mini-batches (containing view combinations) originating from pre-training training and validation data subsets. The mini-batches are used in the projection network pathway. Subsequently, mini-batches originating from fine-tuning training and validation data subsets are given to the pipelines. The mini-batches are used in the gesture classifier pathway. Gradient updates for the respective pathways during each training iteration happen in sequence.Both pre-training and fine-tuning training and validation subsets were created prior to the training phase. Sampling settings were similar to the pre-training and fine-tuning subset sampling settings applicable to all pipelines explained earlier. The fine-tune training and validation subsets do not consider view combination creation during pre-processing. Mini-batches sampled from the fine-tune training and validation subsets consider a batch size of 32 similar to the batch size value considered in pre-training training and validation subset mini-batches. Training iteration steps 1 and 2 see  use diffIn this section, we present and discuss our experimental results.The model complexity and data volume requirement advantage of the STD pipeline, when leaving any of the available domain factors out, is influenced by a domain shift. As expected, when integrating domain classification in an unsupervised pre-training setting, and not leaving out a specific domain factor, no noteworthy performance differences are noted see a. IntereAdditionally, we present results of a discrepancy test between the A values measured on (i) the validation subset during fine-tuning/end-to-end training across time and (ii) on the test subset once the best performing pipeline train checkpoint was restored. The validation subset, as explained in Typical approaches to prevent reducing the volume by half include (i) introducing an augmented version of an input sample within a positive view combination , (ii) prLooking at the user factor leave-out cross-validation results see a, a subsLooking at the environment factor leave-out cross-validation results see b, abysmaWhen looking at the Widar3 random and domain factor leave-one-out cross-validation results presented in Overall, the cross-validation results obtained with the SignFi datasets and stacked CSI amplitude and phase values as input types unfortunately show a low capability of mitigating the domain shift problem. Therefore, we do not consider the SignFi dataset further in our analysis.We suggested the existence of a conflicting relationship between (i) confusing the feature extractor with a domain classifier by maximizing the domain classification loss and (ii) the decoupled NTXENT loss forcing distance between domain arbitrarily sampled negative view combinations. At first glance, m’s summation is minimized, while n’s summation is maximized. When many n’s are filtered, the loss value arbitrarily increases for a specific mini-batch. This teaches the pipeline that the current direction step in the loss space taken is bad irrespective of whether the step leads to better achieved performance, as indicated by a yes or no. This directly influences the potential positive benefit of reducing the distance between views sampled from a different domain. The eventual performance convergence can be explained by the fact that many n’s are not always filtered in every training iteration. As a result, fine-tuning can still use suboptimal structural information in the feature extractor. An inspection into the DAN-F pre-train NTXENT loss for the first orientation leave-out split has confirmed this view . Therefore, we recommend finding methods to reduce the number of n similarities being made ineligible for the eventual loss computation.However, something else is hampering the overall performance effectiveness of the DAN-F pipeline, as illustrated by the cross-validation performance result drop and slow convergence speed. We believe the reduction in overall performance is caused by the reduction in the number of negative view combinations eligible for loss computation further down the pipeline. Let us assume that the overall NTXENT loss is made up of n similarities being made ineligible at the cost of a potential increased model bias towards constructing mini-batches based on positive view combinations, in which views were sampled from the same combination of different domains. The results in The DAN-B pipeline was assumed to have the ability to reduce the number of The leave-out cross-validation results for the comparison between the DAN-F and ALT-F pipelines can be found in Looking at the cross-validation results, an overall performance increase of approximately 10% from the DAN-F to the ALT-F pipeline can be observed. A potential underlying reason may be, even though the current direction step of the decoupled NTXENT loss is considered arbitrarily bad, the resulting effect of it is subsequently reduced by training the classification pathway in training iteration Lastly, we consider a comparison of the aforementioned pipelines with a state-of-the-art solution, i.e., WiGRUNT . We seleThe WiGRUNT pipeline integrates two attention sub-networks into the neural network portion of the pipeline to reduce a domain shift. Attention refers to putting more cognitive attention towards parts of an input sample that matter for the learning task at hand. The first sub-network learns a function to produce an input mask based on the input sample and subsequently combines it with the input sample to prioritize input sample sections. The second sub-network learns a function to produce a mask based on the latent feature space. When combined with features resulting from the aforementioned space, it prioritizes feature relationships arbitrarily across time.In Hypothesis 1.The influence of domain factors can be mitigated in a downstream task classifier by introducing adversarial domain classification in a deep neural network used in the unsupervised pre-training phase.This hypothesis was validated. When the NTXENT loss, and domain classification loss, form a harmonious relationship, the domain shift can definitely be mitigated. This is illustrated by experiment results with the Widar3 dataset, indicating that the ADV-M1 pipeline reduces a domain shift in position (1.2% mean metric improvement and 0.5% variance increase) and orientation (0.4% mean metric improvement and 1.0% variance decrease) in domain factor leave-out cross-validation experiments when being compared against the STD-P pipeline.Hypothesis 2.In situations where the downstream task classifier performance does not improve, or marginally improves, due to the multi-class and single label domain classifier used during pre-training failing to reduce the negative performance impact of domain factors, a multi-class multi-label domain classifier can be introduced instead. The multi-class multi-label domain classifier, during pre-training, does cause the eventual downstream task classifier performance to improve because the negative performance impact of domain factors is reduced. The multi-class multi-label domain classifier exploits the positive and negative view combination matrix created in the NT-Xent loss function during pre-training.This hypothesis was also validated. Where the ADV-S and ADV-M2 pipelines do not show noteworthy performance improvements over the STD-P pipeline during domain factor leave-out cross-validation experiments, the ADV-M1 pipeline does (see results mentioned in prior paragraph).In this paper, we proposed the integration of an adversarial domain classifier in the pre-training phase. The unsupervised representation learning pipeline that was considered during the experimentation uses view combination distance comparison during pre-training. Based on this pipeline, a few hypotheses were drawn in In addition to hypothesis validation, a few noteworthy observations were made during pipeline experimentation. The considered input sample type generalizes, in terms of the overall solution performance, to both considered Widar3 and SignFi benchmark datasets. It should be noted that all other observations were noted based on the Widar3 subset. Observations, for the purpose of generalization, should be re-evaluated with (i) datasets focusing on domain shift reduction which becomes available in the future and (ii) view creation procedures which do not negatively impact pre-training data volume together with the SignFi datasets. When the NTXENT loss, and domain classification loss, do not form a harmonious relationship, the domain shift reduction capability of the domain classification pipelines  is significantly reduced. More discord is created between the NTXENT loss and domain classification loss when a majority of negative view combinations used in the NTXENT loss contain views originating from different domains. In A way to overcome a majority of negative view combinations used in the NTXENT loss containing views originating from different domains is filtering negative view combinations of which views originate from different domains. Filtering can be ingrained into a special domain aware version of the NTXENT loss. However, the domain-aware contrastive loss based on filtering negative view combinations suffers from arbitrary negative loss direction indications. Structuring mini-batches such that positive views in every combination originate from different domains was suggested as a way to overcome arbitrary loss direction indications. However, due to technical limitations, the DAN-B pipeline, which uses the aforementioned mini-batch structure technique, could not be properly tested.DAN-F and DAN-B pipeline variations, called ALT-F and ALT-B, which alternate contrastive learning with task learning in a single training iteration, were also introduced. The DAN-F and DAN-B pipelines were compared against these variations to test whether the variations mitigate a domain shift which shows up in the later classification sub-network feature representation spaces. The experiments showed that when a classification pathway is not explicitly constrained via a domain shift mitigation technique, while being alternated with a constrastive pathway which takes into account domain awareness by means of negative view combination filtering, the domain shift is reduced at arbitrary epochs.For the future research, the highest priority should be put on envisioning domain-aware unsupervised contrastive learning pipelines, which limit view combination distance maximization between views originating from different domains. Examples when considering vision data include negative combination filtering and domain augmentation , and com


Automation in agriculture can save labor and raise productivity. Our research aims to have robots prune sweet pepper plants automatically in smart farms. In previous research, we studied detecting plant parts by a semantic segmentation neural network. Additionally, in this research, we detect the pruning points of leaves in 3D space by using 3D point clouds. Robot arms can move to these positions and cut the leaves. We proposed a method to create 3D point clouds of sweet peppers by applying semantic segmentation neural networks, the ICP algorithm, and ORB-SLAM3, a visual SLAM application with a LiDAR camera. This 3D point cloud consists of plant parts that have been recognized by the neural network. We also present a method to detect the leaf pruning points in 2D images and 3D space by using 3D point clouds. Furthermore, the PCL library was used to visualize the 3D point clouds and the pruning points. Many experiments are conducted to show the method’s stability and correctness. Sweet pepper is one of the most popular fruits for people today because of its nutrition, such as vitamins A, B, and C, and the minerals Ca, P, K, and Fe . The amoPlant pruning is an easy task for humans, but it takes time. In contrast, it is a difficult task for machines. It must perform tasks such as recognizing the plant parts through camera sensors, detecting removable objects and the position of pruning points, and then controlling the robot arm to remove them. However, machines can work day and night continuously. It could save lots of time. Furthermore, it can reduce the risk of disease infection from laborers who usually get on the farm from outside. Automation plays an essential part in intelligent agriculture. This prompted us to study this subject and apply it to sweet pepper, one of the popular plants in smart farms. Leaf pruning was selected as this research object.As with the jobs listed above, our research includes many phases. The first phase is recognizing the plant parts, which was solved by our previous study of the semantic segmentation neural network . This neA sweet pepper dataset was created with more than 4000 images. We labeled plant parts such as stems, leaves, and petioles. An automated pruning system is complex because it includes a mechanical component for robot design and control and a software component for the robot’s visual interface. It decides where to cut. Our research focuses on this part. He et al.  reviewedDuring the development of computer vision, there were many improvements in 3D reconstruction. Many algorithms are proposed to archive this information, such as SIFT , SURF 1, FAST 11, and OREarly on, it often used 2D images as input. Many proposed systems frequently mention the structure from motion (SFM) technique . It inclAt the same time, many visual SLAM systems that perform simultaneous localization and mapping by the camera-based sensor were introduced . These sOur research aims to detect the pruning point of the lowest petioles of the sweet pepper plants. It means that after creating a 3D point cloud of a sweet pepper plant, the lowest petiole and the stem of that plant should be detected in the 3D point cloud. Then the pruning point should be identified on that petiole. However, recognizing objects and calculating their positions in a 3D point cloud take much more time than that in a 2D image. Hence, we proposed a method to create a point cloud that includes all known parts, such as stems, leaves, petioles, and potential pruning points detected in 2D images. These parts are encoded with different colors. This point cloud is called a semantic point cloud.Petioles are soft and small; it is not necessary to identify the diameters, but it requires accurate pruning positions. In many studies relating to plant pruning, lidar cameras or laser scanners were used to detect branches ,28. TherIn summary, our contribution is to propose a system that detects the pruning point of the lowest sweet pepper leaf in 3D space.As mentioned in the introduction, sweet pepper petioles and other parts were detected in 2D images before building the plant’s 3D semantic point cloud. This task cannot be performed by the neural network for image classification or object detection in a bounding box. We need to know which pixels belong to which objects. Therefore, the semantic segmentation neural network was chosen.Our previous research focused on recognizing tomato plant parts by a proposed neural network . This neThere are similarities between sweet pepper and tomato plants. We used the same neural network structure and made a new dataset to train the model. RGB-D images are taken at many green farms, from seedlings to mature trees, to make our dataset more general and the model more accurate. This neural network structure is different from the other neural networks. It has two stages, so it leads to two-phase training. The model was trained with more than 4000 images and 50 epochs for each phase. The neural network was written in Python 3.9 and used PyTorch 1.8. It was trained and tested on a computer with a GPU from Nvidia, the GeForce RTX 3090, and CUDA version 11.2. The best IOU model reached was 74.86 percent. Three-dimensional reconstruction is an essential step in finding the relative cutting position for the robot, which is represented by the camera. Many systems can perform this task, but they take a lot of time. It requires an application that can run in real-time and take little computer resources because other modules, such as the neural network, must use many resources. Therefore, the ORB-SLAM series is suitable for our system. The ORB-SLAM family is easy to install, only uses a CPU, and can work well with ROS (Robot Operating System) . AdditioThere are many updates from the first version of ORB-SLAM  to the nThe output of ORB-SLAM3 is the camera’s position at the moment it gets image input. This position can be updated after detecting a loop. ORB-SLAM3 does not create a dense map. Therefore, if sweet pepper exists in the input images, a 3D point cloud should be created by another module based on the frame poses. This process happens when the robot moves along the field of sweet pepper plants.Most visual SLAM faces cumulative drift problems, which can be solved by loop detection. However, the camera will only go one way in our system. It cannot make a loop. The cutting position must be found online during its movement. The frame pose should be refined when creating a sweet pepper 3D point cloud. Therefore, the ICP algorithm is used to stick the point cloud built from each frame together and eliminate the drift problem.The algorithm’s main idea is to minimize the mean-square distance metric over six degrees of freedom by testing a set of rotations and translations. This algorithm is often used to align a pair of point clouds. Especially when these point clouds come from two close frames, they share many of the same objects. In this case, the ICP algorithm becomes very efficient and performs its task quickly. However, the speed also depends on the number of points. Our frame size is 640 × 480, so the number of points could be numerous. Therefore, the ICP algorithm is applied to semantic point clouds consisting of plant part points. This saves much time.Sweet pepper petioles are soft, and they have small diameters. They should be cut at a position near the stem, such as in We proposed a method to detect the pruning points in 2D images in four steps after they have resulted from the semantic segmentation neural network. This result is a semantic image including sweet pepper plant parts. The images are captured with a resolution of 640 × 480, so a semantic image is a matrix (640 × 480), and each pixel is a matrix item having value x is the stem layer or the petiole layer. The intersection of the two layers is found by adding these two enlarged layers and selecting items having values of 2, as in Equation (4):The predicted results always have many uncertainties. The OpenCV-connected component function is used to remove noise. Next, the petioles and stems are enlarged by a 2D convolution operation with a kernel matrix (7 × 7) with all item values of 1. Therefore, each item of the enlarged layer These intersection regions prove that the petioles connect with the stems and are the petioles’ root points. The pruning points should be near the intersection regions and belong to the petioles. Therefore, in the next step, the intersection layer is enlarged, and its overlapping regions with the original petiole layer are detected. Here, the second intersection layer is created. The points of these intersection regions meet the requirement of pruning points and are called pruning regions. The center points of these regions are chosen as the pruning points. The way to enlarge the layer and find the intersection layer is similar to Equations (3) and (4). Then, these pruning regions are integrated into the original semantic images to create a 3D semantic point cloud.x and center y, the coordinate  value can be calculated in 3D space as follows:u and v are the column and row values of a pixel in 2D images, and depth  returns the distance value from the camera to the pixel saved in the depth image.Creating a 3D point cloud is the same as putting each pixel or group of pixels in RGB images as a point in a 3D coordinate system. With the intrinsic parameters of a depth camera, such as the scaling factor, focal length, and center In RGB-D images, each RGB image has a corresponding depth image. When the RGB image is converted to a semantic image, this semantic image shares the same depth image. Therefore, a 3D semantic point cloud can be created from this semantic image and its corresponding depth image.Each point in a 3D semantic point cloud has a color. This color has its own meaning. In our context, blue points represent pruning regions, and the pruning points in 3D space must be in these areas as well. However, there are many blue regions in the semantic point cloud. The lowest pruning region and its position should be detected. Therefore, a four-step method to find the lowest pruning points in 3D space is proposed as in Firstly, the pruning point cloud is extracted by selecting only blue points from the 3D semantic point cloud. A 3D point cloud is created by many RGB-D images taken from different viewpoints. So that a pruning region in a 3D point cloud can be created from diverse groups of points from different viewpoints, and these groups can be unconnected to each other. Therefore, in step 2, the closet points are inserted into each group to connect them. Then, each group of points is a pruning region of a petiole. Next, each group is separated, and small groups seen as noises are removed. The y values in the 3D coordinate system between the remaining groups are compared. The group with the smallest y value is the lowest pruning region, belonging to the lowest petiole. Finally, the center point of this group is the final pruning point. The red point in the white circle in the right image of The PCL library can store points in a Kd-tree  data structure. It provides a function to find points having a distance of less than a value. A method is proposed to extract each group of pruning region points by applying this function, as in The system consists of five main modules: create semantic images by semantic segmentation neural networks and detect the pruning regions, find camera poses by ORB-SLAM3, create semantic point clouds, register different point clouds by ICP, and detect pruning points in the 3D semantic point clouds. After receiving a start signal, RGB-D images were passed through the semantic segmentation neural network to recognize the plant parts. Then, these results go through the pruning region detection module. On the other hand, RGB-D images were passed through ORB-SLAM3 to detect the camera poses. These are parallel threads. Semantic point clouds are created at the end of both threads based on the previous results. This process will be repeated until the system gets the stop signal. The ICP algorithm is applied at each loop to align the last semantic point cloud with the new one. When receiving a stop signal, the creation of a 3D semantic point cloud process stops, and the final semantic point cloud is passed through the 3D pruning point detection module to get the final pruning point position.Our experiments focus on accurate visual results and stable values of pruning point positions. The system is reliable if the pruning points in the point clouds created by different camera trajectories are the same and the difference in pruning position values is small. The time of execution is also estimated to verify if the system can be applied in practice.The experiments were set up in the lab with sweet pepper plants raised in flower pots. There is some breeze making the plant tremble slightly. The light does not shine directly on the camera. We used one camera, the Intel RealSense L515, to capture RGB-D images. The depth and RGB images are aligned before processing by our application. The camera was placed in three different positions for a sweet pepper plant, as in The application was written as ROS nodes under Ubuntu 20.4. There are three nodes. One node wraps ORB-SLAM 3 to provide camera poses. One node was written in Python to predict plant parts by using a semantic segmentation neural network and detect pruning regions from RGB-D images. The final node was written in C++, creating a 3D semantic point cloud and finding a pruning point in 3D space. The system was installed in a laptop having a CPU of i7–7500U, and a GPU of Nvidia GeForce 940MX.The average time for creating and aligning a semantic point cloud is 0.725 s. The total time to find a pruning point depends on the length of the camera path before stopping.We got the 3D semantic point clouds for three different camera view points and four different camera movement directions. Our experiments worked on natural sweet pepper plants, and the camera was moved by hand. Estimating the ground-truth value to evaluate the experiment results took much work and was expensive. However, in every hardware system, there are always differences. If it is stable, it can be refined after experiments.Twelve experiments were conducted on a plant to check the system’s accuracy. Although the camera’s moving path and starting and stopping positions were different, they returned the same results visually. This proved that the semantic point clouds were created correctly. The correct visual results do not mean the right distance or position of the pruning points to the camera. Therefore, the experiments are divided into three groups. Each group has the same camera stopping position. The pruning point positions are detected and compared to each other in a group. If the differences in each group were significant, the distance from the camera to the pruning point was detected incorrectly. However, the experiments showed that this value was small in each group. It proved that the pruning point positions detected by the system are correct and reliable.The experiment environment was set up with lots of light and breeze to simulate a natural environment. When the robot moves, it could make plants tremble, and the depth information in RGB-D images could change for the same point. It is a challenge, but our system can handle this with acceptable results. The first row of After archiving with the required accuracy, we focus on time consumption. Most of the time is taken by the creating semantic point cloud process. Creating a semantic point cloud of pair RGB-D images in these experiments takes around 0.7 s. It is acceptable because the robot cannot move fast, and images should be taken at a reasonable distance to get more different viewpoints. In addition, if the camera is moved quickly, it could lead to errors when applying the ICP method because the overlap between a pair of point clouds is small. More experiments will be done to detect the maximum moving speed of the camera when the program is applied to a complete robotic system.There are little differences in time, resources, and accuracy when the images of the plants are taken in the field versus those of the potted plant in the laboratory. The plants in the fields are bigger, but we are interested only in the lower part of the plant for pruning. Hence, if we capture images of the lower part of the plant, the time, resources, and accuracy are similar. We took the RGB-D images at many green farms from seedling to mature age to make our dataset more general and the model more accurate.Similar to many other SLAM systems, drift is a big problem. If the camera moves along a curving path, it could lead to errors in creating a semantic point cloud. The camera moving in a straight line is a good way to decrease this problem. In addition, our point cloud is made from many viewpoints. One petiole may not be seen in one depth image, but it can be seen in other depth images. It decreased the occlusion problem and the impact of the light-to-depth images. We are working on a prototype of the system. This research took part, which is taking images and returning the cutting position. We will do more research on controlling robots before experimenting on green farms.We proposed a system to detect sweet pepper leaf pruning points based on a 3D point cloud and a semantic segmentation neural network. This system can run in a real-time and realistic environment. Our experiments proved that the proposed system is stable and reliable, with correct visual results and a distance of pruning points from the camera.In addition, in an attempt to achieve a complete sweet pepper pruning system, this research addressed the important task of providing cutting positions. The proposed system can be applied to the robot’s arm or manipulator with a scissors end effector. In the future, we will continue to study an entire robotic system. This research can be extended to perform sweet pepper shot pruning, width, and height, or leaf area detection based on 3D semantic point clouds, and it can be a good reference for other plant pruning systems. Furthermore, a 3D semantic point cloud can be used for object detection in 3D space. It would use fewer resources than training a neural network to detect an object in the original 3D point cloud.


Animal Health and Welfare Northern Ireland has been enrolling dairy herds across Northern Ireland (NI) in a voluntary Johne's disease (JD) control programme since October 2020. A Veterinary Risk Assessment and Management Plan (VRAMP) questionnaire was completed for each herd enrolled and recommendations for improved farm management practices were provided to farmers. Herd JD testing was recommended but was not mandatory.This study analysed VRAMP responses for 1569 dairy herds that had enrolled in the JD control programme up to October 2022. Univariate and multivariate regression models were applied to the data as appropriate.p > 0.001) more likely to be a high‐probability JD herd. Accidental mixing of neighbouring herds significantly (p = 0.01) increased the risk of a suspected or confirmed case of JD. Herds that utilise rented land (70%) were significantly (p > 0.001) more likely to be at a high risk for JD.Overall, 21.4% of the dairy herds had completed herd JD screening, with 13.7% of herds reporting a confirmed case of JD. A further 31.5% of herds reported suspected case(s) of JD. Eighty‐nine percent of farms had introduced animals from outside the herd. Herds that utilise a mixed calving pen and hospital pen, and herds that do not separate JD‐positive or sick animals within the calving pen, were significantly (The VRAMP analysis identified areas of JD control that should be focused on in NI dairy herds, such as calving pen management and hygiene. The results highlight the importance of common JD recommendations in the management of on‐farm disease risk. Mycobacterium avium subspecies paratuberculosis (MAP) is the causative agent of Johne's disease (JD), which occurs endemically in dairy cattle. Johne's disease in cattle presents as chronic granulomatous enteritis, causing clinical signs such as decreased milk production, wasting, lethargy and diarrhoea, eventually leading to mortality.The Northern Ireland (NI) dairy sector has primarily extensive, pasture‐based farms, with animals housed over winter, grazed on pasture from spring to autumn; most farms utilise spring calving systems, with some farmers choosing all‐year calving. In 2021, approximately 318,000 dairy cows were farmed in NI on 2562 farms,Once a herd is enrolled, authorised veterinary practitioners (AVPs) who are specifically trained by AHWNI for the JDCP visit carry out what is referred to as a Veterinary Risk Assessment and Management Plan (VRAMP). The VRAMP is designed to assess each herd's individual JD risk and consists of general herd management and JD risk‐related questions. At the end of the visit, the AVP provides up to three personalised recommendations for disease control. Alongside the VRAMP, it is recommended that each farm undergo full herd JD screening, with a view to making this compulsory in the future.The objectives of this study were to ascertain the prevalence of risk factors for JD in NI dairy herds and to identify which high‐risk on‐farm practices were more prevalent on NI farms that previously had a confirmed or suspected case of JD.Herds were included in the study if they were voluntarily enrolled in the AHWNI JDCP and had completed a VRAMP between August 2020 and October 2022. A total of 1569 dairy herds were eligible for the study sample. Fifty herds had completed two VRAMP questionnaires by October 2022. The most recent VRAMP for these herds was excluded from the analysis, ensuring that all analyses were carried out on initial VRAMP data only .The VRAMP is a risk assessment tool that is used to assess and quantify a range of infection risks associated with JD.www.wufoo.com, Momentive) an online form building platform. All the data collected were stored securely within the Wufoo platform. Each herd's data are only available to the herdowner, their nominated veterinarian and to AHWNI. The data were downloaded by AHWNI from Wufoo as a CSV file and anonymised before being analysed. There were no incomplete VRAMPs, so none had to be excluded from the analysis.The questionnaire information was collected electronically via a bespoke online web form using Wufoo  and open‐ended questions similar to section one, whereas the second section comprised of a series of matrix questions that asked the AVP to rate risk factors on a scoring scale , with ‘1’ being scored if the farmer does not undertake these practices, or if the way in which the practices are undertaken has minimal risk in relation to JD, and ‘10’ being scored if the farmer undertakes the practice in such a way that could substantially increase JD transmission risk in the herd. The guidance provided to veterinarians carrying out the VRAMP on how to score risk factors is given in the Supporting Information The automatically collated data from Wufoo were exported into Microsoft Excel. Risk values of ‘1’, ‘4’, ‘7’, ‘10’ were renamed in Excel as ‘1’, ‘2’, ‘3’ and ‘4’, respectively; ‘Y’ and ‘N’ were renamed in Excel as ‘2’ and ‘1’, respectively. For frequency and descriptive analysis, risk values of ‘1’ and ‘4’ were categorised as low to medium risk and ‘7’ and ‘10’ as high to very high risk. All questions were given a short name identifier to enable identification of each variable in R. Answers to open‐ended questions requiring numeric responses were assigned to groups; for example, answers to the question ‘How many animals have you brought into the herd in the past year?’ were assigned to the following groups: ‘0’, ‘1–99’ and ‘100 or more’. Once the data were organised, descriptive analysis and frequency statistics were carried out in Microsoft Excel.For statistical analysis, herd data were split into two categories: high probability of JD present and low probability of JD present. The ‘high probability of JD present’ category included herds that answered ‘yes’ to either ‘Have you had any suspected cases of clinical Johne's disease, for example, cows with chronic diarrhoea/chronic wasting?’ or ‘Have you ever had a confirmed case of Johne's disease in your herd?’. These were herds that had previously had a confirmed faecal culture or PCR‐positive case of JD or had reported animals with suspicious clinical signs. ELISA results were not available for consideration alongside the VRAMP analysis, so herds that had ELISA‐positive animals might not be included within this category; however, we cannot assume these did not exist. These herds had a higher probability of having JD present at the point of completing the VRAMP.The ‘low probability of JD present’ category included herds that answered ‘no’ to either ‘Have you had any suspected cases of clinical Johne's disease, for example, cows with chronic diarrhoea/chronic wasting?’ or ‘Have you ever had a confirmed case of Johne's disease in your herd? (Faeces test positive)’. These herds had never had a positive faecal culture or PCR test result for JD; furthermore, they had not reported any animals with suspicious clinical signs. These herds had a lower probability of having JD at the point of completing the VRAMP. However, limited testing and potential gaps in farmer knowledge meant that answering no to these questions does not necessarily mean that the herds were infection free at the point of completing the VRAMP.U‐test was used for ratio data. Odds ratios and confidence intervals were calculated for independent variables with p‐values less than 0.15 in the univariate analysis, comparing a control value (risk factor of 1 × ‘low probability’ herd) to all other values. Independent variables with p‐values less than 0.15 in the univariate analysis were also included in a logistical regression model with a manual backwards elimination with a forwards step applied, as previously described.p < 0.05) after the logistical regression model were retained in the final model. Spearman's correlation was then used to check for co‐linearity between independent variables which were retained with rho values greater than 0.3 reported. A correlation plot between all numerical variables excluding descriptive data (such as QUB ID) was produced. Data analysis of follow‐up questions, which were only available if the answer to a previous question was ‘Yes’, were analysed within the ‘Yes’ group using Pearson's chi square and odds ratios as above.Statistical analyses of the risk‐related data were performed using R (version 4.1.0)Figure Of the 1569 herds in the study, 29.8% had previously had a suspected clinical case of JD, of which 68.5% had a suspected case within the past 3 years and 21.6% had a suspected case during the most recent year of VRAMP collection. Of herds with a suspected case, 38.3% had observed more than four suspected cases in the herd.Table Risk factors related to the use of slurry and slurry spreading equipment showed that over half 51%) of herds in the study used slurry spreading equipment, which was hired from either another farm or a contractor. Of these farmers, 43% responded that the equipment used was not cleaned or flushed through between farms. When using hired slurry equipment, 69% of farmers responded that they did not ensure that the first field that slurry was spread onto was not pasture grazed by livestock less than 6 months of age  were scored as higher risk for JD because they were likely to feed non‐saleable whole milk to calves in the herd occasionally or routinely, with 6% of herds feeding non‐saleable milk every week. In over 90% of herds, over 50% of calves were likely to consume approximately 3 L of colostrum within 2 h of birth and were fed low‐risk colostrum, for example, from their own mother, a low‐risk cow or artificial colostrum , ‘Do you graze rented land?’ (p > 0.001), ‘Are there single or multiple cows in the calving area?’ (p = 0.019), ‘Do JD‐positive or clinical cows calve in same area as the rest of the herd?’ (p > 0.001), ‘How often in the past year have neighbouring cattle broken into your cattle or vice versa?’ (p = 0.004) and ‘How many purchased animals were kept for breeding?’ (p = 0.001).Logistical regression modelling retained six variables—‘Is the calving area used for lame or sick cows?’  had undertaken any form of JD testing. Combining JD testing with milk recording, such as quarterly milk ELISA testing, can streamline the process for farmers, producers and laboratories, as well as reduce the possibility that JD‐positive cows are missed in the herd by providing frequent testing opportunities throughout the year.The adoption of biosecurity measures is important for both farm‐level and province‐wide JD control. Purchasing of cattle from infected herds or infected individuals is one of the main risk factors for JD transmission between herds.Other important recommendations for managing JD infection and preventing further infection in dairy herds include ensuring that JD‐positive, otherwise sick or lame cows are not housed or calved in calving pens with healthy, JD‐negative cows.Northern Ireland dairy herds that used grouped calving pens were significantly more likely to be at a higher risk of JD presence than herds that used individual calving pens Table . UnivariThe removal of calves from maternity pens as soon as possible after birth is recommended to reduce the exposure of calves to pathogens, including but not limited to MAP.Accidental mixing of herds through neighbouring cattle breaking into the farm or vice versa and herds that routinely grazed rented land were found to be significantly more likely to occur in herds that had a higher probability of JD present. Previous studies relating to bovine tuberculosis (bTB) have identified a link between herds that come into contact with infected neighbouring herds and a positive bTB test result.This study has several limitations. First, the JD infection status of any herd was not known at the time of VRAMP completion, so nothing can be said about timelines of infection. Data were only gathered on the most recent suspected case of JD in a herd, not positive cases, exact timings, or how many animals tested positive. Second, there were no data available on herd prevalence of JD at point of enrolment. Any identifying data, including herd size, previous ELISA results and numbers of cases, were not provided to ensure herd anonymity. Third, there was no information on whether herds had adopted JD relevant control measures prior to the VRAMP being conducted, because this is the first stage of the NI JDCP where any information has been documented. If any changes were made prior to the VRAMP completion, then these would have been based upon recommendations from a veterinarian or at the farmers discretion and they would not be included in any AHWNI captured data. Last, most importantly, a truly negative control group could not be defined. If this had been possible, then the significant risk factors may have been different.This study has highlighted that calving pen management and hygiene, as well as herd biosecurity and land management, were all found to be significantly associated with MAP infection in NI dairy herds. The results of this study highlight the importance of general JD‐related management recommendations, such as separation of high‐risk animals and calving area management. These associations, alongside the general overview of JD in NI, support the importance of disease control programmes in dairy herds.Data analysed were collected by Animal Health and Welfare Northern Ireland, downloaded from the Wufoo platform and anonymised by Sam Strain before being provided to Kayleigh Meek. Kayleigh Meek reviewed, coded and collated data and carried out statistical analyses. Irene Grant, Niamh O'Connell and Sam Strain supervised Kayleigh Meek's work. All authors reviewed and edited the manuscript before submission.Sam Strain is Chief Executive of Animal Health and Welfare Northern Ireland. The remaining authors declare they have no conflicts of interest.The authors confirm that the ethical policies of the journal, as noted on the journal's author guidelines page, have been adhered to. When herdowners registered with the Northern Ireland Johne's disease control programme, they signed an agreement giving consent that anonymised data about their herd could be shared with research organisations for the purposes of research relating to Johne's disease.S1. Details of Veterinary Risk Assessment and Management Plan (VRAMP) questions and codes used in statistical analyses of VRAMP responses.S2. Johne's disease risk score guidance—provided to veterinarians trained by Animal Health and Welfare Northern Ireland to complete the Veterinary Risk Assessment and Management Plan (VRAMP) questionnaire.Click here for additional data file.