Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> This paper presents a summary of a comprehensive research project involving development of a completely autonomous multirobot system for railway track maintenance applications. The specifications of this "real-world" application required us to consider a multirobot system conceptualization, rather than a single robot system. <s> BIB001 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> This paper presents a vision-based technique to automatically detect the absence of the fastening bolts that secure the rails to the sleepers. The images are pre-processed by using several combinations of WT and PCA methods.The final detecting system has been applied on a long sequence of real images showing a high reliability and robustness. <s> BIB002 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> Abstract The paper presents the background of the diagnostic analysis for railways and trams (DART), automatic diagnostic analysis tool for the track geometry measurement data—as separate track data files or track data read from the database. The diagnostic module indicates the “weak locations” in the track, which can be further analysed in detail by the diagnostician. DART track diagnostician's computer assistance system helps to save some 90–95% of labour consumption. All software mentioned in the paper has been developed within the framework of the project. <s> BIB003 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> This paper proposes a rail track inspection technique using automated video analysis. This system is aimed to replace manual visual checks performed by the railway engineers for track inspection. We suggest a combination of image processing and analysis methods to achieve high performance automated rail track inspection. This paper addresses the problem of finding missing clips and finding blue clips which have been recently replaced in place of damaged clips. The experimental results show high performance in machine vision based inspection on a large sample of real train video. <s> BIB004 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> Rail inspection is a very important task in railway maintenance, and it is periodically needed for preventing dangerous situations. Inspection is operated manually by trained human operator walking along the track searching for visual anomalies. This monitoring is unacceptable for slowness and lack of objectivity, as the results are related to the ability of the observer to recognize critical situations. The correspondence presents a patent-pending real-time Visual Inspection System for Railway (VISyR) maintenance, and describes how presence/absence of the fastening bolts that fix the rails to the sleepers is automatically detected. VISyR acquires images from a digital line-scan camera. Data are simultaneously preprocessed according to two discrete wavelet transforms, and then provided to two multilayer perceptron neural classifiers (MLPNCs). The "cross validation" of these MLPNCs avoids (practically-at-all) false positives, and reveals the presence/absence of the fastening bolts with an accuracy of 99.6% in detecting visible bolts and of 95% in detecting missing bolts. A field-programmable gate array-based architecture performs these tasks in 8.09 mus, allowing an on-the-fly analysis of a video sequence acquired at 200 km/h <s> BIB005 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> The increasing train speed on railways generates an urgent need for more powerful automatic inspection of railway tracks, including real-time fastening component inspection. To obtain better high-speed performance with lower cost, this paper has proposed a novel structured light method based on motion image (SLMMI) for moving object inspection. The motion images in the proposed method are insensitive to motion, abundant with information, and easy to process, resulting in a low performance requirement of the hardware. Compared to the conventional unstructured light method and structured light method, the proposed method inherits the virtues of both thus offering a fresh perspective when inspecting missing fastening components on high-speed railways. By using the SLMMI and the recognition method based on a neural network, the experimental results yield good performance in terms of speed and accuracy. Furthermore, the robustness of the proposed method is also discussed and simulated by adding typical interferences, such as ambient light, vibration, and obstacles. <s> BIB006 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> This paper considers the development of an integrated robotic system which may be used to automatically locate and then systematically scan rolling contact fatigue cracks on rails to provide more detailed information on the cracks’ sub-surface size. The system consists of a mechanised trolley, a robot, a commercially available AMIGO alternating current field measurement (ACFM) system and a laser distance sensor. The dynamic results, when the trolley moves at a controlled speed to detect defects, and the static results, when the defect location has been determined and the robot arm is used to characterise (size) the defect, are both considered. During field trials it was found that in the detection test most of the artificial cracks were detected automatically by employing a novel combined threshold and signature match (CTSM) method. The results of the characterisation test also suggest that the system can automatically take reliable constant lift-off ACFM measurements over the cracks at close to optimum range for characterisation where the relative angle offset between the ACFM sensor and the crack is bounded to be within 5°. <s> BIB007 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> Thousands of miles of railroad track must be inspected twice weekly by a human inspector to maintain safety standards. A computer vision system, consisting of field-acquired video and subsequent analysis, could improve the efficiency of the current methods. Such a system is prototyped, and the following challenges are addressed: the detection, segmentation, and defect assessment of track components whose appearance vary across different tracks and the identification and inspection of special track areas such as track turnouts. An algorithm that utilizes the periodic manner in which track components repeat in an inspection video is developed. Spectral estimation and signal-processing methods are used to provide robust detection of the periodically occurring track components. Results are demonstrated on field-acquired images and video. <s> BIB008 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> In this paper, we present a real-time automatic vision-based rail inspection system, which performs inspections at 16 km/h with a frame rate of 20 fps. The system robustly detects important rail components such as ties, tie plates, and anchors, with high accuracy and efficiency. To achieve this goal, we first develop a set of image and video analytics and then propose a novel global optimization framework to combine evidence from multiple cameras, Global Positioning System, and distance measurement instrument to further improve the detection performance. Moreover, as the anchor is an important type of rail fastener, we have thus advanced the effort to detect anchor exceptions, which includes assessing the anchor conditions at the tie level and identifying anchor pattern exceptions at the compliance level. Quantitative analysis performed on a large video data set captured with different track and lighting conditions, as well as on a real-time field test, has demonstrated very encouraging performance on both rail component detection and anchor exception detection. Specifically, an average of 94.67% precision and 93% recall rate has been achieved for detecting all three rail components, and a 100% detection rate is achieved for compliance-level anchor exception with three false positives per hour. To our best knowledge, our system is the first to address and solve both component and exception detection problems in this rail inspection area. <s> BIB009 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> Fasteners are critical railway components that maintain the rails in a fixed position. Their failure can lead to train derailments due to gage widening or wheel climb, so their condition needs to be periodically monitored. Several computer vision methods have been proposed in the literature for track inspection applications. However, these methods are not robust to clutter and background noise present in the railroad environment. This paper proposes a new method for fastener detection by 1) carefully aligning the training data, 2) reducing intra-class variation, and 3) bootstrapping difficult samples to improve the classification margin. Using the histogram of oriented gradients features and a combination of linear SVM classifiers, the system described in this paper can inspect ties for missing or defective rail fastener problems with a probability of detection of 98% and a false alarm rate of 1.23% on a new dataset of 85 miles of concrete tie images collected in the US Northeast Corridor (NEC) between 2012 and 2013. To the best of our knowledge, this dataset of 203,287 crossties is the largest ever reported in the literature. <s> BIB010 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Track Maintenance <s> In most railway companies in China, the loss of nuts from rail fasteners is still found by visual inspections performed by maintenance workers; this approach cannot meet the continuing requirements to reduce maintenance costs and improve maintenance efficiency. In this paper, a computer vision system for detecting the loss of nuts from rail fasteners is proposed that is based on kernel two-dimensional principal component – two-dimensional principal component analysis (K2DPCA-2DPCA) and a support vector machine (SVM). The framework of this information-technology-based system is introduced, and the calculation method to find the positions where nuts are missing is also proposed. The K2DPCA-2DPCA feature extraction method and SVM used for feature classification in the computer vision system are described in detail. Finally, the combination of K2DPCA-2DPCA and SVM is compared with other feature extraction and feature classification methods in an experiment on the detection of the loss of nuts from rail fasten... <s> BIB011
Railway track maintenance plays a major role in the railway industry. Trivedi et al. have proposed a fully autonomous robotic system to detect and remove sleeper bolts, feed new fastener, and assemble them in Shinkansen high-speed train lines. Instead of creating one massive robot to perform all tasks, researchers have taken a different approach and created a multi-robotic system, where the individual robot will focus on a simplified task. According to Trivedi et al., the system is technically and economically viable BIB001 . Further, Rowshandel et al have proposed an integrated robotic system for the detection and characterization of rolling contact fatigue cracks in rails. The suggested system consists of a mechanized trolley, a robot, a commercially available alternating current field measurement (ACFM) system, and a laser distance sensor. The trolley, which contains the articulated robot, will travel at a controlled speed scanning for surface cracks on the rail and the robot will take over the scanning process while using the ACFM system once a crack is detected. Further, it has been concluded that the system manages to autonomously detect and characterize surface-breaking defects with high precision BIB007 . Furthermore, researchers have successfully exploited the recent developments of machine vision and classification algorithms to develop RAS in rail track maintenance tasks, especially for inspection and monitoring BIB002 BIB008 . Santur et al. have proposed a robust rail condition monitoring methodology while using a laser scanner mounted on a moving rail car, which detects fractures, scorings, and excessive wear of the track. Random Forest classification learning algorithm is used in their system by considering the Eigenvalues of rail profiles as input. According to the authors, the system demonstrates 98% accuracy  . A similar track geometry diagnostic system has been developed by Madejski, which alerts the weak locations of tracks and helps to save about 90%-95% labour consumption BIB003 . In addition, there are few dozens of RAS that have been developed to detect the missing or worn out track components, such as bolts, clips, ties, tie plates, anchors, and turnout components BIB011 BIB006 BIB005 . For example, an autonomous clip detection and classification system was introduced by Gibert et al., where HOG of target images was analyzed to detect clips and classify their condition as good, missing, or broken by taking advantage of Linear Support Vector Machine (LSVM) learning algorithm. The described system is capable of achieving a probability of detection of 98% and a false alarm rate of 1.23% BIB010 . Another application is developed to detect missing railway clips and recently replaces clips that are different in color by employing basic image processing techniques, such as Gaussian filtering, Canny edge detection, suppressing smaller edges from a binary image, analyzing edge pixels density of binary images, and histogram of colors. According to Singh et al., the system is accurate up to 95.3%, 86.5%, and 84.7% in finding clips, blue clips, and missing clips, respectively BIB004 . In addition, Li et al present a real-time automatic vision-based rail inspection system, which performs inspections at 16 km/h with a frame rate of 20 fps to detect and inspect tie plate, tie, and anchor. The proposed system that is composed with an image and video analysis methodology, which is combined with multiple cameras, Global Position System (GPS) and a distance measurement instrument. The system consists of an acquisition module, rail component detection, optimization module, and an exception detection module to detect damaged or missing components. Further, the systems acquired following detection results, Tie plate: 100% recall rate and 99.3% precision, Tie: 82.3% recall rate and 88.2% precision, and Anchor: more than 96% recall rate and precision BIB009 . A higher technology readiness level robotic solution for rail track inspection is developed by "Autoscan", which uses electromagnetic acoustic transducers (EMATs) that are mounted to a self-powered autonomous inspection cart with a robotic arm. The system is at TRL 6 and it has the ability to reduce track inspection costs by at least 15%. It is expected to record different defects with an accuracy of +/-0.5 mm  . In addition, a commercially available automated rail inspection robotic platform has been introduced by "RailPod". The platform is composed of a vehicle that can be remotely controlled or drive autonomously and it contains the ability to configure based on individual customer need  .