Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> Wayside detector devices, over the years, have proven to be valuable assets to a railroad in enhancing safety and improving efficiencies. Currently, devices such as hot box detectors, dragging equipment detectors, high car detectors, wheel impact detectors have routinely prevented accidents or damages from occurring. With the advent of electronics in terms of its sophistication, capability, cost and miniaturization, a lot of interest arose to wayside devices to further improve efficiencies and reduce manual inspection requirements of freight car components. The integration of these devices between each other and their integration into the railroad maintenance system increase the potential for efficiency gains. At this juncture, Conrail has re-examined the usage of electronics and has advanced the use of technology in fault detection of freight car components. This paper gives an account of freight car failure detection devices which are currently available in service and which are being developed in 1998. Conrail's integration of some of the devices and the automation of reporting the faults and rerouting of defective cars is also presented. <s> BIB001 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> An automated vision system that accurately measures the thickness of a brake shoe for rolling stock is proposed. Images of the brake shoes are captured automatically while rolling stock is passing by an inspection station. Toward this goal, location of the brake shoe in the image has to be determined first because they are not the same in the captured images. Shadow regions between the brake shoes and wheels are utilized that are common in all captured images. The boundary of the shadow regions is modeled by an inverse polynomial, and the constrained curve fitting method is adopted. Then, two curves that correspond to the front and back of the brake shoe are determined using the initial curve and intensity change information. Finally, the thickness is calculated using the detected curves, and experimental results showed that the brake shoe thickness was measured with an accuracy of 0.654mm. <s> BIB002 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> To ensure the safe and efficient operation of the approximately 1.6 million freight cars (wagons) in the North American railroad network, the United States Department of Transportation (USDOT), Federal Railroad Administration (FRA) requires periodic inspection of railcars to detect structural damage and defects. Railcar structural underframe components, including the center sill, sidesills, and crossbearers, are subject to fatigue cracking due to periodic and/or cyclic loading during service and other forms of damage. The current railcar inspection process is time-consuming and relies heavily on the acuity, knowledge, skill, and endurance of qualified inspection personnel to detect these defects. Consequently, technologies are under development to automate critical inspection tasks to improve their efficiency and effectiveness. Research was conducted to determine the feasibility of inspecting railcar underframe components using machine vision technology. A digital video system was developed to record images of railcar underframes and computer software was developed to identify components and assess their condition. Tests of the image recording system were conducted at several railroad maintenance facilities. The images collected there were used to develop several types of machine vision algorithms to analyze images of railcar underframes and assess the condition of certain structural components. The results suggest that machine vision technology, in conjunction with other automated systems and preventive maintenance strategies, has the potential to provide comprehensive and objective information pertaining to railcar underframe component condition, thereby improving utilization of inspection and repair resources and increasing safety and network efficiency. <s> BIB003 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> The Iron Ore Line (Malmbanan) is a 473 km long track section located in northern Sweden and has been in operation since 1903. This track section stretches through two countries, namely Sweden and Norway, and the main part of the track runs on the Swedish side, where the owner is the Swedish Government and the infrastructure manager is Trafikverket (the Swedish Transport Administration). The ore trains are owned and managed by the freight operator and mining company LKAB. Due to the high axle load exerted by transportation of the iron ore, 30 tonnes, and the high demand for a constant flow of ore and pellets, the track and wagons must be monitored and maintained on a regular basis. The condition of the wagon wheel is one of the most important aspects in this connection, and here the wheel profile plays an important role. For this reason an automatic laser-based wheel profile monitoring system (WPMS) has been installed on this line using a system lifecycle approach that is based on the reliability, availabi... <s> BIB004 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> Fault inspection is a key part of ensuring safe operation of freight trains. The development of machine vision technology has resulted in vision-based fault inspection becoming the principal means of fault inspection. An angle cock is an important component in the brake system, and a fault in it could lead to a serious accident. In this paper, we propose an automated vision method to inspect for missing handles on an angle cock during operation of a freight train. Images of the angle cock are acquired and they are analyzed using a proposed gradient encoding histogram and support vector machine that combine to create a detection system. Experimental results show that we achieved a fault detection rate of 99.8% using the proposed system, which represents a good real-time performance and high detection accuracy. <s> BIB005 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> This paper, from the 2014 Rail Conference, describes the Automatic Vehicle Inspection System (AVIS, Bombardier), a complete system that measures, analyzes, monitors, establishes trends and automatically generates work orders for the maintenance crew. It also generates alarms and permits data trending for long-term engineering investigations. In August 2013, the Bombardier site team at the Midrand depot in South Africa, together with experts from MRX technologies and IBM, commissioned an Automatic Vehicle Inspection System (AVIS), completing three years of cutting edge technology development, to deliver a revolutionary integrated solution for the rolling stock maintenance industry. The commissioning was achieved without major problems, all functionalities were tested with conclusive results, and the team was so enthusiastic about the preliminary results that plans were already being devised for the next generation. The system’s benefits are such that a second similar system is currently being installed at Bombardier’s Central Rivers depot in Midlands, UK, and several other projects are currently being planned or developed in the UK, Australia and North America. The authors conclude that the AVIS is a significant addition to Bombardier’s Operations & Maintenance approach and paves the way for exciting new developments in the industry. For the passenger train maintenance industry, the systematic gathering of high quality data is key to measuring and optimizing the maintenance approach. The introduction of an Automated Vehicle Inspection System provides that capability – at an affordable cost. <s> BIB006 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> Condition monitoring of rolling stock has been part of the railway landscape for the past decade; thermal, acoustic and light sensors are used to capture fleet information on bearings, brakes, wheels, under-carriage and body. The data generated from these sensors has helped rail operators direct their maintenance efforts to the components of highest priority; however, in most cases it has not triggered a fundamental philosophical change in the maintenance strategy. Rail operators generally use time-based maintenance interventions to ensure fleet reliability and safety. Technology is enabling rail operators to take advantage of the benefits of moving from a time-based maintenance regime to a condition-based regime. However, technology simply provides the tools. The benefits of higher reliability and lower costs can only be realized when the organizational culture adapts to make best use of the technology. This paper will discuss the technology that can be deployed to enable a condition-based culture and some of the organizational issues that need to adapt to the new technology. The observations in this paper are not based on a structured research program, but rather are observational, based on out daily dealings with several rail operators around the world. <s> BIB007 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> The early detection of faults in rolling stock wheels and axle bearings is of paramount importance for rail infrastructure managers as it contributes to the safety of rail operations. In this paper we report on the key results that have arisen from the development and implementation of a novel condition monitoring system based on high-frequency acoustic emission and vibration analysis installed on a train. The novel system makes use of inexpensive and robust acoustic emission sensors and accelerometers, which can be easily installed on the axle bearing box with minimal intervention required. Experimental work carried out under actual conditions at the Long Marston rail track and on the Lisbon – Cas-Cais suburban line has proven that the developed system is capable of detecting wheel and axle bearing-related defects with various levels of severity. <s> BIB008 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> The status inspection and maintenance actions on the mechanical components of freight trains are significant determinants of railway safety. Fastening bolts, as a key component, are widely used to transmit, connect and fasten various parts of freight trains. Their absence can degrade the original structure and lead to accidents. Recently, machine vision approaches have been widely used to inspect the status of mechanical components, thereby reducing costs and avoiding traffic accidents. This paper proposes a visual inspection system that is based on the machine vision approach and can be used to automatically inspect the status of fastening bolts on freight trains. To detect the presence/absence of a fastening bolt in a complex background, a hierarchical detection framework consisting in fault area extraction and fastening bolt detection is proposed. In the first module, a gray projection method is used to divide the fault area that contains the target from the complex background. Subsequently, a fastenin... <s> BIB009 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> Inspecting the condition of the key components of freight trains is an important task in the rail industry. Bolts on the wheel bearings are key components of a bogie, and bolt defects, such as missing or broken bolts, can lead to serious accidents. To improve the traditional manual inspection procedure, which is both laborious and inflexible, a novel method of automatic image recognition for bolt defects is proposed in this paper. The main procedures are as follows. When a freight train drives through the inspection station, images of the train’s wheels are captured by cameras installed alongside the track. Based on the local binary pattern descriptor, a support vector machine classifier is trained to distinguish between bolt and non-bolt images. The classifier is then combined with a rotate-and-slide window method to localize the three bolt regions in the wheel image. Specifically, a self-updating method is proposed in the training phase to automatically capture the various different situations experienc... <s> BIB010 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> With the development of hardware and software in camera and computing units, visual inspection system (VIS) plays an increasing significant role in fault inspection task. This paper proposes a VIS to inspect the missing of bogie block key (BBK) used on freight trains. BBK is an important component to keep wheel sets from separating out of bogies. The missing of BBK is one of the most common faults threatening the running safety. VIS first acquires a bogie image by the image acquisition system, and then a hierarchical inspection framework containing bearing cap (BC) detection, fault region localization, and BBK classification is proposed. Specifically, a cascaded detector trained by the AdaBoost approach combined with the gradient coded co-occurrence matrix (GCCM) features is used to achieve fast and accurate BC detection. Then, a fault region is located on the basis of the relationship between the BC and BBK. Finally, a BBK classifier based on the GCCM features and a support vector machine is used to process the fault region to identify the missing of BBK. The proposed system has been applied on a long sequence of real images showing high inspection speed and accuracy. <s> BIB011 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> One of the most important safety-related tasks in the rail industry is an early detection of defective rolling stock components. Railway wheels and wheel bearings are the two components prone to damages due to their interactions with brakes and railway track, which makes them a high priority when the rail industry investigates improvements in the current detection processes. One of the specific wheel defects is a flat wheel, which is often caused by a sliding wheel during a heavy braking application. The main contribution of this paper is the development of a computer vision method for automatically detecting the sliding wheels from images taken by wayside thermal cameras. As a byproduct, the process will also include a method for detecting hot bearings from the same images. We first discuss our automatic detection and segmentation method, which identifies the wheel and bearing portion of the image. Then, we develop a method, using histogram of oriented gradients to extract the features of these regions. These feature descriptors are later employed by support vector machine to build a fast classifier with a good detection rate, which can detect abnormalities in the wheel. At the end, we train our algorithm using simulated images of sliding wheels and test it on several thermal images collected in a revenue service by the Union Pacific Railroad in North America. <s> BIB012 </s> Importance and Applications of Robotic and Autonomous Systems (RAS) in Railway Maintenance Sector: A Review <s> RAS in Rolling-Stock Inspection <s> Abstract In railway rolling-stock maintenance, where the environment is highly unstructured and unpredictable, the detection and localization of smaller components is difficult and remains a key challenge in introducing autonomous systems. This paper presents the design and evaluation of a system, which allows industrial robots to inspect the electric head of railway couplers. The system developed comprising multiple sensors and active-compliant end of arm tooling, is robust to noise and ambiguities. Successful validation of the system is achieved using both an actual coupler in its working environment and an experimental setup that includes a commercial grade industrial robot and a mock coupler. <s> BIB013
Rolling-stock inspection is expensive, labor-intensive, and critical for safe operation. Therefore, certain inspection tasks have been automated over the past several decades. For example, the implementation of wayside detectors, cameras mounted to the rolling-stock undercarriage, and automated inspection tasks in maintenance depots BIB007 . In most cases, rolling-stock wheels and bearings are responsible for derailments BIB001 . Therefore, a number of automated wayside detectors are designed and developed to overcome this issue. In Conrail, the derailments due to defects in the wheels were reduced almost 50% by introducing wayside detectors BIB001 . Papaelias et al. have proposed a rolling-stock wheel and axle bearing monitoring system that is based on high-frequency acoustic emission and vibration analysis mounted to a train. The study has found that it is viable to detect and classify axle bearing defects by integrating high-frequency acoustic emission data with vibration data based on their severity BIB008 . However, these wayside monitoring systems need to be robust and withstand harsh weather conditions. Therefore, Asplund et al. have tested an automatic laser-based wheel profile monitoring system (WPMS) for accuracy and performance in extreme weather conditions. The study shows that laser-based WPMS indicates a deviation of less than 0.2 mm for the flange height, 0.3 mm for the flange thickness, and 0.32 mm for the flange slope as compared to measurements collected through handheld MiniProf measurement equipment BIB004 . Hart et al. have proposed a multi-spectral rolling-stock undercarriage inspection technique that combines visible and infrared images, which allow for recording both physical and thermal conditions, and the correlation between two. The scope of this study includes the monitoring and inspection of disc brake condition, bearing performance, and detection of any anomalies in electrical systems, such as induction motor and air-condition unit  . A similar thermal imagery approach is used by Deilamsalehy et al. to detect sliding wheels and hot bearings. The proposed methodology consists of a wheel and bearing detection module, hot bearing detection module using thresholding, and a Support Vector Machine (SVM) classifier, which uses a Histogram Oriented Gradient (HOG) of thermal images as input to detect flat wheels. The developed system is capable of detecting simulated and real-world defective wheels, with 98% accuracy BIB012 . Furthermore, a study has been conducted to determine the feasibility of inspecting rail underframe components of freight cars, such as center sill, side-sills, and cross-bearers. The study has proposed both a hardware and software to collect the required visual data and detect and assess rail undercarriage components for any deformation or buckling and presence of breaks or cracks. The study concludes that presented image acquisition and machine vision algorithms demonstrated the feasibility of automating the inspection of structural components BIB003 . Liu et al. have proposed an automated status inspection of fastening bolts on freight trains while using machine vision. The developed algorithm delivered the best performance by combining gradient orientation co-occurrence matrix and SVM classifier. The system is able to detect the existence or nonexistence of a bolt in a complex environment with 99.96% accuracy and it is expected to evaluate the performance of the system in adverse weather conditions BIB009 . In addition, Li et al. have proposed another automatic bolt defect recognition system-which focuses on freight train wheels. Similar to the method that was proposed by Liu et al., they have utilized the SVM classifier to distinguish the presence or absence of a bolt. In contrast, Li et al. have used a binary pattern descriptor as input to the classifier and their system consists of a self-updating scheme, which automatically captures various real-life scenarios. The system, as presented by Li et al., has demonstrated 100% bolt defect detection rate with 8.1% false alarm rate BIB010 . Additionally, Kim and Kim have proposed an automatic system that accurately measures brake shoe thickness with an accuracy of 0.654 mm. They have successfully exploited the shadow regions between the brake shoe and wheels that look similar in all images to localize the brake shoes by modeling the boundaries as inverse polynomials BIB002 . Furthermore, Zhou et al. have presented a real-time automated visual inspection system of angle cocks, which is a critical component of a rolling-stock brake system. The developed system incorporates a gradient encoding histogram and SVM in the processing module. It has been claimed that the system achieves 99.8% accuracy in fault detection BIB005 . In addition, the Bombardier team at Midrand, South Africa has commissioned an Automatic Vehicle Inspection System (AVS), together with expertise in MRX technologies and IBM, which is proven to be extremely successful. The system is capable of monitoring multiple aspects, such as brake pad and disc conditions, axle temperature, wheel profile, wheel damage, pantograph wear, and any abnormalities on train exterior profile while using cutting-edge three-dimensional (3D) image acquisition and processing tools BIB006 . Edwards at al. and Liu et al. have presented online visual inspection systems that capable inspecting rail car safety appliances and bogie block keys of freight trains respective. Both systems utilize machine vision, advanced image processing algorithms, and state of the art machine learning algorithms to identify the defects  BIB011 . Besides, Vithanage et al. have presented an image processing system to identify and localize the connector pins of Scharfenberg type automatic couplers mounted on Siemens Class 380 trains. The system claims to have high detection rate and robustness to environmental noises. The proposed system expects containing an off the shelf industrial robot to execute the inspection process  . This is system was further developed to have a unified 3D vision and a force-position controller. The prototyped work cell is validated while using an industrial grade 6-DoF articulated robot. It has been estimated to improve the standard operating time by approximately 34% BIB013 .