One of the most common modes of representing engineering schematics are Piping and Instrumentation diagrams (P&IDs) that describe the layout of an engineering process flow along with the interconnected process equipment. Over the years, P&ID diagrams have been manually generated, scanned and stored as image files. These files need to be digitized for purposes of inventory management and updation, and easy reference to different components of the schematics. There are several challenging vision problems associated with digitizing real world P&ID diagrams. Real world P&IDs come in several different resolutions, and often contain noisy textual information. Extraction of instrumentation information from these diagrams involves accurate detection of symbols that frequently have minute visual differences between them. Identification of pipelines that may converge and diverge at different points in the image is a further cause for concern. Due to these reasons, to the best of our knowledge, no system has been proposed for end-to-end data extraction from P&ID diagrams. However, with the advent of deep learning and the spectacular successes it has achieved in vision, we hypothesized that it is now possible to re-examine this problem armed with the latest deep learning models. To that end, we present a novel pipeline for information extraction from P&ID sheets via a combination of traditional vision techniques and state-of-the-art deep learning models to identify and isolate pipeline codes, pipelines, inlets and outlets, and for detecting symbols. This is followed by association of the detected components with the appropriate pipeline. The extracted pipeline information is used to populate a tree-like data-structure for capturing the structure of the piping schematics. We evaluated proposed method on a real world dataset of P&ID sheets obtained from an oil firm and have obtained promising results.
However, we discovered a significant body of work on recognition of symbols in prior art. @cite_19 proposed Fourier Mellin Transform features to classify multi-oriented and multi-scaled patterns in engineering diagrams. Other models utilized for symbol recognition include Auto Associative neural networks @cite_4 , Deep Belief networks @cite_20 , and consistent attributed graphs (CAG) @cite_15 . There are also models that use a set of visual features which capture online stroke properties like orientation and endpoint location @cite_16 , and shape based matching between different symbols @cite_11 . We see that most of the prior work focuses on extracting symbols from such engineering diagrams or flow charts. To the best of our knowledge, there exists no work which has proposed an end-to-end pipeline for automating the information extraction from plant engineering diagrams such as P &ID.
@cite_4 @cite_19 @cite_15 @cite_16 @cite_20 @cite_11
2031948510 2798365843 1929903369 2125380992
We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0 of the symbols having 55.1 of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7 . This result is superior to the state-of-the-art method with the precision of 92.4 . Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams. Compared to earlier multistage frameworks using CNN features, recent end-to-end deep approaches for fine-grained recognition essentially enhance the mid-level learning capability of CNNs. Previous approaches achieve this by introducing an auxiliary network to infuse localization information into the main classification network, or a sophisticated feature encoding method to capture higher order feature statistics. We show that mid-level representation learning can be enhanced within the CNN framework, by learning a bank of convolutional filters that capture class-specific discriminative patches without extra part or bounding box annotations. Such a filter bank is well structured, properly initialized and discriminatively learned through a novel asymmetric multi-stream architecture with convolutional filter supervision and a non-random layer initialization. Experimental results show that our approach achieves state-of-the-art on three publicly available fine-grained recognition datasets (CUB-200-2011, Stanford Cars and FGVC-Aircraft). Ablation studies and visualizations are provided to understand our approach. Deep convolutional neural networks (CNN) have seen tremendous success in large-scale generic object recognition. In comparison with generic object recognition, fine-grained image classification (FGIC) is much more challenging because (i) fine-grained labeled data is much more expensive to acquire (usually requiring domain expertise); (ii) there exists large intra-class and small inter-class variance. Most recent work exploiting deep CNN for image recognition with small training data adopts a simple strategy: pre-train a deep CNN on a large-scale external dataset (e.g., ImageNet) and fine-tune on the small-scale target data to fit the specific classification task. In this paper, beyond the fine-tuning strategy, we propose a systematic framework of learning a deep CNN that addresses the challenges from two new perspectives: (i) identifying easily annotated hyper-classes inherent in the fine-grained data and acquiring a large number of hyper-class-labeled images from readily available external sources (e.g., image search engines), and formulating the problem into multitask learning; (ii) a novel learning model by exploiting a regularization between the fine-grained recognition model and the hyper-class recognition model. We demonstrate the success of the proposed framework on two small-scale fine-grained datasets (Stanford Dogs and Stanford Cars) and on a large-scale car dataset that we collected. Symbol recognition is a well-known problem in the field of graphics. A symbol can be defined as a structure within document that has a particular meaning in the context of the application. Due to their representational power, graph structures are usually used to represent line drawings images.An accurate vectorization constitutes a first approach to solve this goal. But vectorization only gives the segments constituting the document and their geometrical attributes.Interpreting a document such as P&ID (Process & Instrumentation)diagram requires an additional stage viz. recognition of symbols in terms of its shape. Usually a P&ID diagram contain several types of elements, symbols and structural connectivity. For those symbols that can be defined by a prototype pattern, we propose an iterative learning strategy based on Hopfield model to learn the symbols, for subsequent recognition in the P&ID diagram. In a typical shape recognition problem one has to account for transformation invariance. Here the transformation invariance is circumvented by using an iterative learning approach which can learn symbols with high degree of correlation.