Modern Trends in Hyperspectral Image Analysis: A Review <s> E. FORENSIC DOCUMENT EXAMINATION <s> Hyperspectral imaging (HSI) of historical documents is becoming more common at national libraries and archives. HSI is useful for many tasks related to document conservation and management as it provides detailed quantitative measurements of the spectral reflectance of the document that is not limited to the visible spectrum. In this paper, we focus on how to use the invisible spectra, most notably near-infrared (NIR) bands, to assist in visually enhancing old documents. Specifically, we demonstrate how to use the invisible bands to improve the visual quality of text-based documents corrupted with undesired artifacts such as ink-bleed, ink-corrosion, and foxing. For documents of line drawings that suffer from low contrast, we use details found in the invisible bands to enhance legibility. The key components of our framework involve detecting regions in the document that can be enhanced by the NIR spectra, compositing the enhanced gradient map using the NIR bands, and reconstructing the final image from the composited gradients. This work is part of a collaborative effort with the Nationaal Archief of the Netherlands (NAN) and Art Innovation, a manufacturer of hyperspectral imaging hardware designed specially for historical documents. Our approach is evaluated on historical documents from NAN that exhibit degradations common to documents found in most archives and libraries. <s> BIB001 </s> Modern Trends in Hyperspectral Image Analysis: A Review <s> E. FORENSIC DOCUMENT EXAMINATION <s> Hyperspectral imaging (HSI) integrates conventional imaging and spectroscopy, to obtain both spatial and spectral information from a specimen. This technique enables investigators to analyze the chemical composition of traces and simultaneously visualize their spatial distribution. HSI offers significant potential for the detection, visualization, identification and age estimation of forensic traces. The rapid, non-destructive and non-contact features of HSI mark its suitability as an analytical tool for forensic science. This paper provides an overview of the principles, instrumentation and analytical techniques involved in hyperspectral imaging. We describe recent advances in HSI technology motivating forensic science applications, e.g. the development of portable and fast image acquisition systems. Reported forensic science applications are reviewed. Challenges are addressed, such as the analysis of traces on backgrounds encountered in casework, concluded by a summary of possible future applications. <s> BIB002 </s> Modern Trends in Hyperspectral Image Analysis: A Review <s> E. FORENSIC DOCUMENT EXAMINATION <s> Hyperspectral imaging and analysis refers to the capture and understanding of image content in multiple spectral channels. Satellite and airborne hyperspectral imaging has been the focus of research in remote sensing applications since nearly the past three decades. Recent use of ground-based hyperspectral imaging has found immense interest in areas such as medical imaging, art and archaeology, and computer vision. In this paper, we make an attempt to draw closer the forensic community and image analysis community towards automated forensic document examination. We believe that it has a huge potential to solve various challenging document image analysis problems, especially in the forensic document examination domain. We present the use of hyperspectral imaging for ink mismatch detection in handwritten notes as a sample application. Overall, this paper provides an overview of the applications of hyperspectral imaging with focus on solving pattern recognition problems. We hope that this work will pave the way for exploring its true potential in the document analysis research field. Keywords—Multispectral imaging, Hyperspectral imaging, Hyperspectral document analysis, forensic document examination, ink mismatch detection <s> BIB003 </s> Modern Trends in Hyperspectral Image Analysis: A Review <s> E. FORENSIC DOCUMENT EXAMINATION <s> Hyperspectral images in the near infrared range (HSI-NIR) were evaluated as a nondestructive method to detect fraud in documents. Three different types of typical forgeries were simulated by (a) obliterating text, (b) adding text and (c) approaching the crossing lines problem. The simulated samples were imaged in the range of 928-2524 nm with spectral and spatial resolutions of 6.3 nm and 10 μm, respectively. After data pre-processing, different chemometric techniques were evaluated for each type of forgery. Principal component analysis (PCA) was performed to elucidate the first two types of adulteration, (a) and (b). Moreover, Multivariate Curve Resolution-Alternating Least Squares (MCR-ALS) was used in an attempt to improve the results of the type (a) obliteration and type (b) adding text problems. Finally, MCR-ALS and Partial Least Squares-Discriminant Analysis (PLS-DA), employed as a variable selection tool, were used to study the type (c) forgeries, i.e. crossing lines problem. Type (a) forgeries (obliterating text) were successfully identified in 43% of the samples using both the chemometric methods (PCA and MCR-ALS). Type (b) forgeries (adding text) were successfully identified in 82% of the samples using both the methods (PCA and MCR-ALS). Finally, type (c) forgeries (crossing lines) were successfully identified in 85% of the samples. The results demonstrate the potential of HSI-NIR associated with chemometric tools to support document forgery identification. <s> BIB004 </s> Modern Trends in Hyperspectral Image Analysis: A Review <s> E. FORENSIC DOCUMENT EXAMINATION <s> Hyperspectral imaging helps in identifying patterns and objects in an observed hyperspectral scene on the basis of their unique spectral signatures; such identification is otherwise difficult using regular imaging. Recently, ink mismatch detection analysis based on hyperspectral imaging has shown enormous potential in distinguishing visually similar inks. Such analysis provides significant information to forensic document examiners to determine the authenticity of the questioned documents. However, a major challenge still exists in disproportionate ink mismatch detection because it is inherently an unbalanced clustering problem. The presented approach deals with ink mismatch detection in unbalanced clusters by using hyperspectral unmixing scheme. It identifies the spectral signatures (endmembers) of the inks and their corresponding proportions (abundances). Our results show that HySime outperforms other methods in signal subspace estimation. Hyperspectral unmixing is done by using minimum volume enclosing simplex algorithm. Efficacy of the purposed approach is demonstrated by successfully distinguishing varying disproportionate ink datasets generated from UWA database and results are compared with existing state of the art methods in hyperspectral ink mismatch detection field. We expect that these finding will further encourage the use of hyperspectral imaging in document analysis, particularly towards automated questioned document examination. <s> BIB005
Traditionally, forensic document experts and paleographers used chemical solution based methods to study the extrinsic and intrinsic components of the important historic documents  . This is due to the fact that the inks used on documents throughout the history were composed of diverse substances having distinct chemical and physical properties. All of these substances have their own unique way for reacting with different substrates depending upon the reaction environment. These chemical solution based methods helped in document analysis. But unfortunately these techniques were time consuming, sensitive to temperature changes and destructive in nature i.e. harms to the important documents were irreversible. To overcome such limitations, HSI has emerged as an effective non-destructive tool for improving readability BIB001 , ink aging and forensic document analysis BIB002 . Hyperspectral document imaging works on the principle that each ink present in the document has its own unique spectral signature. Many mathematical tools that are being used in hyperspectral remote sensing can be used on hyperspectral document images for classification, improving legibility of extremely deteriorated text, ink aging and fraud detection. Moreover HSI is non-destructive, automated and environment insensitive tool for document examination. In hyperspectral document imaging, ink mismatch detection analysis provides important information to forensic document examiners to determine the authenticity of legal documents. Forgery, backdating and fraud can be detected using ink analysis of documents. HSI has been used for forensic document analysis in the past BIB004 - BIB003 . The HSI based techniques have had a significant impact on forgery detection as compared to the traditional techniques. Different inks were successfully classified after obliteration of the text in a document BIB004 . Abbas et al. BIB005 proposed hyperspectral unmixing for discrimination between inks from different pens in a document. Recently, we have worked on a nondestructive automated forgery detection system, which was able to successfully discriminate between different visually similar inks taken from the publicly available UWA Writing Ink Hyperspectral Images (WIHSI) Database BIB005 with different number of inks and different mixing ratios (Combination A∼H). Initially, we used Fuzzy C-Means Clustering (FCM) to differentiate two inks present in a multispectral document in different mixing ratios  . In our latest work, the state of the art deep learning technique, Convolutional Neural Network (CNN) has been employed for classification of spectral responses of ink pixels for forgery detection in hyperspectral document images. Figure 9 shows the final segmentation results of our proposed technique. 98% classification accuracy was achieved which shows the high potential of HSI and deep learning in document forgery detection.