TECHNICAL FIELD

The present disclosure relates to systems and methods for improving label-free vibrational imaging by stimulated Raman scattering (SRS). Improvements include systems and methods for implementing high-speed delay scanning (which may advantageously include tunable delay range capabilities) and systems and methods implementing computational techniques (including deep learning) for improving the signal-to-noise ratio (SNR) of spectroscopic images with both spectral and spatial domains.

BACKGROUND

Stimulated Raman scattering (SRS) microscopy is a high-speed vibrational imaging modality that is advantageously capable of producing chemical maps in dynamic living systems based on intrinsic molecular vibrations. This capability enables direct visualization of complex biological processes (without perturbation) as can be applied to a wide range of biomedical applications, including, e.g., tracking voltage spiking during neuron firing, identifying the cancer margin of fresh, unprocessed tissues, and discovering biomarkers and therapeutic targets of aggressive cancers. When evaluating an SRS system, speed, spectral bandwidth, and signal-to-noise ratio (SNR) are the three major aspects, which together characterize the temporal resolution, chemical specificity, and reliability of the system.

In SRS microscopy, a sample is coherently excited by photons from two lasers: one is pump photon with the frequency of ωPand the other is a Stokes photon with the frequency of ωS. When the difference in frequency bet wen pump and stokes photons Δω=ωP−ωSis equal to a particular Raman-active molecular vibration of the sample, SRS signals equivalent to changes in the intensity of the pump and Stokes beams (including both stimulated Raman loss (SRL) and stimulated Raman gain (SRG)) are generated due to the nonlinear interaction between the photons and the molecules. SRS imaging is free from the nonresonant background, exhibits identical spectrum as spontaneous Raman, and is linearly proportional to the concentration of the analyte, thereby allowing for quantification thereof.

In the original implementation of SRS microscopy, lasers with narrow spectral bandwidth were used to excite a single Raman-active vibrational mode for fast imaging with high spectral resolution. Single-color (i.e., non-stereoscopic) SRS is advantageously fast having reached near real-time speeds similar to video-rates. However, with a single-color SRS excitation scheme, other vibrational modes of the sample are not excited, thus failing to utilize the full advantage of the rich Raman spectroscopic information and making it impossible to distinguish mixed chemical species with overlapped Raman bands in the sample. For quantitative analysis, such as the concentration ratios between different chemical compounds, multicolor imaging with multiple chemical contrasts is highly desirable. Furthermore, simultaneous mapping of different chemical species in the same sample is extremely important for the investigation of co-distribution or dynamic correlation between pairs of biomolecules in many in vivo biological and biomedical applications. In addition, since the cross-phase modulation signal and two-color two-photon absorption (TPA) from pigments or blood may exist as a global non-Raman background in SRS, additional contrast away from any Raman resonance may be required to remove the background for acute quantitative investigations.

Spectroscopic SRS was developed to acquire a Raman spectrum at each pixel, enabling the simultaneous study of chemicals with overlapping Raman bands in complex biological samples. Spectroscopic SRS has been achieved in several ways, including:Spectral scanning of a narrowband laser such as disclosed in Ozeki, Y. et al. High-speed molecular spectral imaging of tissue with stimulated Raman scattering.Nat. Photonics6, 845-851 (2012) and Zhang, D. et al. Quantitative vibrational imaging by hyperspectral stimulated Raman scattering microscopy and multivariate curve resolution analysis.Anal. Chem.85, 98-106 (2013);Parallel detection of a complete spectrum by a detector array such as disclosed in Liao, C.-S. et al. Microsecond scale vibrational spectroscopic imaging by multiplex stimulated Raman scattering microscopy.Light Sci. Appl.4, e265 (2015); andSpectral focusing via temporal scanning of linearly chirped pulses such as disclosed in Liao, C.-S. et al. Stimulated Raman spectroscopic imaging by microsecond delay-line tuning.Optica3, 1377-1380 (2016) (“Liao”). Spectral focusing is further depicted and described with respect toFIG. 4which depicts linearly chirping pump and Stokes pulses and focus their entire bandwidth into a narrow spectral region. In this way, each temporal delay Δt between the chirped pulses corresponds to a Raman shift ω.

Despite major advances in instrumentation that push the speed and the spectral bandwidth, most SRS applications are focused on the carbon-hydrogen (C—H) stretching region (2800-3100 cm−1) where strong Raman bands reside. However, the highly crowded SRS signals in the C—H region severely limit the chemical specificity of SRS in a complex biological environment. Thus, recent SRS applications have explored the so-called fingerprint region (500-1800 cm−1). Vibrational spectra in the fingerprint region display a higher degree of heterogeneity, which is important for differentiation of multiple species. Fingerprint region SRS can therefore significantly enhance chemical specificity by providing a specific and well-separated Raman spectrum for each biochemical component. However, the intensity of Raman signals in the fingerprint region is often lower as compared to the C—H region which can result in a high signal-to-noise ratio (SNR). To maintain high imaging speed, the laser power can be increased (within the damage threshold of the sample) to prevent the signal from being overwhelmed by noise. Thus, it is advantageous to provide a spectral acquisition scheme with high power efficiency. In addition to SNR considerations, Raman peaks for different biochemicals in the in the fingerprint region may be narrow and close to each other. Thus, it is further advantageous to provide a spectral acquisition scheme with high spectral resolution.

To acquire high-fidelity fingerprint SRS spectra at the microsecond level, a high-speed spectral acquisition scheme that can achieve both high power efficiency and high spectral resolution is desirable. Spectral acquisition schemes that utilize spectral focusing exhibit high power-efficiency (since all the energy of the pulses is used). They are also able to record a spectrum within a ˜200 cm-1 window at the microsecond level (see Liao). However, existing implementations for high-speed spectral focusing using an edge-reflected resonant mirror (such as disclosed in Liao) typically exhibit a relatively small maximum delay range (2 ps in Liao) with respect to temporal delays between pump and the Stokes beam pulses thereby limiting the degree of chirping and leading to insufficient spectral resolution for the (28 cm−1spectral resolution in Liao). Thus, there exists a need for spectral acquisition schemes that are able to acquire fingerprint SRS spectra at the microsecond level with sufficient spectral resolution for the fingerprint region (e.g., spectral resolution below 10 cm−1).

Due to the physical limits, advances of instrumentation alone are not enough to achieve reliable high-speed fingerprint spectroscopic SRS imaging. The physical limits lead to the trade-offs between speed, spectral bandwidth, and SNR, which can be conveniently expressed as a 3D hyperplane design space (FIG. 1). Various computational methods have been proposed to extend the design space. Matrix completion (Lin, H., Liao, C.-S., Wang, P., Kong, N. & Cheng, J.-X. Spectroscopic stimulated Raman scattering imaging of highly dynamic specimens through matrix completion.Light Sci. Appl.7, (2018) and Soldevila, F., Dong, J., Tajahuerce, E., Gigan, S. & de Aguiar, H. B. Fast compressive Raman bio-imaging via matrix completion.Optica6, 341 (2019)) and compressed sensing (Romberg, J. Imaging via compressive sampling.IEEE Signal Process. Mag.25, 14-20 (2008) and Studer, V. et al. Compressive fluorescence microscopy for biological and hyperspectral imaging.Proc. Natl. Acad. Sci.109, E1679-87 (2012)) methods have been used to sub-sample images to increase speed while avoiding information loss. Denoising algorithms with models on object structures (Dabov, K., Foi, A. & Katkovnik, V. Image denoising by sparse 3D transformation-domain collaborative filtering.IEEE Trans. Image Process.16, 2080-2095 (2007) and Buades, A., Coll, B. & Morel, J. M. A non-local algorithm for image denoising.Proc.—2005IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognition, CVPR2005 II, 60-65 (2005)) have also been proposed to recover the SNR of microscopic images with low light exposure or low pixel dwell times.

Most of these conventional computational methods depend on the formulation of forward models to describe the underlying imaging process, such as the modulation of measurements by a mask, the blurring of the image by the optical point-spread function, the thermal and electronic noise of photodetector and the laser shot noise. However, formulating such forward models often requires tedious system calibration, and sacrifices by way of simplifications for the sake of computational tractability. Thus, computational methods that can bypass model design and directly learn features of the image to formulate mappings from raw experimental data to reliable results should outperform conventional computational methods such as described above.

Deep learning (Lecun, Y., Bengio, Y. & Hinton, G. Deep learning.Nature521, 436-444 (2015)) is one appealing approach that bypasses model design and has the potential to extend the design space (as depicted inFIG. 1). Given training data of input/output image pairs, a deep neural network can learn nonlinear mappings that find optimal approximate solutions to a variety of complicated inverse problems which would otherwise be challenging to address using conventional analytical methods. Deep learning has been applied in the context of some vibrational imaging applications, such as image restoration of single-color SRS images in the C—H region under low light exposure (Manifold, B., Thomas, E., Francis, A. T., Hill, A. H. & Fu, D. Denoising of stimulated Raman scattering microscopy images via deep learning.Biomed. Opt. Express10, 3860 (2019)) and automated detection of the tumor margin from fresh tissue (Zhang, L. et al. Rapid histology of laryngeal squamous cell carcinoma with deep learning based stimulated Raman scattering microscopy.Theranostics9, 2541-2554 (2019)). However, there exists a need for computational methods that apply deep learning to processing spectroscopic images with both spectral and spatial domains (e.g., as those spectroscopic SRS images).

Needs for better spectral acquisition schemes and better computational methods which enable, inter alia, improved spectroscopic imaging in spectral and spatial domains (including improved spectroscopic SRS imaging of the fingerprint region) are met by the systems and methods disclosed herein.

SUMMARY

In exemplary embodiments, a high-speed delay scanning assembly is disclosed for imaging modalities that utilize varying temporal delays between pulsed interrogation beams. The assembly may advantageously include a fast linear scanner and a stepwise reflective surface (e.g., a stepwise mirror or a blazed grating), where the scanner is configured to repeatedly scan a first pulsed interrogation beam along a scan line across the stepwise reflective surface in a Littrow configuration. This scanning changing a path distance of the first beam (with each step of the stepwise reflective surface) thereby introducing a sequence of varying temporal delays relative to a second pulsed interrogation beam (which may advantageously have a constant beam path). In example embodiments, the scanner may be a polygon scanner, thereby enabling rapid and repeated linear scanning across the stepwise reflective surface. In some embodiments, the assembly may be used for spectroscopic SRS imaging (e.g., where the first pulsed interrogation beam is one of (i) a Stokes beam or (ii) a pump beam). In other embodiments, the assembly may be used for transient absorption spectroscopy (e.g., where the first pulsed interrogation beam is one of (i) a pump beam or (ii) a probe beam). In yet further embodiments, the assembly may be used for Impulsive Stimulated Raman Scattering (ISRS) (e.g., where the first pulsed interrogation beam is one of (i) a Stokes beam or (ii) a pump beam). In some embodiments, the stepwise reflective surface may retroflect the first pulsed interrogation beam back along its original path, whereby the first interrogation beam is combined with the second pulsed interrogation beam and linearly chirped by a high dispersion medium (e.g., high dispersion glass rods) to temporally separate different frequency components prior to sample interrogation. The first and second pulsed interrogation beams may be generated by (i) femtosecond lasers, (ii) supercontinuum lasers, or (iii) broadband lasers. In example embodiments, a maximum delay or delay range may be tunable by rotating the scan line relative to the stepwise reflective surface thereby changing an angle between the scan line and a contour line of the stepwise reflective surface while still maintaining the Littrow configuration. Thus, in some embodiments, the maximum delay or delay range may be selected to match pulse chirping of the first pulsed interrogation beam. Advantageously the stepwise configuration may provide for high spectral linearity.

In further exemplary embodiments, improved techniques are disclosed for image restoration of spectroscopic images with spectral and spatial domains. This in example embodiments, a method for image restoration may include (i) providing an encoder-decoder convolution neural network (CNN) characterized by an architecture that includes convolutions involving both spectral and spatial domains; (ii) training the CNN with a pairs of raw and ground truth spectroscopic images which include both spectral and spatial information, wherein training determines parameters for the CNN that minimize a loss function between a CNN predictions and corresponding ground truth; and (ii) applying the trained CNN to improve signal to noise in raw images. In some embodiments, the CNN may be a 3D U-net network (e.g., that includes a 3×3×3 convolution filter). In other embodiments, the CNN may be a spatial-spectral residual net (SS-ResNet) that includes two parallel filters including a first convolution filter on the spatial domain and a second convolution filter on the spectral domain. Thus, for example, the CNN may include a parallel convolution sublayer which includes a first convolution filter (with a (1,3,3) kernel size) for the spatial domain and a second convolution filter (with a (3,1,1,) kernel size) for the spectral domain. In example embodiments, image restoration may also include applying pixel-wise LASSO unmixing to suppress the crosstalk between different chemical maps (such unmixing may advantageously be based upon a principle that at each spatial location, only a few chemical components have dominant contributions). Advantageously, the disclosed image restoration techniques may be applied with respect to spectroscopic SRS images (including those in the fingerprint domain).

In further exemplary embodiments, a fast spectroscopic SRS imaging system is disclosed. The system may advantageously include both a high speed scanning assembly (including a fast linear scanner and a reflective blazed grating, wherein the scanner is configured to repeatedly scan a first pulsed interrogation beam along a scan line across the blazed grating in a Littrow configuration thereby continuously changing a path distance of the first beam and introducing a substantially continuous sequence of varying temporal delays relative to a second pulsed interrogation beam) and an image processor (configured to restore raw images based on application of a trained encoder-decoder convolution neural network (CNN) characterized by an architecture that includes convolutions involving both spectral and spatial domains).

Any two or more of the features described in this specification, including in this summary section, may be combined to form implementations not specifically described in this specification.

At least part of the techniques described in this specification may be configured or controlled by executing, on one or more processing devices, instructions that are stored on one or more non-transitory machine-readable storage media. Examples of non-transitory machine-readable storage media include read-only memory, an optical disk drive, memory disk drive, and random-access memory. At least part of the techniques described in this specification may be configured or controlled using a computing system comprised of one or more processing devices and memory storing instructions that are executable by the one or more processing devices to perform various control operations including high-current testing. The devices, systems, and/or components described herein may be configured, for example through design, construction, arrangement, placement, programming, operation, activation, deactivation, and/or control.

The details of one or more implementations are set forth in the accompanying drawings and the following description. Other features and advantages will be apparent from the description and drawings, and from the claims.

