The individual relative weight of SaO2-retinal and SvO2-retinal to obtain the measured SO2-transcranial was calculated for all subjects.Paired retinal and cerebral oxygen saturation measurements were performed in healthy volunteers. Arterial and venous retinal oxygen saturation and diameter were measured using a non-invasive spectrophotometric retinal oximeter. Cerebral oxygen saturation was measured using near-infrared spectroscopy. Correlations between SO2-transcranial was positively correlated with both SaO2-retinal and SvO2-retinal  and negatively correlated with retinal venous diameter . Estimated SO2-transcranial based on retinal oximetry showed a tolerance interval of (-13.70 to 14.72) and CCC of 0.46  with measured SO2-transcranial. The average relative weights of SaO2-retinal and SvO2-retinal to obtain SO2-transcranial were 0.31 ± 0.11 and 0.69 ± 0.11, respectively.Twenty-one healthy individuals aged 26.4 ± 2.2 years were analyzed. SOThis is the first study to show the correlation between retinal and cerebral oxygen saturation, measured by NIRS and retinal oximetry. The average relative weight of arterial and venous retinal oxygen saturation to obtain the measured transcranial oxygen saturation as measured by NIRS, approximates the established arterial:venous ratio of 30:70 closely, but shows substantial inter-individual variation. These findings provide a proof of concept for the role of retinal oximetry in evaluating cerebral oxygenation. The brain is highly dependent on aerobic metabolism to meet its energy demand.  Prolonge® transcranial oximeter  the relative contribution of arterial and venous blood  to the measured composite value is 30:70, but the inter-individual variability of this ratio is considerable. [Transcranial near-infrared spectroscopy (NIRS) is increasingly used in clinical practice to measure cerebral saturation non-invasively.  NIRS relderable. –20 This derable. –23These shortcomings could partially be addressed by looking at the central nervous system oxygen saturation from a different point of view. Thanks to the transparency of the ocular media, the human eye offers a convenient way to study the circulatory system. In recent years, several new diagnostic tools have been developed to investigate the retinal circulation. Dual-wavelength spectrophotometric retinal oximetry is an imaging technique that allows the visualization and measurement oxygen saturation in retinal blood vessels by comparing the relative reflectance of retinal blood vessels at 570nm and 600nm.  At 570nm2-retinal) and venous oxygen saturation  measured by spectrophotometric retinal oximetry and cerebral oxygen saturation  measured by NIRS.This pilot study aims to provide a proof of concept for the use of retinal oximetry in assessing cerebral oxygen saturation by investigating the correlation between retinal arterial .® technology, CAS Medical Systems, Branford, CT, USA). Heart rate, pulse oxygen saturation (SO2-pulse), systolic and diastolic blood pressure were measured. SO2-transcranial was recorded once the pupil was dilated to a diameter of at least 5mm and continuously monitored during 10 minutes. Following two minutes of dark-adaptation, retinal fundus images were acquired using a Oxymap® T1 oximeter . The experimental setup is summarized in Subjects were asked to refrain from caffeine, nicotine and physical exercise for 12h before testing. After instillation of phenylephrine 5%  and Tropicamide 0,5%  in the left eye, a standard protocol, that has shown to result in good pupil dilation  without influencing retinal oxygen saturation values, was performed.  Particip® Analyzer software version 2.4.0 using the following protocol: first-degree retinal vessels with a width of more than six pixels and a length between 50 and 200 pixels were selected from the color-coded saturation map. Second-degree vessels were only analyzed if first-degree vessels did not meet the criteria. Around the optic disc, an area of 15 pixels as well as branching vessels and their origin were manually excluded. Vessels were manually categorized as arteries or veins and subsequently averaged over the four quadrants. Vessel diameters were measured automatically using the Oxymap® Analyzer software. [The quality of the oximetry images was evaluated using a standardized protocol. Images were subjectively rated on focus, contrast, glare and shadow using a pass/fail system. Failure to pass any of these criteria led to exclusion of the image.  Image anoftware. 2-transcranial was calculated using the established relative arterial and venous weight of the FORE-SIGHT® transcranial oximeter of 30:70 using i) and venous (1 − wi) relative weight to obtain measured SO2-transcranial was calculated in all subjects using i with a SaO2-retinal of 95%, a SvO2-retinal of 60% and a SO2-transcranial of 70%, the arterial (wi) and venous weight (1 − wi) would be 0.29 and 0.71 respectively, since 0.29 x 95% + 0.71 x 60% = 70%.Estimated SO2-transcranial were estimated by substituting SO2-pulse for arterial and by estimating venous oxygen saturation  using The arterial and venous oxygen saturation contributing to the measured SO2-transcranial and estimated SO2-transcranial was evaluated using Lin’s concordance correlation coefficient (CCC) and Bland-Altman analysis. [2-transcranial and estimated SO2-transcranial, its 95% confidence intervals and tolerance intervals (for 99% of future measurements to be within 95% confidence) were calculated. Statistical significance was based on two-sided p-values of <0.05. As this study was set up as a pilot study, power analysis and sample-size calculations were not performed. Individual data points and the R script that was used to generate the statistical analyses and figures can were uploaded as supporting information.  using the “BlandAltmanLeh”, “epiR” and “Tolerance” package. –49 Normanalysis. ,51 In termation. , S2 FileA total of 22 healthy individuals, 6 men and 16 women, were initially recruited. One subject was excluded because of inadequate quality of the oximetry images. Descriptive statistics of demographic characteristics of the included subjects are summarized in 2-pulse, SaO2-retinal, SvO2-retinal and SO2-transcranial were 98.2 ± 1.1%, 95.3 ± 2.1%, 61.6 ± 4.0% and 72.2 ± 3.5%, respectively. SO2-transcranial showed a significant positive correlation with both SaO2-retinal and SvO2-retinal  and a negative correlation with venous diameter . (Average SO 0.017).  Correlat2-transcranial based on the fixed arterial:venous ratio of 30:70 showed a CCC of 0.46  with the measured SO2-transcranial. Bias between measured and estimated SO2-transcranial was 0.51 with a 95% confidence and 95% tolerance interval of -1.07 to 2.10 and -13.70 to 14.72 respectively. Bland-Altman analysis is shown in The estimated SO2-retinal and SvO2-retinal to obtain the measured SO2-transcranial were 0.31 ± 0.11 and 0.69 ± 0.11, respectively, with large inter-individual variability ranging from 10 to 49 for arterial weight, as summarized in The average relative weights of SaO2-retinal correlates well with mixed venous oxygen saturation. [The human retina provides a unique view on the cerebral vasculature. Many studies have pointed out structural and functional homologies between the retinal and the cerebral circulation. –45 Strucuration.  Mixed ve2-retinal and SvO2-retinal, measured by retinal oximetry, are positively correlated with SO2-transcranial, measured by NIRS. The analysis per vessel type revealed a significant correlation between measured and estimated SO2-transcranial. Where previous studies have demonstrated a significant correlation between SaO2-retinal and SO2-pulse, our study could not reproduce this finding. [2-pulse, which has been reported also in previous studies and is thought to result from a counter-current effect happening in the narrow space between the central retinal artery and vein inside the optic nerve. [2-retinal showed a borderline significance with estimated SvO2-transcranial. The above mentioned findings strengthen the hypothesis that the structural similarities between retinal and cerebral circulation also extend to a functional level. Apart from a common embryonic origin, the retinal and cerebral circulation have a common vascularization pattern during development, a non-fenestrated endothelial barrier, are covered by pericytes and glial cells and share the same control mechanisms. [First, this study shows that both SaOfinding.  Of interc nerve.  SvO2-rethanisms. ,45,57 Nohanisms.  Local auhanisms. , 60 Thes2-retinal and SvO2-retinal to obtain the measured SO2-transcranial closely matches the established arterial:venous ratio of 30:70 calibration of FORE-SIGHT® NIRS technology. [2-retinal and SvO2-retinal to SO2-transcranial showed large inter-individual variation ranging from 10% to 49% for the arterial and from 51% to 90% for the venous contribution. These variations are in line with studies that investigated the relative contribution of arterial and venous cerebral oxygen saturation to SO2-transcranial. [2-transcranial and estimated SO2-transcranial could therefore imply that retinal oximetry, being able to measure arterial and venous saturation separately, measures some additional information that is averaged out by NIRS.Second, we found that the average relative partition of SaOhnology. ,20 Sincecranial. ,20 The r2-transcranial. As explained above, the retinal and cerebral circulation share similar vascular regulatory mechanisms. As such, a possible explanation for these findings could be that dilated retinal veins are indicative of cerebral venodilatation which would in turn result in a higher relative contribution of the venous compartment in SO2-transcranial. [Third, and of particular interest, is the negative correlation between retinal venous diameter and SOcranial. ,44,61 Mocranial. ,63 Becaucranial. , 65 CombTaken together, we believe these findings provide proof of concept for the potential added value of retinal oximetry in assessing cerebral oxygen saturation. The ability of retinal oximetry to easily measure arterial and venous oxygen saturation separately, without "contamination" of superficial peripheral circulation may be especially relevant in situations where the relative contribution of arterial and venous saturation might deviate from the established arterial:venous ratio, for instance during extracorporeal circulation or in the post-resuscitation setting, when the accuracy of NIRS could be limited due to therapeutic hypothermia, vasopressor medication, alterations in (auto)regulation of the cerebral and superficial circulation,… ,66Current retinal oximeters have several limitations that inhibit the use in the settings mentioned above. First, retinal oximetry provides a snapshot in time but not a continuous measurement. Next, the size of the current retinal oximeters limits its bedside usability. In addition, the price is steep. However, recently developed hyperspectral image sensors at the size of a regular color image sensor will likely provide an answer to both issues.  By repla2-transcranial is measured continuously, retinal oximetry only provides a snapshot of the saturation in the retinal vessel at one particular moment. We tried to accommodate for this potential source of error by measuring the retinal oxygen saturation in steady state conditions after at least 20 minutes of adaptation in a seated position. However, this limitation might hamper future studies in non-steady state conditions. Last, we recognize the small sample size of the study. However, we believe this pilot study does provide a proof of concept for the relevant role of retinal oximetry in estimating cerebral oxygen saturation.The present study must be interpreted within the context of its potential limitations. First, the lack of exact measurements of the true cerebral saturation precludes firm conclusions about the correlation between retinal and cerebral oxygen saturation. While we believe several of our findings point towards this correlation, it remains to be investigated in future studies. Second, a major limitation of this study is the facts that measurements were performed only under normoxic conditions with an oxygen concentration of 21%. Future studies measuring desaturation data during graded hypoxia are warranted. Third, whereas SOIn conclusion, this is the first study to describe the relation between retinal and cerebral oxygen saturation measured by retinal oximetry and NIRS. The ability of retinal oximetry to measure arterial and venous oxygen saturation separately without "contamination" of superficial peripheral circulation may have future applications in clinical practice. Further studies are warranted to investigate the exact relation of retinal oximetry with cerebral and systemic oxygen saturation.S1 FileIndividual data points for all subjects.(CSV)Click here for additional data file.S2 FileR script for all analyses and figures included in the paper.(R)Click here for additional data file.


In cases of natural disasters, epidemics or even in dangerous situations like an act of terrorism, battle fields, a shooting or a mountain accident, finding survivors is a challenge. In these kind of situations it is sometimes critical to know if a person has vital signs or not, without the need to be in contact with the victim, thus avoiding jeopardizing the lives of the rescue workers. In this work, we propose the use of video magnification techniques to detect small movements in human bodies due to breathing that are invisible to the naked eye. Two different video magnification techniques, intensity-based and phase-based, were tested. The utility of these techniques to detect people who are alive but injured in risk situations was verified by simulating a scene with three people involved in an accident. Several factors such as camera stability, distance to the object, light conditions, magnification factor or computing time were analyzed. The results obtained were quite positive for both techniques, intensity-based method proving more adequate if the interest is in almost instant results whereas the phase-based method is more appropriate if processing time is not so relevant but the degree of magnification without excessive image noise. In recent years, there have been many advances in computer science that have made possible the creation of novel systems with application in different fields such as medicine, archeology, industry or video games, inter alia. For example, in forensic medicine computer vision and close-range photogrammetry procedures allow to reconstruct the scene of a crime. In this context, some 3D image reconstruction systems have been recently developed for specific use in forensic medicine. They allow the non-contamination of the crime-secene, the reduction of data collection time, and the lack of contact with the victims. A system composed of a 3D laser scanner, a profilometer and a low-cost digital camera for crime scene documentation was proposed in . Its devThe introduction of artificial intelligence (AI) also constitutes a major advance in technologies applied to emergency response and forensic medicine. In , an artiAnother important contribution of the technology in medicine, including AI, is the development of solutions to help people with disabilities. There are a large number of works based on communication technologies, computing, digital electronics and signal processing for human vital sign monitoring, including their detection or measurement remotely . The mosIn the signal processing field there are also some valuable studies. For instance, a system combining radar applications with advanced signal processing algorithms for the detection, location in search and rescue of trapped victims in dangerous environments . With a There are other works exclusively based on images remotely measured that look for changes in pixel values. For instance, a system that analyzes color variations in the skin tone was proposed in . An analIn this work we propose a new approach to detect alive people in hazardous environments which has two main characteristics: i) No physical contact with the injured people is necessary, avoiding unnecessary risk to doctors, police or rescue workers, ii) the only measuring device required is a low cost video-camera.Our goal is to look for a fast and cheap method based on signal processing to reveal vital signs or imperceptible movements to the naked eye in injured people, by just processing a video of the scene. To reveal subtle temporary body movements due to breathing, a non-professional video camera or even the camera of a mobile phone can be used. Motion analysis is carried out using two video image processing techniques: one called Eulerian Linear Video Magnification (ELVM) , and othThe idea behind both methods is the combination of spatial and temporal filtering. First, a multi-scale signal representation of the images, named pyramid  is carried out. Pyramids are constructed using spatial filters that decompose the frames into different spatial frequency bands in order to obtain, from an input image, another image with more appropriate features for a specific application. Gaussian pyramids are suitable for color magnification while Laplacian and Steerable pyramids are more appropriate for motion magnification. Second, a temporal filtering is applied to the elements of the pyramid showing the motion at specific temporal frequencies selected by the user. This is done in a uniform way for all spatial levels (or bands) and for all pixels within each level.α, specified by the user, which amplifies the original motion in order to let us appreciate those movements that are imperceptible for the naked eye. Then, the spatial pyramid with the magnified values is collapsed and added back to the original to obtain the final video with the motion exaggerated. These two procedures do not explicitly estimate displacements, but rather exaggerate the motion to reveal hidden information. One of the main differences between ELVM and PBVM is the type of pyramid used. While ELVM computes a full Laplacian pyramid, PBVM computes the phase variations of a complex-valued Steerable pyramid over time. In both cases the magnification is applied over each level in the pyramid, and not over the original images. This is due to the fact that the goal is to magnify the levels of the pyramid that contain the frequencies of the movement. Finally, the resulting signal is multiplied by a magnification factor A more detailed explanation of these techniques is provided below.f is the highest frequency contained in the signal and fs is the sampling rate (how often samples are taken per unit of time).The first step of the proposed method is, of course, to take a video of the scene. The video is recorded with a video camera from a stable position, in order to avoid undesirable noise due to the camera movements. Thus, only movements of the object scene are recorded. In addition, if the area around the victim is considered unsafe for the emergency staff (e.g. rescue workers in a terrorist attack), the camera can be placed on a vehicle or a robot that should be stopped at the time of recording the video. In order to achieve the best results, it is advisable to use an FHD, or even better a 4K, video camera. Furthermore, it is convenient to record images as clear as possible and the camera frame rate must satisfy the Nyquist-Shannon theorem:Before the development of the Eulerian methods, the most common technique for motion exaggeration was the Lagrangian approach. This method focuses on individual points and analyzes their change in location over time. Therefore, it performs a tracking of points over time, so it requires a unique matching method between points or patches. For this reason, Lagragian methods are expensive in terms of computation. In contrast, the goal of the Eulerian approach is to study a certain characteristic at pixel level, such as the intensity, over time. Thus, Lagragian methods estimate where a given point moves to, while Eulerian methods measure flux properties of the scene.One of the most notable characteristics of the Eulerian approach is that it is based on local pixel-level operations in an efficient computation way, avoiding the need for solving optimization problems, in contrast to the Lagragian methods based on dense optical flow estimation or matching sparse feature points. As a drawback, both are limited to small motions between frames.According to , 17, in I represents a one dimensional image intensity profile that is translated over time. If I = f(x), then the intensity at instant t can be expressed as I = f(x + δ(t)), where δ(t) represents the translation of the signal over time. As the goal is to magnify the movement in an amount α, named “magnification factor”, which is selected by the user, obtaining a synthesized signal From a mathematical point of view, let us assume that Applying a first-order Taylor expansion to the previous signal, a relatively simple way of estimating the magnified signal is obtainedI, (1 + α) times the result of applying a temporal bandpass filter to the signal at each instant of time, represented by α given by:It consists in adding to the original signal rding to , there iInitially, a representative wavelength , λ, of the lowest band in the Laplacian pyramid, which reveals small motions, can be tried. This wavelength λ must provide information about the dimensions of the image at that level. Then, an estimation of this parameter can be obtained by calculating the diagonal length of the image .A Laplacian pyramid is a multi-scale signal representation of an image which is based on recursively applying a lowpass filter and a reduction in size to obtain a series of images with less detail and smaller dimensions. To build it, the difference between the downsampled and the original images at each dimension must be computed. If this pyramid decomposition is applied over an image and then rebuild it, the original image without loss (except for possible rounding errors) is obtained. When the original image is successively smoothed using a Gaussian filter (blur), and scaled down, a Gaussian pyramid is constructed. Accordingly, a Gaussian pyramid is a sequence of low-pass, down-sampled images. A Laplacian pyramid is similar to a Gaussian pyramid; in fact, each image at certain level of the Laplacian pyramid is the difference between two corresponding adjacent levels of the Gaussian pyramid. Only the smallest level is not a difference of two images, allowing in this way to reconstruct the original image from the difference images. Then, a Laplacian pyramid can be considered a sequence of band-pass, down-sampled images. fh and fl, where  is the interval that contains the frequencies of motion to magnify. Butterworth filters are characterized by having a magnitude response that is maximally flat in the passband and monotonic overall.Once the pyramid for each frame has been constructed, a temporal filter is applied to the temporal series of pixels in each spatial band of the pyramid, in order to extract the frequency bands of interest. The temporal filtering approximates spatial translation of the signal according to α is applied to each of them. Several values of α can be tested but, in order to improve the results, it is advisable to use a constant value for the spatial bands that are within the limits given by Once each level of the pyramids has been temporary filtered, the amplification factor The intensity-based video magnification approach has a notable drawback: the noise increases linearly with the amplification factor, so it supports small amplification factor values. In order to avoid this inconvenience, the phase-based video motion magnification, that is based on the Fourier transform, was developed . It is tPhase-based video motion magnification allows separating the amplitude and phase on each spatial scale and orientation of the pyramid, once the intensity of the frames has been decomposed into Fourier series, assuming that the video is mostly static. According to the Fourier shift theorem it is possible to relate a phase shift in the frequency domain with a translation into the spatial domain, so it is possible to associate phase differences with motion.The local phases over time at every spatial scale and orientation of the Steerable pyramid are computed for all the frames, and their differences with the corresponding values for the first frame calculated. Then, a temporal bandpass filter is applied to those differences in order to select the frequency range of interest and remove any temporal DC component (the average luminance of the frame). Then, as in EVLM, a magnification parameter is applied to the phase shifts, which allows to make small movements of the objects visible. Finally, the pyramids containing the amplified signals are reconstructed and added back to the original input to construct the magnified video.For the specific application proposed in this work, the detection of breathing movements, most of the videos have oscillatory movements without a long duration and small amplitudes. However, this method can be used to amplify non-periodic motions as well, as long as they are within the passband of the temporal filter.As mentioned before, the first step in PBVM is the computation of a 2D Discrete Fourier Transform (DFT) over all frames in the video, and the subsequent application of the spatial filters, with different size and orientation. This produces a linear multi-scale and multi-orientation image decomposition named Steerable pyramid –22. EachI(x + δ(t)) is decomposed into a Fourier seriesω corresponds to each band of the pyramidω(x + δ(t)) contains motion information.Following the notation in The procedure to create a Steerable pyramid is shown in ω(x + δ(t)), after applying the mentioned bandpass filterOnce the pyramid has been built, the phase on each spatial scale and orientation separated, and their differences computed, a temporal bandpass filtering is applied, looking for the isolation of the motion in the specific temporal frequencies to be magnified. So, if the phase in  is ω times higher than the original.Once the temporary filter has been applied, the result is multiplied by an amplification factor α value, it is possible to make perceptible different movements in the input video, although there are some limitations in the degree of magnification. As α increases, the output video has more noise and artifacts. In fact, as with the ELVM, there is an upper limit for α when an octave-bandwidth (4 orientations) Steerable pyramid is used (see [Changing the sed (see ):αδ(t)<α is given by:When a half-octave (8 orientations) Steerable pyramid instead of a octave-bandwidth one is used, the threshold for g to Eqs  and 7),,α is givα times magnified pyramid levels. This is performed by the inverse process of the steerable pyramid creation. In practice all the operations are performed as component-wise multiplications in the Fourier domain.The final step of the video magnification procedure consists of reconstructing the video from the Once all levels of each frame have been reconstructed, they are summed. Subsequently, the lowpass residuals of each frame that has the lowest resolution are added. Finally, adding this result to the original signal, the output video is obtained. It should bee taking into account that throughout the process several bandpass filters have been applied, consequently, it is necessary to add a high-pass residue image to reconstruct the final image.In order to verify that the proposed methodology works properly, an accident such as a gas leak or a terrorist attack, with three injured people involved, was simulated. Victims are unconscious at first glance, and the video magnification procedure explained before was applied to make small movements produced by breathing perceptible. The three people who appear in the video are co-authors of the article, and all of them have given their verbal consent to carry out the simulation. They are all professors at the University of Oviedo, from where they were recruited at a meeting held on May 2, 2017. The simulation was submitted to the Ethical Committee of the Principality of Asturias (CEICR), which evaluated it and provided the appropriate authorization to carry it out.A video camera (Panasonic Lumix DMC-TZ80) which has a 30x optical zoom and HD resolution, mounted on a tripod located several meters away from the people, was used. After setting the camera on movie mode, volunteers were asked to lie down in the three different positions in which a victim can be normally found: lying on their back, face-down and sideways.Movies were recorded filming the whole scene without additional optical lenses. The length of the video was long enough to capture the movement to be magnified. Normally a video of a couple of minutes or even less was sufficient. Daylight was used as the illumination source in combination with normal artificial fluorescent light to improve the image sharpness. The movie was recorded in color with 50 frames per second (fps) and 1920 × 1080 pixel resolution. As an adult takes 12-18 breaths per minute or 0.2-0.3 breaths per second, the sampling rate requirements are fulfilled. It was saved in MP4 format and transferred to a PC to be processed using Matlab R2015b software.α increased. In order to increase the magnification without increasing the noise, the PBVM algorithm was tested. Visually, the best result was obtained for a magnification factor of around α = 50. For those values satisfying α ≤ 10, the motion magnification was almost imperceptible, and for α ≥ 50 motion magnification was too big, originating a blurry video. The difference between the results using both procedures, with the same amplification factor α = 50, is shown in After performing many tests with values of the magnification factor between 5 and 60 in increments of 5, it was found that ELVM with small values of the magnification factor produces an almost imperceptible magnification, and that the quality of the image obtained by this algorithm gets worse as α using PBVM method, although the edges of the moving bodies appear somewhat distorted. Nevertheless, color changes of the entire image and noise are almost imperceptible, and no significant noise is added to the scene. The original video, and the two magnified videos corresponding to https://figshare.com/articles/Video_magnification_to_detect_human_vital_signs/5769840).Although in the ELVM approach the magnification was not large enough to be appreciated in In order to quantify the capacity of both approaches, ELVM and PBVM, to increase the intensity signal, a set of pixels of a region where the movement is more evident  were selected, and the intensity in the original video, as well as in the output magnified videos, were obtained. Results are shown in IO and IM are the intensities in the original and in the final video, respectively; MSE being the mean square error, given by:The increasing of the pixel intensity due to magnification is normally associated with a decrease of the quality of the video. To check this, the quality of both output videos by means of the Peak Signal to Noise Ratio (PSNR) was estimated:For an image in RGB format, the MSE is calculated as the arithmetic mean of the MSEs for each red, blue and green color component.α value on the performance of an algorithm. The PSNR is dimensionless, as both numerator and denominator represent pixel values. However, it is usually expressed in decibels (dB), although it is less sensitive to changes in the MSE. As it has no absolute meaning, it is meaningless to say that a determined value of PSNR is good or not. It is only a way of comparing the effects of the The simulation explained in the previous section was repeated in different situations concerning light conditions in the room, distance to the objects, camera movement and camera frame rate, in order to determine the best for video magnification. Below, is discussed how each of these aspects affects the results of video magnification.Since the main idea of this article is to magnify small movements that are not visible to the naked eye, it is crucial to ensure that the camera is not subject to any vibration or sudden movement, such as those produced by pressing the record button. If these movements have a frequency within the range that the user has chosen, they will also be magnified. Moreover, camera displacements modifying the framing result in unacceptable magnified videos. Therefore, it is very important to place the camera on a stable element, preventing inappropriate movements at the time of video recording. In this sense, it is advisable to operate the camera remotely. In fact, an experiment was conducted filming with a camera mounted on a UAV , and even though the video was fairly stable it was not possible to magnify the motion due to breathing without magnifying the small movements of the camera. The results did not improve even using a video stabilization algorithm. As a result, the magnified videos showed distorted frames that do not allow to perceive breathing movements.Similarly, lighting conditions is another factor to be considered. Suitable conditions allows to obtain images with greater sharpness. For this reason, it is advisable to record the video in good environmental conditions, minimizing the existence of shadows. However, movements due to breathing were also detected in scenarios with poor lighting conditions. This suggest that the propposed method is valid in many real situations.Regarding the distance of the camera to the objects, it is advisable to locate the camera close enough to allow detecting the victims. For a camera with HD image resolution, the camera can be placed up to several tens of meters away. Obviously, this distance increases with the video resolution. In our experiment, distances from 3 to 10 meters were tested, and it was possible to detect breathing movements even at the largest distance.Regarding the speed of video recording, it should be enough to record the frequencies of vibration according to the Nyquist criterion. Then, low-cost video cameras of 20 fps are suitable for our particular problem.Concerning the magnification factor, although it is a critical aspect to take into consideration, it is also true that it may vary over a fairly wide range of values without producing noticeable effects. Our recommendation is to try several values in steps of 5 to 10 units.According to the tests performed, it can be said that the average value of the PSNR is higher using the intensity linearization procedure than the phase-based procedure. This means that the first method produces an output magnified video with less noise. However, this is only valid for small magnification factors, since the quality of the output magnified video decreases with the value of the magnification.In contrast, attending to the processing time, the phase-based method is considerably slower than that based on intensity linearization. Therefore, if obtainig results in almost real time is not priority, the phase-based method should be employed. Otherwise, it is better to employ the linearization method, although videos of worse quality will be obtained using the same magnification, making more difficult to identify signs of life. In this context, ELVM provided adequate results in just a couple of minutes when the original video was resampled to half the original cell size. This time could be reduced if the frames are cropped to select just the area containing the people under study, since this reduces the image extension. On the other hand, the PBVM algorithm took about 15 minutes in the same conditions, although, as mentioned, the magnified video showed the movement due to breathing much more clearly.In a standard video, hidden signals with small amplitude may exist. Often, these signals cannot be detected with the naked eye because of its limited sensitivity. We propose to use video magnification techniques to detect slow body movements due to breathing, as a method to identify injured people in accident or violent scenarios. Two different video magnification procedures were tested in a simulated scenario with injured people. Both methods, one based on a linear decomposition of the pixel intensity, and the other based on amplifying phased variations of the coefficients of a Steerable pyramid, proved to be useful. However, although the first method is simple and fast, the noise increases linearly with the amplification factor, so it only supports small values of this factor. The second methods provided the best results in terms of magnification because it supports a larger amplification factor, although it requires a considerably longer runtime.Obviously, the processing time depends on the length of the video, which must be long enough to capture a representative sample of the movement to be magnified, showing phenomena occurring at the temporal frequencies selected by the user. In addition, the global processing time of a video increases with the number of levels that form the pyramid, but more decomposition means less noise in the output video.Among the different factors concerning the utility of the video magnification, a fundamental aspect to be considered at the time of recording the video is that the camera has to remain stable during data collection. In addition, light conditions are also a factor to be considered when striving for good results.


Paenibacillus polymyxa WLY78 is a nitrogen fixer and it can be potentially applied to biofertilizer in agriculture. In this study, P. polymyxa WLY78 is labelled with gfp gene. The GFP-labelled P. polymyxa WLY78 is used to inoculate wheat, maize and cucumber seedlings grown in the gnotobiotic system and in soil, respectively. Observation by confocal laser scanning microscope reveals that the GFP-labeled bacterial cells are mainly located on the root surface and epidermis of wheat, and only a few cells are present within cortical cells. In maize and cucumber seedlings, bacterial cells were colonized in epidermal and cortical cells, intercellular spaces and vascular system of root, stem and leaf tissue interiors besides on root surfaces. Higher densities of the bacterial cells in roots, stems and leaves indicated that P. polymyxa WLY78 cells could migrate from roots to stems and leaves of maize and cucumber. This study will provide insight into interaction between P. polymyxa WLY78 and host cells. Thus, the use of chemical nitrogen fertilizers . However, whether P. polymyxa WLY78 can colonize in these plants is not yet known. In this study, P. polymyxa WLY78 was labeled with GFP and then the GFP-labeled bacterium was used to inoculate wheat, maize and cucumber roots. The colonization patterns of P. polymyxa WLY78 in these plants were observed under the confocal laser scanning microscope. Our study will not only reveal the colonization patterns on different plants, but also provide clues for studying the plant-growth promoting traits in P. polymyxa.bstances , 6. JustP. polymyxa WLY78, which was isolated by our laboratory from roots of bamboo, was used here [sed here . The bacP. polymyxa WLY78, a plasmid vector which carried a gfp gene and could replicate in P. polymyxa was here constructed. The pHY300PLK plasmid, which was a shuttle vector in both E. coli and B. subtilis, was used as a vector backbone [gfp (mut3a) gene together with its promoter in a 4.4-kb DNA fragment was obtained from plasmid pGFP4412 by digestion with EcoRI and HindlII [HindIII and EcoRI restriction enzymes, thus yielding the plasmid pGFP300 which carries a gfp (mut3a) gene. The competent cells of P. polymyxa WLY78 were prepared as described [P. polymyxa WLY78 was obtained by transferring pGFP300 into competent P. polymyxa WLY78. The GFP-labelled P. polymyxa WLY78 was observed under Olympus FluoView™ FV1000 confocal microscope.In order to obtain GFP-labelled  HindlII . Then thescribed . GFP-labWheat seeds were surface-sterilized by soaking in 20% sodium hypochlorite solution for 20 minutes, and then they were rinsed repeatedly with sterile water, and finally the seeds were soaked in sterile-distilled water for 4 hours. Then the seeds were placed on the wet filter papers in sterile dishes at room temperature, and after about 10 days later, the root seedlings were approximately 2 centimeter (cm) in length. Similarly, maize and cucumber seeds were surface-sterilized and then germinated.P. polymyxa WLY78 cells (2.4×108 cells/ml) were inoculated to the flask.For the gnotobiotic system, the germinated seedling was transferred to a sterile flask (6 cm in diameter and 10 cm in height) filled with 100 mL 1/2 × Murashige & Skoog (MS) semisolid agar medium . Then 207 cells/g soil. The plants were grown in a light growth chamber at 27℃ with 16 hours of light per day and 70% humidity.For soil system, the germinated seedling was transferred to a flowerpot. Each flowerpot was filled with sterilized soil and the concentration of the GFP-labelled bacterial cells in soil was about 1.74×10P. polymyxa WLY78. In the soil system, the plants were taken out from the soil, rinsed with sterile water and observed directly under the laser confocal. The resulting images were obtained using the FV10-ASW 4.0 Viewer software or Zen 2012 software.In the gnotobiotic system, the plants were taken out from the semisolid agar medium after inoculation for certain times. Plant root surfaces were rinsed with sterile water and then were directly observed under the laser confocal . Some tissues of roots, stems and leaves were sectioned in longitudinal and transverse directions to examine the colonization of GFP-labelled P. polymyxa WLY78 is a nitrogen-fixing bacterium with great potential use as biofertilizer in agriculture. In order to investigate the interaction between the bacterial cells and plants, P. polymyxa WLY78 was labelled with the green fluorescence protein (gfp) gene. As was shown in Materials and Methods, GFP-carrying plasmid pGFP300 which is able to replicate in both B. subtilis and E. coli is here constructed and transformed into P. polymyxa WLY78. Three positive transformants with tetracycline resistance (TcR) are obtained. Colony PCR analysis showed that each of the three transformants carries the gfp gene.P. polymyxa cells emit bright green fluorescence  observation shows that the GFP-labeled rescence . FurtherP. polymyxa WLY78 is very important for monitoring colonization of plants. In order to assess this property of the GFP-labeled P. polymyxa WLY78, the bacterial cells were cultivated in LB with tetracycline (25 μg/μL) to the phase of logarithmic growth, and then they were transferred to the fresh LB medium in the absence of antibiotic pressure with the dilution of 1:100. After three days of incubation in LB medium, the GFP-labeled P. polymyxa WLY78 cells were still able to emit green fluorescence, suggesting that gfp gene was stably expressed in P. polymyxa WLY78.The stability of GFP-carrying plasmid in P. polymyxa WLY78 was used to inoculate wheat seedlings grown in the gnotobiotic system and in soil, respectively. The gnotobiotic model system can be used for studies on the interaction between P. polymyxa and plants, while soil system might reflect the natural pattern of colonization in soil system.The GFP-labelled P. polymyxa cells were found to colonize in the root hair zone and on the surface of the primary roots (The wheat roots at 15 hours and 1, 3, 5, 7, 9, and 11 days after inoculation, respectively, were cut or sectioned from the plants, and then the root sections were examined under a laser confocal microscope. 15 hours after inoculation, the GFP-labelled ry roots . One dayry roots . Howeverry roots . At the ry roots . The bacry roots .P. polymyxa cells expand within the wheat roots. Ten days after inoculation, it was found that P. polymyxa cells were still tightly colonized on the surface of roots (As shown in of roots .P. polymyxa WLY78 to wheat planted in soil was also performed. This system is more similar to the natural environment. The root samples were flushed with water before observation under laser confocal microscope.In this study, inoculation of the GFP-labeled As shown in Compared to that in gnotobiotic system, bacterial population on the roots in the soil system is much lower. We think that the lower density of bacterial cells on the roots in the soil system may be caused by washing roots before microscopy observation.P. polymyxa WLY78 was used to inoculate maize seedlings grown in the gnotobiotic system. One day after inoculation, GFP-labeled bacterial cells were found to be attached to the surface of maize roots (Similarly, the GFP-labelled ze roots . Three dze roots  and the ze roots . Five daze roots . Longituze roots . After oze roots . Moreoveze roots . Ten dayze roots  and stemze roots  were alrze roots . And a lze roots .P. polymyxa WLY78 was also used to inoculate cucumber seedlings grown in the gnotobiotic system. One day after inoculation, a significant proportion of the bacterial population irregularly scattered on the primary root surface (The GFP-labelled  surface  and the  surface . Bacteri surface , but few surface . Three t surface , being c surface . Meanwhi surface , suggest surface . The lon surface  and with surface .P. polymyxa WLY78 was used to inoculate wheat, maize and cucumber seedlings grown in the gnotobiotic system. Observation by confocal laser scanning microscope reveals that in maize and cucumber seedlings, bacterial cells were colonized in epidermal and cortical cells, intercellular spaces and vascular system of root, stem and leaf tissue interiors besides on root surfaces. Higher densities of the bacterial cells in roots, stems and leaves indicated that P. polymyxa WLY78 cells could migrate from roots to stems and leaves of maize and cucumber. However, we found that wheat seedlings grown in both the gnotobiotic system and in soil, the majority of the bacterial cells were distributed on root surface and epidermis, and only a few cells are located within cortical cells in both systems. The results obtained in wheat are a little different from those from maize and cucumber. The result that only a few bacterial cells are located within cortical cells of wheat may be due to the reason that we could not expertly use confocal laser scanning microscope to observe bacterial colonization pattern at the beginning of this study.In this study, the GFP-labelled P. polymyxa WLY78 has the abilities of phosphate solubilization and IAA production besides nitrogen-fixation [P. polymyxa WLY78 inside roots, stems and leaves and on root surfaces are contributive to plant-growth promotion of the bacterium when plants were inoculated with this bacterium.Our recent studies have shown that fixation . We deduP. polymyxa WLY78 is similar to those of the Gram-negative diazotroph Azospirillun brasilense Yu62 [Bacillus megaterium C4 [P. polymyxa WLY78 is an associated nitrogen-fixer. Our data are a little different from Timmusk et al’s report [P. polymyxa B1 colonized predominantly on the root tip. Our data support the previous results that the colonization pattern is closely related to bacterial strains. This study will provide some information for developing nitrogen-fixing cereals [The current colonization pattern of the nse Yu62 , 13 and erium C4 , suggests report  that P.  cereals .


Here we show that stabilization of the L2-L3 region by tertiary interactions, and the ligand binding site by ligand binding, cooperatively influences the structural stability of terminal base pairs in the P1 region in the presence of Mg2+ ions. The results are based on molecular dynamics simulations with an aggregate simulation time of ~10 μs across multiple systems of the unbound state of the Gsw aptamer and a G37A/C61U mutant, and rigidity analyses. The results could explain why the three-way junction is a central structural element also in other riboswitches and how the cooperative effect could become contextual with respect to intracellular Mg2+ concentration. The results suggest that the transmission of allosteric information to P1 can be entropy-dominated.Riboswitches are genetic regulatory elements that control gene expression depending on ligand binding. The guanine-sensing riboswitch (Gsw) binds ligands at a three-way junction formed by paired regions P1, P2, and P3. Loops L2 and L3 cap the P2 and P3 helices and form tertiary interactions. Part of P1 belongs to the switching sequence dictating the fate of the mRNA. Previous studies revealed an intricate relationship between ligand binding and presence of the tertiary interactions, and between ligand binding and influence on the P1 region. However, no information is available on the interplay  Riboswitches are genetic regulatory elements that regulate gene expression on the RNA level. Most riboswitches are found in the 5'-untranslated region of bacterial mRNA ,2. Ribosxpt-pbuX operon of Bacillus subtilis investigated here belongs to the class of purine binding riboswitches, binds guanine and hypoxanthine and acts transcriptionally :[RNA] ratio > 18:1 :[RNA] ratio > 18:1 3+ in crystal structures of Gsw aptamers 3+ ions in addition. Only minor differences were observed when comparing the mean Rg from MD simulations with 12 and 20 Mg2+ ions. The mean Rg in the absence of Mg2+ ions is increased by ~1 Å for both Gswapt and Gswloop. The more compact structure in the presence of Mg2+ ions is maintained due to the ions compensating the negative charges of the RNA backbone. For all three Mg2+ ion concentrations, the differences in mean Rg between Gswapt and Gswloop are only marginal (≤ 0.2 Å), indicating only a minor influence of the mutation on the compactness of the structure.In order to investigate the influence of Mgolecules . In the 2+ ions in the MD simulations results in a less compact structure than when Mg2+ is present and in the ligand-bound crystal structures.In summary, we observe that the G37A/C61U mutation does not have an influence on the structural compactness of the RNA. The absence of Mg2+ ions and the G37A/C61U mutation on the dynamics of the RNA, we calculated atomic root mean-square fluctuations (RMSF) and averaged them per nucleotide  for Gswapt (Gswloop) than in the absence of Mg2+ ions.In all MD simulations, the J2/3 region shows a pronounced mobility in agreement with experiments  and the leotides  that areleotides . The preapt in the absence and presence of 20 Mg2+ ions . These results provide the first evidence at the atomistic level that the G37A/C61U mutation has a long range effect in terms of increasing the structural dynamics at the ligand binding site, as has been inferred from the influence on the ligand binding ability of Gswloop [and the P1 region, which has not been considered so far.In g2+ ions , we obseg2+ ions  are not  Gswloop ,18, and apt but much more marked, we observe a Mg2+-induced decrease in structural dynamics of the aptamer for Gswloop: The Mg2+ ions now decrease the structural dynamics across the entire aptamer domain (2+ ions (2.2 Å). In addition, the L3 loop, the P3 region, and the joining J2/3 region are affected by the Mg2+-induced decrease in structural dynamics. Finally, the differences in RMSF between Gswloop and Gswapt in the presence of 20 Mg2+ ions , apt and Gswloopapo structures. In order to simulate the presence of guanine in the binding site of Gsw, we added additional constraints to the network connecting the nucleotides U47, C74, and U51, which bind guanine in the crystal structure (plrc(i) per nucleotide in the presence of the ligand constraints and, finally, the difference Δplrc(i) = plrc(i)lig − plrc(i)apo apo . The res2+ ions in the MD simulations (plrc(i) for nucleotides in the J1/2, J2/3 and J3/1 regions, which is not surprising. Similarly, nucleotides in P1, P2, and P3 adjacent to the ligand binding site show an increased plrc(i) in the presence of the ligand constraints, including nucleotides in the “P2 tune box” [In the absence of Mgulations , the preune box” . These r2+ ions (plrc(i) of nucleotides in the P2, P3, L2, and L3 regions (|Δplrc (i)|≤1%). In contrast, the presence of the ligand constraints result in a much larger increase in plrc(i) for nucleotides in the J2/3 region (up to 42%) and in the J3/1 region (up to 20%). As for the absence of Mg2+ ions, we again observe an increase in plrc(i) for all nucleotides in the P1 region by the presence of the ligand constraints, even though the values are higher in the absence of Mg2+ ions. Thus, in the presence of 20 Mg2+ ions, the stabilizing effect of ligand constraints is more local and channeled towards the P1 region.In the presence of 20 Mg2+ ions , we obseplrc(i) between Gswapt and Gswloop in the absence of Mg2+ ions (plrc(i) are found. Nucleotides in the binding region show a stronger influence of the ligand constraints in Gswloop. This is likely due to the larger flexibility observed for Gswloop in the binding region in the apo state, which becomes more restricted by the presence of the ligand constraints. In contrast, the presence of the ligand constraints has a slightly larger influence on nucleotides in the P1 region for Gswapt (up to 3%) than for Gswloop. In the presence of 20 Mg2+ ions, the stabilizing effect of the ligand constraints towards the terminal ends in the P1 region is more pronounced for Gswapt, which shows Δplrc(i) values for this region 2 to 7% higher than for Gswloop, with the higher values being closer to the ligand binding site.The detailed comparison of Δg2+ ions  shows thplrc(i) in the presence of ligand constraints for nucleotides at the ligand binding site and in its environment. In Gswapt and when 20 Mg2+ ions are present, the rigidifying effect towards the terminal ends of P1 is largest. Considering that the tertiary loop interactions are already preformed in the unbound state of Gswapt even in the absence of Mg2+ ions [2+ [In summary, we observe an increase in g2+ ions ,32, yet ions [2+ ,62, thisCoop(i) on the stabilization of nucleotides i in the P1 region from respective plrc(i) values according to Coop(i) > 0, interactions at both sites together have a larger effect on nucleotide i than the sum of the effects of the separate interactions.In order to verify that the tertiary loop-loop interactions and the presence of the ligand constraints rigidify the terminal nucleotides in the P1 region cooperatively, we computed the cooperative effect 2+ ions, almost all of the nucleotides in the five terminal base pairs show Coop(i) values that are not significantly different from zero  values that are positive and significantly or even highly significantly different from zero  reduced for both Gswapt and Gswloop, in agreement with experimental results on other riboswitch aptamer domains [Likely the largest challenge arose from modeling the structural dynamics of Gswructures ,18. Thisructures ,31,32 mapo state . Accordiructures , despitees . This measure had been used successfully to analyze stabilizing effects on the binding partners in the context of protein-protein complex formation [plrc(i) in the presence of ligand constraints was found for nucleotides at the ligand binding site and in its environment, the rigidifying effect towards the terminal ends of P1 was largest for Gswapt and when 20 Mg2+ ions were present (Coop(i) then confirmed that stabilization of the L2-L3 region by tertiary interactions, and the ligand binding site by ligand binding, cooperatively influences the structural stability of the terminal base pairs in the P1 region in the presence of Mg2+ ions  is alre[E. coli ,82) such[E. coli .apt and Gswloop. A recent NMR study of the structurally similar adenine-sensing riboswitch uncovered a three-state behavior of the complete riboswitch including the expression platform [In our study, we only investigated the aptamer domain of the two Gsw variants Gswplatform : A liganapt, and the G37A/C61U mutant will be referred to as Gswloop, as introduced by Schwalbe et al. [2+ concentrations  were set up according to experimental findings on the Mg2+ dependence of Gswloop properties [2+ ions [2+ ions were added to the RNA system with a first hydration shell of six water molecules. This was done to prevent an early “sticking” of the Mg2+ ions to the RNA during the equilibration phase. To prevent clashes between the hydration shell and the RNA during the placement step, dummy ions with equal charge but larger radius of 4 Å, which is close to the radius of a hexahydrated Mg2+ ion [leap of Amber 11; these dummy ions were then replaced with a hexahydrated Mg2+ ion [2+ parameters see [+ ions [The setup of the molecular dynamics (MD) simulations in the present study is similar to that in our previous work . Howevere et al. . For botoperties ,18,31. DMg2+ ion , were plMg2+ ion . For detters see . The systers see . The dis [+ ions  were add-1 Å-2 to all solute atoms were applied. In addition, harmonic restraints were applied to the Mg2+ ions and the water molecules in their first hydration shell. Subsequently, isothermal isobaric ensemble (NPT)-MD simulations were used for 50 ps to adjust the solvent density. Finally, the force constants of the harmonic restraints on positions of RNA, Mg2+, and first hydration shell waters were gradually reduced to 1 kcal mol-1 Å-2 during 250 ps of NVT-MD simulations. This was followed by 50 ps of NVT-MD simulations without applying positional restraints. From the following 550 ns of NVT-MD simulations at 300 K performed by the GPU version of pmemd [Each system was then prepared based on a protocol described earlier . In detaof pmemd , conformapt and Gswloop with three Mg2+ concentrations each), three independent MD simulations of 550 ns length each were performed summing up to a total simulation time of ~10 μs.For each of the six system setups (Gswcpptraj [radius of gyration (ROG), a measure of the compactness of a structure, was calculated omitting the P1 region. This was done for conformations generated by MD simulations as well as for the crystal structures. Prior to the calculation of root mean-square atomic fluctuations (RMSF), global translational and rotational differences between the structures along the trajectory need to be removed by least-squares fitting. In order to reduce the influence of very mobile regions on the picture of internal motions [apt in the absence of Mg2+ ions. These nucleotides were also used in the case of Gswloop then. Root mean-square deviations (RMSD) of atomic positions were calculated as a measure for structural similarity with respect to the initial structures. The RMSD was calculated after aligning the conformations onto the initial structure, using those 80% of the nucleotides with the lowest RMSF. The RMSD was then calculated for all atoms of the RNA as well as for all atoms of substructural parts of the aptamer . Block averaging over blocks of 10 ns length was used to determine the statistical uncertainty within one trajectory. The values reported in 2+ ions was determined by “following” the hydration shell around the ions over the simulation. Here, “following” means that we calculated the number of water molecules around each Mg2+ ion within a distance of 3.5 Å. Changes in the number of water molecules in the first hydration shell were subsequently confirmed by visual inspection. Furthermore, the direct chelation of Mg2+ ions to the RNA was determined by calculating the distance of each Mg2+ ion to the RNA. We considered a distance below 3.5 Å between a Mg2+ ion and an RNA atom indicative that the hydration shell of this Mg2+ ion is incomplete. The direct chelation was then confirmed by visual inspection of the trajectory.The MD simulations were analyzed using cpptraj  of the Acpptraj . The ini motions ,96–98, c aptamer . In ordeapt in the presence of 20 Mg2+ ions and Gswloop in the absence of Mg2+, all projections markedly overlap when comparing MD simulations of the same system, and when comparing the first and the second half of the trajectories. For Gswloop in the absence of Mg2+ ions, the projections onto PC1 show a larger spread between the MD simulations; however, the projections onto PC2 and PC3 overlap well. These results demonstrate that the three independent MD simulations per system sample similar regions in PC space, and the sampling agrees between the first and second halves of the trajectories. Additionally, we calculated the RMSD average correlation (RAC) [cpptraj [loop in the absence of Mg2+ ions  PC1/PC2, PC1/PC3, and PC2/PC3. As shown in on (RAC) ,99 as im[cpptraj . The RACg2+ ions , whereasIn order to investigate the influence of the presence of a ligand in the binding site on the rigidity characteristics of the aptamer domain, we applied the FIRST software (version 6.2) . FIRST cet al. [-1 as an energy cutoff Ecut for hydrogen bonds  that atom i belongs to the largest rigid cluster according to n1(i) is the number of occurrences of atom i as part of the largest rigid cluster, determined over all N snapshots extracted from the trajectory [plrc(i) was calculated in the absence (apo) and presence (lig) of a ligand, as was the differenceFrom the rigid cluster decomposition, we calculate the probability ajectory . plrc(i)plrc(i) over the three independent MD simulations for each of the six system setups is shown in The average of Δ2+ ions and their first hydration shell water molecules as proposed by Fulle et al. [2+ ions and their first hydration shell water molecules, or Mg2+ ions and the RNA, were modeled as covalent bonds, whereas interactions between water and RNA were modeled as hydrogen bonds. These snapshots were subsequently converted using the ambpdb program from the AmberTools suite of programs to FIRSTdataset files. In order to model the presence of the ligand guanine for the rigidity analyses by FIRST, we added constraints to the FIRSTdataset file connecting nucleotides U47, C74, and U51 in a pair-wise manner; these nucleotides interact with the ligand in the crystal structure (loop (i) without a ligand in the binding site, (ii) with guanine at the position of the original ligand, and (iii) with the added constraints representing the ligand, and investigated the rigid cluster decomposition of the binding nucleotides   . Withoutand C74) . The preand C74) . Similarand C74) . The difplrc(i) values for the aptamer domain without ligand constraints  and with ligand constraints . In order to investigate the interplay between the presence of tertiary interactions in the loop region and the presence of a ligand in the binding site, we evaluate cooperative effects on the P1 region. To do so, we computed Coop(i) > 0, interactions at both sites together have a larger effect on nucleotide i than the sum of the effects of the separate interactions. We are mainly interested in the effect on the terminal nucleotides in the P1 region, and thus in Coop(i) for i ∈ {15, 16, 17, 18, 19, 77, 78, 79, 80, 81}.If S1 Figapt with 20 Mg2+ ions.The ions  explore the space around the Gsw aptamer (black) in a MD simulation of Gsw(TIFF)Click here for additional data file.S2 Figapt  and Gswloop . A: Simulations with 12 Mg2+ ions; B: Simulations with 20 Mg2+ ions.Results are shown for three simulations of Gsw(TIFF)Click here for additional data file.S3 Figapt in the absence of Mg2+ (A) and the presence of 20 Mg2+ (B).The RMSD values for the three independent simulations are shown in different grey colors for Gsw(TIFF)Click here for additional data file.S4 Figapt in the absence of Mg2+ (A) and in the presence of 20 Mg2+ ions (B) and for Gswloop in the absence of Mg2+ (C) and in the presence of 20 Mg2+ ions (D) (colored in different grey tones for the three independent simulations) is overlaid with the crystal structure of Gswapt (PDB ID 4FE5 [loop (PDB ID 3RKF [The average conformations for Gsw ID 4FE5 ) for (A) ID 3RKF ) for (C)(TIFF)Click here for additional data file.S5 Figapt (A) and Gswloop (B) in the presence of 20 Mg2+ ions using the ff99 force field (blue) and the ff10 force field (red) for comparison. Different line types correspond to three independent simulations. Grey boxes correspond to χ dihedrals in the crystal structures with PDB ID 4FE5 [Normalized histogram for simulations of simulations of Gsw ID 4FE5  and 3RKF ID 4FE5 . The sim ID 4FE5  to a sim(TIFF)Click here for additional data file.S6 Figapt (A) and Gswloop (B) in the presence of 20 Mg2+ ions using the ff99 force field (blue) and the ff10 force field (red) for comparison. Different line types correspond to three independent simulations. Grey boxes correspond to χ dihedrals in the crystal structures with PDB ID 4FE5 (17) and 3RKF (18). The simulations using the ff10 force field were extended from our previous study [Normalized histogram for simulations of Gswus study  to a sim(TIFF)Click here for additional data file.S7 Figapt; B: Gswloop. Hydrogen bonds are shown as black dashed lines. Bases are colored as in A: Gsw(TIFF)Click here for additional data file.S8 FigLarger differences are colored red, smaller differences blue. Nucleotides responsible for ligand binding are shown as sticks.(TIFF)Click here for additional data file.S9 Fig2+ ions (red) or in the presence of 12 (green) and 20 (blue) Mg2+ ions for Gswapt (A) and Gswloop (B). The colored boxes on the top depict substructures of the riboswitch.The differences are shown in the absence of Mg(TIFF)Click here for additional data file.S10 Figapt with 20 Mg2+ ions  and Gswloop in the absence of Mg2+ ions  are projected onto the first (PC1) and second (PC2) eigenvector. The projections for the first  and second half  of the trajectories are shown separately. Histograms are shown on top and on the right side of the projections. Different blue and red colors correspond to the three independent MD simulations of each system.The principal component analysis was performed over all MD trajectories. The trajectories for Gsw(TIFF)Click here for additional data file.S11 Figapt with 20 Mg2+ ions  and Gswloop in the absence of Mg2+ ions  are projected onto the first (PC1) and third (PC3) eigenvector. The projections for the first  and second half  of the trajectories are shown separately. Histograms are shown on top and on the right side of the projections. Different blue and red colors correspond to the three independent MD simulations of each system.The principal component analysis was performed over all MD trajectories. The trajectories for Gsw(TIFF)Click here for additional data file.S12 Figapt with 20 Mg2+ ions  and Gswloop in the absence of Mg2+ ions  are projected onto the second (PC2) and third (PC3) eigenvector. The projections for the first  and second half  of the trajectories are shown separately. Histograms are shown on top and on the right side of the projections. Different blue and red colors correspond to the three independent MD simulations of each system.The principal component analysis was performed over all MD trajectories. The trajectories for Gsw(TIFF)Click here for additional data file.S13 Figapt in the absence (A), with 12 Mg2+ (C), and 20 Mg2+ ions (E), and of Gswloop in the absence (B), with 12 Mg2+ (D), and 20 Mg2+ ions (F) is shown for time intervals up to 500 ns. The RAC was calculated with respect to the first conformation, omitting the first 50 ns of each trajectory.The RAC for trajectories of Gsw(TIFF)Click here for additional data file.S14 Fig2+ ions (red) or in the presence of 12 (green) and 20 (blue) Mg2+ ions for Gswapt (A) and Gswloop (B). The colored boxes on the top depict substructures of the riboswitch. C, D, E, F: Average difference in the probability (Δplrc(i), apt  and Gswloop  in the absence of Mg2+ ions  and in the presence of 20 Mg2+ ions .A, B: Average difference (±SEM). The differences are shown in the absence of Mg(TIFF)Click here for additional data file.S15 Figloop was subjected to an energy minimization before the rigidity analysis. Here, only the binding nucleotides are shown. A: The binding site of Gswloop is empty. B: Guanine is present in the binding pocket. C: Constraints representing the bound ligand Click here for additional data file.S1 Table(PDF)Click here for additional data file.S2 Table(PDF)Click here for additional data file.S3 Table(PDF)Click here for additional data file.S4 Table(PDF)Click here for additional data file.S5 Table(PDF)Click here for additional data file.


Bacteria in a biofilm colony have the capacity to monitor the size and growth conditions for the colony and modify their phenotypical behaviour to optimise attacks, defence, migration, etc. The quorum sensing systems controlling this involve production and sensing of diffusive signal molecules. Frequently, quorum sensing systems carry a positive feedback loop which produces a switch at a threshold size of the colony. This all-or-none switch can be beneficial to create a sudden attack, leaving a host little time to establish a defence. The reaction-diffusion system describing a basal quorum sensing loop involves production of signal molecules, diffusion of signal molecules, and detection of signal molecules. We study the ignition process in a numerical solution for a basal quorum sensor and demonstrate that even in a large colony the ignition travels through the whole colony in a less than a minute. The ignition of the positive feedback loop was examined in different approximations. As expected, in the exact calculation the ignition was found to be delayed compared to a calculation where the binding of signal molecules was quasistatic. The buffering of signal molecules is found to have little effect on the ignition process. Contrary to expectation, we find that the ignition does not start when the threshold is reached at the center—instead it allows for the threshold to be approached in the whole colony followed by an almost simultaneous ignition of the whole biofilm aggregate. Quorum sensing (QS) is a biological regulation process utilised by bacteria to control behaviour in accordance with size, density, and growth-rate of a bacterial population . The proVibrio fischeri where it regulates camouflage light in large cell colonies in the host .The reactions in b1 signifies the production of monomer regulator with concentration r1 = [R]. In the model the reaction between regulators is governed by on- and off-rates, denoted by λi, i = 1, …, 4. The dimer degradation rates are assumed to be equal λd = λ2 = λ3 = λ4 [λ1. All degradation rates include proteolytic degradation as well as dilution by cell division. The degradation rates are assumed to be considerably slower than the on- and off-rates.The term  λ3 = λ4  and signs = [S] is modelled using a diffusion equation containing a production term ρvκs:κs by the volume fraction occupied by cells, ρv, ensures correct normalisation of the production term. Note that the production term shifts away from background production, ρvbs, already when the activated regulator level reaches Additionally, an equation governing the production and diffusion of signal molecules is required. The change in signal molecule concentration ack loop .t, and distance from center. Additionally, the boundary of the colony is assumed to be absorbing, When solving the model numerically, a spherical geometry with radius et al. Below, we shall refer to the solution of Eqs . Thi. Thi5). trations . Both rmra, [The static approximation of the model leads to an instantaneous switch and can be solved by letting all time derivatives vanish . At the ion, ra, . Here alpdepe in Matlab [ρv, at a rate λ0 = 0.5 h−1. The full set of parameters used in the simulation is listed in The system of differential equations was solved using the numerical solver n Matlab , 31. Thera, is displayed as a function of the size, Σ. The curves represent the full model with and without buffering (broken lines) compared with the static and quasi-static solutions (full lines).In First, we note that the exact model with buffering is indistinguishable from the model without buffering, thus indicating that buffering of the signal molecules does not play a significant role.et al. the feed-back loop appears to ignite at a somewhat lower size than the full solution and the quasi-static solutions. This is due to the approximation, that the concentration of activated regulator is taken to be equal to its value at the center throughout the colony. It therefore reflects a limitation of the simple factorized model in In the factorized static approximation by Ferkinghoff-Borg ra.)The quasi-static solution resembles the exact solution, differing only slightly around the time of ignition. As expected, the ignition of the system occurs prior to the ignition in the exact solution. In contrast to the quasi-static solution, at ignition, the full solution jumps only part of the way up to the factorized static solution. The reason for this difference is that, in the full model, the dimer concentration needs to build up and does so at the same rate as the dilution from growth and degradation in the cells. The exact and quasi-static solutions show an increase of more than two decades in activated regulator concentration, shortly after reaching the ignition point. Examining the circles, it can be seen that the ignition takes less than a minute. (Here we define “ignition” as a jump of 2 orders of magnitude in concentration of activated regulator, The activation of the feed-back loop as a function of the radial coordinate and time may be examined in We have modelled a generic single-loop quorum sensing system with positive feedback. The primary goal of the study has been to study of the space-time structure of the ignition of the switch produced by the positive feedback in the quorum sensor.The exact solution exhibits a delayed response compared to both the quasi-static and static approximations, as it is limited by the time it takes the regulator stages to react and build up concentration of activated regulator. Inclusion of buffering terms produces no further retardation of the system.Using a 3D representation to depict both the spatial and temporal dependency of the activated regulator concentration, the collective behaviour of the colony could be studied. A slow build-up over five minutes to the ignition concentration of activated regulator followed by a quick ignition was observed. The process exhibits the desired behaviour, as the entire colony is either in an on- or off-state. These observations indicate that a partial ignition is difficult to achieve, even for slow systems. The model demonstrates that even the largest naturally occurring biofilm aggregates in chronic infections  ignite f


Video summarization is a process to extract objects and their activities from a video and represent them in a condensed form. Existing methods for video summarization fail to detect moving (dynamic) objects in the low color contrast area of a video frame due to the pixel intensities of objects and non-objects are almost similar. However, edges of objects are prominent in the low contrast regions. Moreover, to represent objects, geometric primitives  are distinguishable and high level shape descriptors than edges. In this paper, a novel method is proposed for video summarization using geometric primitives such as conic parts, line segments and angles. Using these features, objects are extracted from each video frame. A cost function is applied to measure the dissimilarity of locations of geometric primitives to detect the movement of objects between consecutive frames. The total distance of object movement is calculated and each video frame is assigned a probability score. Finally, a set of key frames is selected based on the probability scores as per user provided skimming ratio or system default skimming ratio. The proposed approach is evaluated using three benchmark datasets—BL-7F, Office, and Lobby. The experimental results show that our approach outperforms the state-of-the-art method in terms of accuracy. Due to the advancement of technology, video surveillance has been used widely in emerging places to help ensure a safe and secure life style. Government, public safety organizations, and transportation agencies mainly rely on real-time video surveillance systems for security, traffic management, and emergency operations. Surveillance video cameras are setup in offices, railway stations, bus stops and other places. These cameras are used to monitor the activities within these places by trained professionals who always observe video from the monitoring centre. To investigate a crime or to find any specific events from a long video recording, it can take many work hours . FurtherVideo summarization (VS) is a process to extract the most informative set of frames known as key frames or a set of video fragments from the original video. The main purpose of VS is to generate a short video so that an observer can get a complete idea about all the high priority entities and events. VS must be as concise as possible and should contain all significant contents of the entire video. It should also maintain continuation of information and be free of repetition without losing any important video data. To summarize a video stream by considering all these properties, it is necessary to extract some important features of the video. These features are later applied to construct a very short version of the original video. Objects and people within a video play an important role for video summarization . The reaIt remains a challenging problem to extract dynamic objects from a video with low contrast, illumination change, noise, and multimodal dynamic environment  7]. How. How7]. The existing methods for object detection apply complete circles or ellipses to represent curve fragments . HoweverIn this paper, a novel approach is introduced for extracting dynamic objects applying geometric primitives such as line segments, angles and conic parts, and for generating a summary of a long video. The straight contours, corners, and curved contours of an object are presented by line segments, angles, and conic  parts respectively. For this purpose, an edge image is generated from a video frame by applying the Canny edge detection method. After that, lists of connected edge-segments and line segments fitted with each edge-segment are obtained from the edge image by applying the method developed by Kovesi . A singlO(N5) space is required for an ellipse to accumulate the parameter space, where N is the size of each dimension of the parameter space. Moreover, finding an optimal threshold for selecting an ellipse from high dimensional space is another problem . Ho. HoO(N5)The key contributions of the proposed method for video summarization are as follows:A new parameter-less tangent estimation method is proposed for conic part construction;Conic parts are applied to model curve contour for object detection instead of circular or elliptical arcs;A new method for dissimilarity measure of geometric primitives is proposed for recognizing the activity of objects;Geometric primitives, such as line segments, angles and conic parts are applied for extracting objects in a video with low contrast or illumination changes.The remaining of this paper is organized as follows. Section II describes the related work proposed in the literature on video summarization methods in recent years. A brief description of Pascal’s theorem is provided in Section III. The detail of the proposed method is discussed in Section IV. Extensive experimental results as well as an analytical discussion are provided in Section V and concluding remarks are presented in Section VI.Objects/people play the most significant role in a video for summarization. In , a set oIn addition, a Bayesian foraging strategy is applied in  for objeMoreover, image signature is applied for foreground object detection and then fused with motion information to summarize egocentric video in . A modulRecently, a surveillance video summarization method is proposed in . Single To the best of our knowledge, existing approaches for video summarization did not apply geometric primitives  although they have the capabilities to represent objects with complex shapes and structures effectively in challenging environments such as video with low contrast and illumination change. These geometric primitives have several important properties as mentioned in the introduction section. For example, they are independent of object size, efficient for comparisons and matching, and invariant to scale and viewpoint changes. Thus, in this paper a new video summarization method utilizing geometric primitives is proposed.p1, p2, p3, p4, p5, and p6 are six vertices of a hexagon inscribed in a conic (green dotted ellipse) where no three vertices are co-linear and sides of the hexagon are not parallel. The pair of opposite sides is represented with the same color. The point q1 is the intersection between opposite sides p6p1 (black line) and p4p3 (black line). The intersection of opposite sides p1p2 (light green line) and p5p4 (light green line) is q2. The opposite sides p2p3 (magenta line) and p6p5 (magenta line) meet at q3 point. According to Pascal’s theorem, the intersecting points  are co-linear and the line connecting these points is Pascal line (blue line).In this work, the curve segments (conic parts) are extracted using Pascal’s theorem  14]. Th. Th14]. Pascal’s theorem can also be applied when five vertices from a hexagon are provided. The sixth vertex can be calculated using the provided five vertices. This sixth vertex will also be on the conic sections and satisfy the property of co-linearity. Interested readers are referred to  for moreThe introduced method has four main steps. They are as follows- (i) Geometric primitives extraction, (ii) Measure the displacement of geometric primitives, (iii) Assignment of probability score, and (iv) Key frame selection. The main steps of the proposed method are shown in In the proposed method, objects in a video frame are represented by geometric primitives, such as line segments, angles and conic parts. The motivation is that these primitives are independent of object size, efficient for comparisons and matching, and invariant to scale and viewpoint changes. Moreover, they are an effective feature in a challenging environment. To extract geometric primitives, the conventional Canny edge detection method is applied to obtain a binary edge image from a video frame. In A single line segment is considered as a straight line (Φ)  is Pascal line and is represented by a blue line. The expected tangent line (t) of the conic at point p1 is obtained by connecting p1 and q3. Similarly, we can get a tangent line at any point of the conics. In this way, two tangents (t1 and t2) are constructed at each end point (p1 and p5) of the edge segment.To validate an edge segment as a conic part, tangents are drawn first at each endpoint of an edge segment and an arbitrary point on the edge segment is required. The existing methods for tangent estimation construct a tangent on a digital curve based on a parameter  19]. Ho. Ho19]. a circle , an ellia circle , a paraba circle  and a hya circle . To avoiThese tangents are then used to construct a conic part using Pascal’s theorem. In our method, five points are obtained from an edge segment so that these points can divide it into four equal parts. We follow this approach as it represents the conic more accurately than random sampling.Algorithm 1 getTangentBeginq1 between p1p2 and p5p4 Find the intersecting point q2 between p5p1 and p3p2 Find the intersecting point q3 between p4p3 and q1q2 Find the intersecting point t on p1 by connecting p1 and q3 Draw the tangent EndIn the real world, part of an object can be a circular or an elliptical or a parabolic or a hyperbolic curve. As a result, the object detection methods proposed with the circular arc does not fit accurately with an elliptical or a parabolic or a hyperbolic curve and vice versa. Therefore, an innovative conic part construction method is introduced.t1 and t2) and a selected point (p3) from the edge segment, a conic part is constructed based on Pascal’s theorem. Consider, tangents t1 and t2 at p1 (start point) and p5 (end point) of a circle (t1 and t2) intersect at point q1. The tangent t1 and p5p3 meet at point r. The point q2 is selected from line p5r. The intersecting point (q3) is obtained from q1q2 and p1p3. The line q1q2q3 is Pascal line (blue line in p6) on the conic is obtained by intersecting p1q2 and q3p5. If the point q2 is moved from r to p5, the conic part p1p3p5 is obtained. Following this process, a conic part is obtained for the corresponding edge segment. The edge segments are fitted with the corresponding conic parts using the Least Square Fitting (LSF) method with residual two pixels.Using two tangents Beginq1 between t1 and t2 Find the intersecting point r between t1 and p5p3 Find the intersecting point q2 from p5r Select a point q3 between q1q2 and p1p3 Find the intersecting point p6 between p1q2 and q3p5 Find the intersecting point q2 from r to p5, p6 will move from p1 to p5 and conic part p1p3p5 will be constructed connecting p1, p3 and q5 Move EndIf the connected edge segments fit with conic parts obtained by Pascal’s theorem, these conic parts represent curve segments (Ψ) of objects in a video frame (see green curve in Object activities are indicators of events within a video . FurtherFn) and previous frame (Fn − 1) with geometric primitives are denoted by Gn and Gn − 1). The pixel values of Gn or Gn − 1) contains A×2 array of ( (D of Gn with respect to Gn − 1), Gn, Gn, and Gn = 1, 1, 1, and 1 , Gn − 1, Gn − 1, and Gn − 1 = 1, 1, 1, and 1 , Gn, Gn, and Gn = 1, 1, 1, and 1 , Gn − 1, Gn − 1, and Gn − 1 = 0, 0, 0, and 0  in Gn with respect to Gn − 1)( is measured and categorized into a part of stationary or dynamic objects based on the threshold τ. The line segments (Φ), angles (Ω) and conic parts (Ψ) in Gn that belong to dynamic objects are denoted by dΦn, dΩn, and dΨn respectively where d represents dynamic objects. The stationary geometric primitives are neglected.The dissimilarity score The geometric primitives of frame number 4720 and 4721 from the bl-3 video of BL-7F dataset  plu pluτ was methods  51]. In. Inτ wastp is the number of key-frames selected by the proposed method, fp is the number of frames that are not key-frame selected by the proposed method, and fn is the number of key-frames not selected by the proposed method. However, they alone cannot provide an unbiased measurement of the performance of the proposed method. For example, a method with high precision and poor recall or vice versa cannot be an excellent method. Therefore, a method with both higher precision and recall is an excellent approach. To represent this measure, F1-measure is defined as combining both precision and recall and is represented as follows:-An objective evaluation was performed to justify the effectiveness of the proposed method. In this regards, a set of objective evaluation metrics, such as precision, recall and F1-measure were computed. The definition of precision and recall are as followsThe amount of conic parts in a video frame is very low compared to other geometric primitives, such as line segments and angles (corners). Among all the conic parts, elliptical segments may exist more than circular, parabolic or hyperbolic segments. However, conic parts still have a significant role to detect dynamic objects and to summarize a video. To identify the role of each geometric feature, such as line segments, angles, and conic parts for detecting dynamic objects and generating the summary of a video, we compare the result obtained by the individual geometric primitive (feature) on bl-0 video of BL-7F dataset. In The proposed approach is compared with the single-view summarization of the GMM based method  as the pIn The results of the precision, recall, and F1-measure obtained by both the GMM-based method (intra-view) and the proposed method for the BL-7F dataset are provided in The reasons for the failure of the proposed method to perform better in bl-12 video of BL-7F dataset have been evaluated. In bl-12 video, some ground truth key-frames do not contain any dynamic objects or object activities. Similarly, some frames are not selected as ground truth key-frames although they contain significant dynamic object or object activities as provided in [After observing the proposed method both quantitatively and qualitatively, it is certain that the proposed method based on geometric primitives performs better than the GMM based method (intra-view) . The maiIn this paper, an innovative approach is proposed to summarize video using geometric primitives, such as line segments, angles, and conic parts. Existing video summarization methods fail to detect dynamic objects in low contrast regions. However, edges are prominent in low contrast regions. Again, to represent objects, geometric primitives  are higher level and more distinguishable descriptors than edges. Existing object detection methods apply circular or elliptical arcs or entire circles or ellipses for object segment representation. However, elliptical arcs do not fit accurately to circular curves or vice-versa. Therefore, a conic part is applied for fitting the curve segments. To measure the activities of objects, a new cost function is proposed to calculate the displacements of geometric primitives between two successive frames. Experimental results has shown that the proposed summarization method using geometric primitives outperforms the recent state-of-the-art method. The proposed method performs very well in case of stationary camera. In the future, we will consider the moving camera for video summarization.