Smart video surveillance helps to build more robust smart city environment. The varied angle cameras act as smart sensors and collect visual data from smart city environment and transmit it for further visual analysis. The transmitted visual data is required to be in high quality for efficient analysis which is a challenging task while transmitting videos on low capacity bandwidth communication channels. In latest smart surveillance cameras, high quality of video transmission is maintained through various video encoding techniques such as high efficiency video coding. However, these video coding techniques still provide limited capabilities and the demand of high-quality based encoding for salient regions such as pedestrians, vehicles, cyclist/motorcyclist and road in video surveillance systems is still not met. This work is a contribution towards building an efficient salient region-based surveillance framework for smart cities. The proposed framework integrates a deep learning-based video surveillance technique that extracts salient regions from a video frame without information loss, and then encodes it in reduced size. We have applied this approach in diverse case studies environments of smart city to test the applicability of the framework. The successful result in terms of bitrate 56.92%, peak signal to noise ratio 5.35 bd and SR based segmentation accuracy of 92% and 96% for two different benchmark datasets is the outcome of proposed work. Consequently, the generation of less computational region-based video data makes it adaptable to improve surveillance solution in Smart Cities. Video surveillance is a key element for groundwork of a balanced smart city. Video data from all-over the smart city is collected from cameras acting as visual sensors. The collected visual data can be used to analyze objects and their characteristics from recorded scenes. There are several applications which helps to structure basic services of smart city such as smart traffic management and enhanced public security , 14. TheModern smart city surveillance cameras are equipped with video coding techniques such as High-efficiency video coding H.265/HEVC)  develope65/HEVC  The main problem for high-efficiency smart city video surveillance encoding is to (1) extract salient-regions efficiently with low inference time. (2) Maintain a trade-off between visual quality and bitrate of the salient regions. Firstly, the existing DL-based S-R segmentation techniques are dense and requires high inference time. Therefore, there is a need of such DL-based network which can extracts salient-regions closer to human visual system with low computational cost and less inference time. Secondly, the video obtained by high-definition smart city surveillance cameras can only be transmitted over networks with higher bandwidths. Therefore, it is mandatory to reduce the video size to transmit it over low bandwidth networks. Traditional encoding techniques such as HEVC, can be used to reduce video size. However, these techniques cannot preserve high-quality of S-Rs while compressing the surveillance video that is why these techniques cannot be directly used for smart city surveillance.I.We explore the-state-of-the-art techniques by comparing existing tools and evaluating the salient region-based segmentation techniques and video encoding schemes to identify valid problem. We have also identified that the larger video size for high-quality of S-Rs can interrupt the continuous stream of smart city surveillance video in emergency scenarios.II.A DL-based Shallow Semantic Segmentation Network (S-SSN) is proposed to extract the salient region from input surveillance video frames. The flow of CNN-based layered architecture and the role of the softmax layer is also modified. Although existing literature presents a lot of efforts towards semantic segmentation, to best of our knowledge use of the shallow approach for semantic segmentation in smart city surveillance scenario is not explored much.III.For efficient smart city surveillance, an efficient segmentation-based encoding framework is proposed and evaluated through case study in smart city environment. We have also performed extensive experiments on varied smart city surveillance videos such as, cross-road, banks, pedestrian in smart city.This paper presents an Efficient Shallow Segmentation based Encoding (ESSE) framework which ensures the high quality of salient regions in smart city surveillance video by reducing its size as well. It helps to identify suspected person/vehicle, detect traffic and roads. The scheme uses a less complex solution for deep learning based semantic segmentation. In ESSE, a shallow semantic approach is used for salient region segmentation as it requires fewer layers and parameters that reduces time complexity, which is suitable for surveillance. Moreover, we modified the HEVC encoding based on the extracted S-Rs which helped to reduce video size while preserving quality of salient regions from smart city surveillance video frames. The major contributions of this work are enumerated as follow:The rest of the paper is structured as follows: Sect. This section explores the importance of video surveillance in smart cities, the existing schemes for efficient encoding and transmission of surveillance videos in smart cities and then deep learning based salient region segmentation techniques are also explored.Video surveillance of smart cities is an important part for development of balanced and secure environment. The first priority video surveillance is to get as much salient information as possible from the surveillance cameras in the shortest time. In  Ruben etIn this section, a collection of HEVC coding-based schemes are explored that highlight the S-Rs. HEVC based video coding is used in surveillance for city areas, such as presented in . Kim et Our proposed scheme overcomes this issue by utilizing a semantic segmentation approach for salient-region extraction. Moreover, we have identified that applying DCN on classic VGG uses a dense approach with a large number of parameters. It increases computational cost, which is not suitable for the surveillance scenario. In the proposed approach, an improvement has been achieved by reducing both the number of parameters and network depth. The existing video encoding techniques reduces the resolution ratio of surveillance video, which also affects the performance of video analysis tasks, e.g., object/road localization, detection and recognition, vehicle tracking, and crowd monitoring . TherefoThis section reviews related techniques that use image segmentation to extract salient objects and their regions. Image segmentation is used to extract the S-R in surveillance video frames. The conventional ML image segmentation techniques can be used for salient region extraction. i.e., K-means , Density1,…,XN}a set of labels Y = {Y1,…,YN} is predicted taking values in ΨN. Pixel-level accuracy is used as evaluation matrix for semantic segmentation. Accordingly, these evaluation matrices use a confusion matrix C of pixel level, which sums inferences for all the images of dataset as given in  framework for smart city video surveillance of vehicular traffic, roads and human beings. It demands a high quality semantic segmentation technique to extract the salient regions like humans and vehicles where video size is also reduced to ensure uninterrupted transmission of surveillance video especially for emergency scenarios. For bulk video analysis the data should be light weight to prevent storage and quality issue. In this work, we present a framework for video efficient surveillance video transmission for targeted visual analysis based on salient video encoding which helps to attain high quality for targeted salient region with low file size. The proposed framework could help to segment salient region such as humans and vehicles of each type from video frame and transmit in low bandwidth capacity. The proposed framework can help in intelligent transport management, street crime, and road hazards. The proposed framework is divided into two phases as shown in Fig. In Phase I, the salient region (S-R) is segmented from the input video frames by using our proposed Shallow Semantic Segmentation Network (S-SSN) which is trained using Mapillary vistas dataset divided into training data  and testing data (2000 images). For training the parameters are set such as In phase II, the extracted frames are preprocessed to generate a set of the salient map (SM) for each S-R per frame. Next, the SM is further utilized to create QP maps for all extracted N frames. Furthermore, a set of QP maps is also maintained for all N frames. Meanwhile, YUV frames are also created from the original input video. Next, the modified HEVC encoder takes each QP map from the set to enhance the QP values of each YUV frame. In this case, we modified the default HEVC by considering the QP values to sustain the quality for salient regions like the vehicles and pedestrians in the frame. We reduce the quality of irrelevant areas by increasing QP values that result in reducing the overall size of the video. Modified HEVC encoder generates the encoded video bit-stream with a reduced size that requires less bandwidth. It enables the continuous the remote monitoring of ambulance through surveillance camera and the video can be transmitted even when bandwidth is low. This proposed scheme can facilitate remote visual analysis by providing high quality for S-Rs to analyze the data at cloud. It also reduces the video size that enhances the chances of uninterrupted video sharing even if bandwidth is limited. A list of notations is presented in Table 1–L9 from left to right take the frame as input and then down-sample it.The proposed S-SSN is a DL-based segmentation network that is based on a shallow semantic approach. In the approach, a shallow architecture is used for the segmentation network instead of dense architecture. This is because shallow architecture has less time complexity due to fewer parameters, and therefore, it suits surveillance scenarios. The S-SSN semantically extracts those classes which are important for road surveillance, i.e., vehicles, motorbike, pedestrian, and roads. The dataset used during training and testing phase of proposed S-SSN is open-source Mapillary Vistas and Camvid, which is considered to be a benchmark . It compDuring up-sampling, layers start from L10–L17, where the output image of size 28 × 28 × 256 from down-sampling is fed into the network as input. The input image is up-sampled by using the convolution layer, which comprises of 256, 128, and 64 filters of size 5 × 5. The up-sampling and normalization layers return an image of size 224 × 224 × 5, where the 224 × 224 is height and width of image and 5 are the number of classes for segmented salient regions, i.e., vehicle, pedestrian, road, bicycle/bike, and the background. In the end, S-SSN uses a softmax layer that calculates the probability of each class in contrast to all possible numbers of classes. The S-SSN generated an extracted salient region frame for N number of frames, i.e. S-SSN involves the CNN layers, including convolution layers, max-pooling layers, up-sampling layers, ReLU, and batch-normalization. All layers are stacked to calculate activation function at given in  where \dessed in  at everyThe softmax stacks these layers in a particular order, as expressed in . In thisIn Phase-II, we created SM to generate QP maps. These maps are further used for video encoding using HEVC, which is discussed in the following sub-sections.m − 1}, and m represents rows count in a frame. Step 8 explores the iterations for the number of columns where column index c ranges from {2, 3, …, n − 1}, and n is the number of columns in the frame. In step 9, the forward pass is calculated by moving diagonally on the extracted frame where wn and wd are diagonal weights in neighbors which are used to calculate distance from each pixel location of the frame. During backward pass on the video frame, step 13 shows iterative repetition in reverse order for rows index r ranging from {m − 1, m − 2, …, 2}. Similarly, column index c is iterated in a reverse manner ranging from {n − 1, n − 2, …, 2}, as shown in step 14. In step 15, the backward pass moves diagonally in backward direction to calculate the distance of each pixel from its location by using The frames with extracted salient regions are preprocessed to enhance the visibility and area around S-R. Next, a for each S-R is generated using distance transform . In AlgoSM is used to create a QP map, which is further utilized to adjust the quality of S-Rs and non-SRs. The set of SM creates a set of QP maps for N frames. QP maps entclass1pt{minimaWe have utilized default HEVC reference test model HM 16  for modiThis Section explores the performance of the implemented ESSE scheme as compared to its counterparts. For the experimental setup, the Adadelta optimizer is used for training the proposed S-SSN with default parameters by using Keras and Tensorflow python libraries. All experiments are performed on the Google Colab platform with Nvidia Tesla K80 GPU having 11 GB VRAM and 12 Gb of RAM . HEVC reIt is quite essential to measure the bit-rate due to reduction in the size of encoded video to evaluate the surveillance video on low bandwidth networks. We use the same evaluation matrices as in  by consiWe calculated the PSNR value of the complete frame to analyze a fair comparison of ESSE framework with default HEVC. The average PSNR value of default HEVC for the complete frame is higher than ESSE since the proposed ESSE focuses on preserving the S-R while maintaining the low bit-rate by assigning higher QP values to the regions which are non-S-Rs. Therefore, the proposed ESSE achieved the higher PSNR value. The results from Table A quantitative measure is to check the accuracy of S-R segmentation and compare it with state-of-the-art semantic segmentation networks, including SegNet , U-Net , and DN mean IoU  is the pFigure Figure This Section presents case study test scenarios results for our proposed ESSE framework for smart city video surveillance on baseline QP of 20 for S-Rs. We have conducted extensive experiments in different environments and luminance conditions for vehicles, pedestrians, motorbike and bicycles to evaluate the performance of our proposed ESSE framework. Case studies are as follow:In first case study we conducted the experiments for pedestrian in smart city environment. For testing our proposed framework, we have conducted experiments for three different scenarios for pedestrian. We have utilized testing sequence  for scenIn case study 1 scenario 2, we have tested the proposed ESSE for aerial view static camera . Figure In second case study the experiments are performed for multiple scenario of vehicles on roads in smart city environment. For testing purpose we have utilized testing sequence  for statThe second scenario of case study 2 is aerial view static camera . The oriThe third scenario of case 2 is moving dash board camera . The resThe third case study is conducted for different motorbikes and bicycle in smart city environment. The video sequences of static camera  is takenThe second scenario of third case study is a moving camera  where thFigure We have evaluated the proposed ESSE framework in term of Pixel accuracy , number of parameters, mIoU, bit-rate and PSNR. We have compared the proposed S-SSN with deep learning based state of the art techniques such as Segnet, U-Net and DN which are trained on two different benchmark datasets CamVid and Mapillary vistas for five different classes i.e., pedestrian, vehicle, road, cyclist and motorcyclist. The result elucidates from Figs. The overall performance of proposed ESSE framework is also compared with state of the art techniques i.e., perceptual based HEVC and default HEVC. The results are explained in Figs. In a smart city surveillance scenario, the visual data collected through camera is required to be transmitted in high visual quality for analysis. The transmission of high quality visual data is a challenging task on low capacity bandwidth communication channels. To address this issue, we have proposed a surveillance framework ESSE, that integrates deep learning based salient-region (S-R) extraction and efficient video encoding. The ESSE is presented in two phases. In Phase-I, a shallow-semantic segmentation network (S-SSN) is proposed to extract S-Rs and validated for two benchmark datasets. The phase-II generates S-R based salient map, which helps to create a QP map for saliency-based HEVC encoding. The proposed S-SSN achieved pixel-level validation accuracy of 96% and 92% for CamVid and Mapillary vistas datasets respectively. The efficacy of proposed ESSE is also evaluated by conducting extensive smart city surveillance case study for diverse road conditions. The detail results of case study analysis are presented in subsection (4.1–4.4) of result and analysis. The major finding of ESSE framework is the capability of extracting S-Rs and transmitting them in high quality while maintain low bit-rate. Therefore, the proposed solution has a wide applicability for areas having low bandwidth resources. The limitation of this proposed solution is that the training phase of ESSE framework is based on supervised learning. In future, we shall consider the DL semantic segmentation method based on un-supervised learning for Multi-View HEVC  to detec


Protein oligomerizationis a commonly encountered strategy by whichthe functional repertoire of proteins is increased. This, however,is a double-edged sword strategy because protein oligomerization isnotoriously difficult to control. Living organisms have thereforedeveloped a number of chaperones that prevent protein aggregation.The small ATP-independent molecular chaperone domain proSP-C BRICHOS,which is mainly trimeric, specifically inhibits fibril surface-catalyzednucleation reactions that give rise to toxic oligomers during theaggregation of the Alzheimer’s disease-related amyloid-βpeptide (Aβ42). Here, we have created a stable proSP-C BRICHOSmonomer mutant and show that it does not bind to monomeric Aβ42but has a high affinity for Aβ42 fibrils, using surface plasmonresonance. Kinetic analysis of Aβ42 aggregation profiles, measuredby thioflavin T fluorescence, reveals that the proSP-C BRICHOS monomermutant strongly inhibits secondary nucleation reactions and therebyreduces the level of catalytic formation of toxic Aβ42 oligomers.To study binding between the proSP-C BRICHOS monomer mutant and smallsoluble Aβ42 aggregates, we analyzed fluorescence cross-correlationspectroscopy measurements with the maximum entropy method for fluorescencecorrelation spectroscopy. We found that the proSP-C BRICHOS monomermutant binds to the smallest emerging Aβ42 aggregates that arecomprised of eight or fewer Aβ42 molecules, which are alreadysecondary nucleation competent. Our approach can be used to providemolecular-level insights into the mechanisms of action of substancesthat interfere with protein aggregation. Here we employed the molecularchaperone domain proSP-C BRICHOS as a reporter for small, soluble,and secondary nucleation competent Aβ42 oligomers using fluorescencecross-correlation spectroscopy (FCCS). Because Aβ42 aggregationis a highly heterogeneous process, we applied the maximum entropymethod for FCS (MEMFCS)9 fitting routineto analyze our data.Aggregationof protein intoamyloid fibrils is associated with severe medical conditions, likeAlzheimer’s disease, Parkinson’s disease, and type IIdiabetes.10 Therefore, it has served asa valuable model chaperone for the development of new approaches tostudy the kinetics of Aβ42 aggregation.14 However, in solution, the proSP-C BRICHOS domainexists as an equilibrium mixture between mainly trimers and monomers,where the former is thought to be an inactive storage conformation.16 To fluorescently label chaperone active proSP-C BRICHOS domain subunits,we rationally designed a stable monomeric variant by site-directedmutagenesis based on the trimeric proSP-C BRICHOS domain structure.16 The proSP-C BRICHOS monomer mutant, like thewild type (WT), does not bind Aβ42 monomers, has high affinityfor Aβ42 fibrils, and specifically inhibits secondary nucleation.We were able to show that the proSP-C BRICHOS monomer mutant bindsto a heterogeneous mixture of soluble Aβ42 species, and importantly,we measured binding to apparently secondary nucleation competent aggregatescomprised of eight or fewer Aβ42 monomers.The proSP-C BRICHOS domain has shown aunique feature in exclusivelyblocking secondary nucleation reactions on the surface of Aβ42fibrils.Figure S1). Size exclusionchromatography (SEC) profiles and native polyacrylamide gel electrophoresis(PAGE) demonstrate that proSP-C BRICHOS T187R, in contrast to theWT, forms almost exclusively monomers . Both proSP-C BRICHOS proteins have overall similar secondary structures,expose hydrophobic surfaces, and maintain their respective secondaryand quaternary structures after overnight incubation at 37 °Cas observed by circular dichroism (CD) spectroscopy and bis-ANS fluorescence. Electrospray ionization massspectrometry (ESI-MS) confirms that proSP-C BRICHOS T187R is purelymonomeric and the determined molecular mass of 18328 Da is in goodagreement with the calculated mass of 18324 Da for the mutant monomer and maximum growthrate (rmax). The data show that both proSP-CBRICHOS variants increase τ1/2 in a concentration-dependentmanner, plateauing at a high relative proSP-C BRICHOS concentration,while rmax decreases exponentially . To compare equal numbers of proSP-CBRICHOS molecules, we assumed that proSP-C BRICHOS WT forms solelytrimers in solution and corrected the concentration accordingly . While the delay of τ1/2 and the decrease in rmax at low BRICHOS:Aβ42ratios can be explained by the molecular stoichiometry, the effectson both fitting parameters are somewhat more pronounced for monomericproSP-C BRICHOS T187R at high BRICHOS concentrations.To study the capacities of mainly trimeric proSP-CBRICHOS WT andthe monomeric proSP-C BRICHOS T187R mutant to interfere with Aβ42fibril formation, we performed bulk thioflavin T (ThT) fluorescenceaggregation experiments. Under quiescent conditions, the aggregationof Aβ42 follows a sigmoidal growth curve that can be characterizedby the reaction half-time . We werenot able to detect any specific binding to monomeric Aβ42 ofeither WT, which is in line with previous findings,10 or proSP-C BRICHOS T187R using concentrations of ≤50μM . FCS revealed no changes in the diffusion of Aβ42 monomersfluorescently labeled with HiLyteFluor488 (HiLyteFluor488-Aβ42)in the presence of different amounts of unlabeled proSP-C BRICHOST187R , but the diffusion time remained the sameas for HiLyteFluor488-Aβ42 alone, τD,Aβ 42 (70 ± 10) μs. Dual-color FCCS using HiLyteFluor488-Aβ42and proSP-C BRICHOS T187R fluorescently labeled with Atto655 (proSP-CBRICHOS T187R-Atto655) further corroborated that there are no detectableinteractions between monomeric Aβ42 and proSP-C BRICHOS T187R. This is evident from the unchanged,within experimental error, characteristic decay times of the temporalautocorrelation curves (ACCs) for monomeric HiLyteFluor488-Aβ42 μs) andmonomeric proSP-C BRICHOS T187R-Atto655  μs) in mixed preparations , as well as the lack of cross-correlation betweenthe two signals even at a 1:10 Aβ42:proSP-C BRICHOS T187R ratio. A detailed analysis of cross-talkeffects is provided in the Methods in the Supporting Information.Next, we set out toinvestigate proSP-C BRICHOS WT and proSP-CBRICHOS T187R affinities for monomeric or fibrillar Aβ42 usingsurface plasmon resonance (SPR) 2. The prKD values, one in the micromolarrange (KD1) and one in the nanomolar range(KD2) (KD2) of proSP-C BRICHOS T187R for Aβ42fibrils is approximately 5-fold greater than that of proSP-C BRICHOSWT. The dissociation rate constant of the second component (kd2) measured for proSP-C BRICHOS WT is in goodagreement with previously published values in which only one proSP-CBRICHOS concentration was used to determine the kinetic parameters 1. The aprameters1.10 Howetrations 2B,C and 17 The WT proSP-C BRICHOS domain is a highly specificsecondary nucleation inhibitor under quiescent conditions.17 Consequently, we investigated if the effect on the Aβ42 aggregationmechanism is similar for the monomeric proSP-C BRICHOS T187R mutant,using a bulk ThT fluorescence assay. We performed a series of kineticexperiments and fitted our data with the analytical solution thatdescribes the time evolution of Aβ42 fibril formation as a functionof several microscopic rate constants, i.e., primary nucleation (kn), secondary nucleation (k2), and elongation (k+).5Inhibition of Aβ42 fibril formationby different classes ofmolecular chaperones is known to be associated with their interferencewith different microscopic processes during self-aggregation.5 To test if the aggregation of Aβ42 in the presence of proSP-CBRICHOS T187R is dominated by secondary nucleation, we recorded theaggregation of varying Aβ42 monomer concentrations in the absenceand presence of a constant proSP-C BRICHOS T187R concentration. Wefound that the γ exponent is similar with and without proSP-CBRICHOS T187R , suggesting that the underlying Aβ42 aggregationmechanism is in both cases monomer-dependent secondary nucleation. Next, we investigated theeffects of proSP-C BRICHOS T187R on the microscopic rate constantsof Aβ42 aggregation. The reaction profiles obtained at a constantAβ42 concentration and varying BRICHOS concentrations were fittedwith the combined rate constants for primary 18 we measured Aβ42 fibrillation kineticswith 20% preformed seeds in the absence and presence of increasingproSP-C BRICHOS T187R concentrations . Tostudy which microscopic rate constant is predominantly affected byproSP-C BRICHOS T187R, i.e., kn, k2, or k+, we fittedthe data sets with constant Aβ42 concentrations and varyingproSP-C BRICHOS T187R concentrations globally, where only a singleparameter is allowed to vary freely across all BRICHOS concentrations,while the two others are fixed to the value of Aβ42 alone. Ourresults indicate that the global fits describe the data sets bestwith k2 as the sole free fitting parameter[residual sum of squares (RSS) =0.8] compared to kn (RSS = 4.3) or k+ (RSS =1.5) . Suppression of secondarynucleation mechanisms is associated with a decrease in the level offormation of Aβ42 oligomer intermediates.10 Therefore, we simulated the time evolution of the nucleationrate based on the parameters determined from (i) global fits of differentAβ42 concentrations with and without 0.3 molar equivalent ofBRICHOS  and (ii) individualfits of 3 μM Aβ42 with and without varying molar equivalentsof BRICHOS . Importantly, this reduction is consistentwith both individual calculations. Taken together, our data show thatmonomeric proSP-C BRICHOS T187R inhibits mainly secondary nucleationevents, similar to the WT, and thereby reduces the number of surface-catalyzednuclei.Global fit analysis of the aggregation traces obtained with varyingAβ42 monomer concentrations in the absence and presence of 0.3molar equivalent of proSP-C BRICHOS T187R, where the fitting parameters Having established how monomeric proSP-CBRICHOS T187R inhibits Aβ42 fibril formation, we set out toinvestigate interactions with soluble Aβ42 aggregates. To thisaim, we used FCCS and monitored the time course of Aβ42 aggregationin solutions containing different initial concentrations of unlabeledAβ42  in the presence of 100 nM HiLyteFluor488-Aβ42,and 483 nM proSP-C BRICHOS T187R, of which 100 nM was fluorescentlylabeled with Atto655 (proSP-C BRICHOS T187R-Atto655). In this way,the Aβ42:proSP-C BRICHOS T187R ratio was 10.5:1, 21:1, and 41.5:1for experiments with 5, 10, and 20 μM unlabeled Aβ42,respectively, allowing us to examine in detail the interactions ofproSP-C BRICHOS T187R with Aβ42 aggregates of different sizesformed throughout the aggregation process.D1) was within experimentalerror indistinguishable from the diffusion time of the free monomericspecies  μs for HiLyteFluor488-Aβ42, and τD1,633 = τD,T187R = (175 ± 35) μsfor proSP-C BRICHOS T187R-Atto655). The diffusion time of the firstcomponent did not change during the course of Aβ42 aggregation,indicating that a fraction of unbound HiLyteFluor488-Aβ42 andproSP-C BRICHOS T187R-Atto655 was always present in the reaction mixture,whereas the diffusion time of the second component increased overtime (n = 2) in eq 8 (Methods), where the diffusion time of the first componentwas fixed to the value of the respective monomeric species, whereas the diffusion time (τD2) and the relative amplitude of the second component (f2 = 1 – f1) were allowed to freely change. The results of the two-componentanalysis of ACCs, summarized in ACCs for HiLyteFluor488-Aβ42and proSP-C BRICHOS T187R-Atto655 showed that proSP-C BRICHOS T187R-Atto655binds to soluble Aβ42 aggregates of different sizes 4A. This vertime 4B. Ofno488 [1–a3)] and ACC633 [1–aa3)]. Most notably, the relativecontribution of the second component in the ACC488 is low,while it is steadily increasing in ACC633. This is becauseHiLyteFluor488-Aβ42 is readily displaced by unlabeled monomericAβ42 that is present in large excess in the reaction mixture.As a consequence, a large proportion of HiLyteFluor488-Aβ42remained unbound and the average number of HiLyteFluor488-Aβ42molecules in the observation volume element (OVE) [1)] and thebrightness of HiLyteFluor488-Aβ42 molecules, as reflected bythe counts per molecule and second {CPM488 [1)]}, remainedunchanged over time. In contrast, proSP-C BRICHOS T187R-Atto655 forwhich the ratio of labeled versus unlabeled proSP-C BRICHOS T187Rmolecules was 1:3, could readily bind to the Aβ42 aggregates,and the likelihood that more than one proSP-C BRICHOS T187R-Atto655molecule could bind to the same Aβ42 aggregate was not significantlydecreased by the unlabeled fraction. Consequently, as one can clearlysee from the apparent decrease in the number of proSP-C BRICHOS T187R-Atto655molecules [1)] and the strong increase in brightness at advancedstages of the Aβ42 aggregation process [1)], more than one proSP-CBRICHOS T187R-Atto655 molecule could bind to the same Aβ42 aggregate.This is particularly visible for high initial Aβ42 concentrations(10 and 20 μM), where up to 10 proSP-C BRICHOS T187R-Atto655molecules bound to the same large Aβ42 aggregate, which canbe clearly seen from the brightness that increased from CPM = 2 kHzfor small/medium-sized Aβ42 aggregates characterized by τD < 5 ms to CPM = 20 kHz for Aβ42 aggregates characterizedby τD = 75 ms. Furthermore, due to the intensivedisplacement of HiLyteFluor488-Aβ42 by unlabeled Aβ42,the fraction of bound HiLyteFluor488-Aβ42 molecules, i.e., therelative amplitude of the second component (f2), was always lower for HiLyteFluor488-Aβ42 than forproSP-C BRICHOS T187R-Atto655 molecules [i vs aai) and 1 vsee1)]. Finally, the size of Aβ42 aggregates at theend of observation (tend = 12.0 h) was somewhat smallerwhen the initial Aβ42 concentration was the lowest (5 μM),i.e., when the Aβ42:proSP-C BRICHOS T187R ratio was the lowestand the capacity of proSP-C BRICHOS T187R to inhibit Aβ42 aggregationwas the largest, which can be clearly seen from the diffusion timesthat for most of the time remained <50 ms (We first note the striking difference between thesimultaneouslyacquired ACCd <50 ms 4F.eq 8) withoutthe triplet state term [n = 2 (Methods)] revealed that the cross-correlation amplitude  is solely dueto signal bleed-through from the green to the red channel  or if this component reflects trueinteractions between proSP-C BRICHOS T187R-Atto655 and HiLyteFluor488-Aβ42-labeledAβ42 oligomers. As a first step, we used paired t test analysis to compare the short decay time of the CCCs, τD1,CCC in a time series, with the short decay timesof the simultaneously recorded ACCs, τD1,488 (HiLyteFluor488-Aβ42)and τD1,633 (proSP-C BRICHOS T187R-Atto655). We observedthat τD1,488 < τD1,CCC < τD1,633, with τD1,CCC being significantly longerthan τD1,488 . We also observed that the effect size wasdependent on the concentration of Aβ42 and was largest for 5μM Aβ42 . This prompted us to further examine the CCCs acquiredin measurements with 5 μM Aβ42.The largely bimodal distribution of diffusion times observed forHiLyteFluor488-Aβ42 and proSP-C BRICHOS T187R-Atto655 molecules4 is easyFigure S8) revealed that the distribution profiles of the ACCs488are Gaussian. This is expected because a large portion of HiLyteFluor488-Aβ42is unbound and freely diffusing as it is displaced by unlabeled Aβ42,which is present in large excess. In contrast, the distributions ofdiffusion times for the corresponding CCCs show a positive skew towardlonger lag times. Consequently, for lag times that are longer thanτD1,488, the amplitude of the CCC is larger thanthe amplitude of the corresponding ACC488 . These results show that the short characteristic decay times ofthe CCCs contain contributions from both, the signal bleed-throughfrom the green to the red channel, which defines to a very large extentthe position of the peak but also indicates true interactions betweenproSP-C BRICHOS T187R-Atto655 and HiLyteFluor488-Aβ42-labeledAβ42 oligomers, which can be clearly seen from the positiveskew of the CCCs that builds up as the cross-correlation signal increasesabove the background from the signal bleed-through of green-labeledmolecules.Comparison of theMEMFCS-derived distributions of diffusion times at different timepoints during the aggregation of 5 μM Aβ42 S8 revealD1,633, the diffusion time of monomeric proSP-C BRICHOS T187R-Atto655. This suggests thatcomplexes of proSP-C BRICHOS T187R-Atto655 with Aβ42 aggregatesform and are sufficiently small to not significantly change the distributionof the diffusion time of proSP-C BRICHOS T187R-Atto655 monomers. Themolecular weight of monomeric Aβ42 is 4.5 kDa, and proSP-C BRICHOST187R-Atto655 monomers have a molecular weight of 18.8 kDa. Giventhat the diffusion time of spherical molecules scales with the thirdpower of the molecular weight, an Aβ42 oligomer that is boundby one proSP-C BRICHOS T187R-Atto655 monomer and gives rise to a cross-correlationsignal but does not change the diffusion time of the aggregate toa measurable extent must be smaller than a proSP-C BRICHOS T187R-Atto655trimer (56.4 kDa). Binding of one proSP-C BRICHOS T187R-Atto655 monomerto an Aβ42 oligomer that contains eight or fewer Aβ42monomers would give τD1,633 ≈ τD1,CCC, and hence, they are indistinguishable within the errorof the measurement.Furthermore, we observed a positive cross-correlation signal, whichis well above the signal bleed-through background, at τD1,633, changes over time dependingon the total concentration of Aβ42 , is related to the total number of moleculescarrying a red label .19 For 5 μM Aβ42, where the aggregationcompared to the other concentrations is the slowest and the ratioof labeled to unlabeled Aβ42 is the highest, a transient increasein the number of complexes between one proSP-C BRICHOS T187R-Atto655and small Aβ42 oligomers is observed. This complex formationreaches a maximum value after approximately 3–3.5 h and slowlydecreases thereafter 11 overlapvery well with the smallest Aβ42 aggregates with which the proSP-CBRICHOS monomer mutant interacts. The resolution limit in our experimentalsetup does not allow us to distinguish between smaller, supposedlymore unstable oligomers (2–4-mers) and somewhat larger convertingor just converted oligomers (5–9-mers).11 Nevertheless, direct interactions of the BRICHOS domainwith early unstable oligomers would result in slower rate constantsfor primary nucleation, while interactions with fibrillar structuresthat just converted from oligomers would result in effects on secondarynucleation and/or elongation. We found that the proSP-C BRICHOS monomermutant does not interact with Aβ42 monomers, has no effectson primary nucleation, but strongly associates with Aβ42 fibrilsand efficiently prevents secondary nucleation pathways during Aβ42fibrillation. From this, we conclude that the proSP-C BRICHOS monomermutant binds to Aβ42 aggregates that consist of eight or fewerAβ42 monomers that are already secondary nucleation potent (11 are at the verge of fibril-forming structures.Recent experimental data combinedwith mathematical modeling tosimulate the dynamics of oligomer formation and conversion showedthat Aβ42 monomers assemble into a heterogeneous mixture ofsmall, more unstable oligomers and converting oligomers that are morelikely to transform into fibrillar structures.n potent 7. Furthe16 Many hydrophobic residues are located on face A of thiscentral β-sheet, and it was suggested that it is complementaryto the substrates.28 Recently, a high-resolutionstructure of an Aβ42 tetramer in a lipid environment was reported.7 This Aβ42 tetramer forms a six-strandedβ-sheet core that almost exclusively contains hydrophobic residues.7 We speculate that the BRICHOS domain can exposeface A and bind to the hydrophobic core region of, e.g., soluble Aβ42tetramers. This would also imply that already the six-stranded β-sheetcore of an Aβ42 tetramer can promote secondary nucleation reactions.Furthermore, high-resolution structures of mature Aβ42 fibrilsshow a cluster of hydrophobic residues that are buried in the intermolecularcontact site between two Aβ42 protofibrils.31 We hypothesizethat this hydrophobic cluster could be surface-exposed in small secondarynucleation competent aggregates and hence constitutes a potentialbinding site for proSP-C BRICHOS on small pre- and protofibrillaraggregates.The proSP-C BRICHOS structure is composed of acentral five-strandedβ-sheet that is flanked by two α-helices.16 We speculate that the trimermight adopt a binding competent conformation independent of its monomerizationbut with weaker affinity for Aβ42 aggregates.Another interesting observation of our study wasthat the WT proSP-CBRICHOS domain, compared to the monomer mutant, has an ∼5-foldlower apparent affinity for immobilized Aβ42 fibrils. Nevertheless,both BRICHOS domain variants appear to be equally efficient in preventingAβ42 fibril formation, and only at high protein concentrationsis the monomer mutant slightly more efficient. This difference couldbe explained by the lower affinity of the proSP-C BRICHOS trimer conformationfor fibrils, by bulk effects close to the fibrillar binding sitesor a shift of the equilibrium between different quaternary structuresin WT proSP-C BRICHOS preparations. However, our results stronglysuggest that WT trimers bind to Aβ42 secondary nucleation competentaggregates and contribute to the observed inhibition of Aβ42aggregation. Previously, molecular dynamics simulations suggestedthat only the proSP-C BRICHOS monomer can expose its potential substratebinding site while necessary conformational changes are blocked inthe trimer.32 but the abilities of these aggregatesto promote secondary nucleation in relation to their size remainedunknown. Our findings extend the knowledge of the smallest secondarynucleation potent Aβ42 aggregates and their interactions witha molecular chaperone domain. The approach presented here can be usedto study structural properties of amyloid aggregates using the uniqueproperties of different molecular chaperones.Molecular structures of Aβ42 oligomers and interactions betweenmolecular chaperones and small Aβ aggregates have been shown,


Compression at a very low bit rate(≤0.5bpp) causes degradation in video frames with standard decoding algorithms like H.261, H.262, H.264, and MPEG-1 and MPEG-4, which itself produces lots of artifacts. This paper focuses on an efficient pre-and post-processing technique (PP-AFT) to address and rectify the problems of quantization error, ringing, blocking artifact, and flickering effect, which significantly degrade the visual quality of video frames. The PP-AFT method differentiates the blocked images or frames using activity function into different regions and developed adaptive filters as per the classified region. The designed process also introduces an adaptive flicker extraction and removal method and a 2-D filter to remove ringing effects in edge regions. The PP-AFT technique is implemented on various videos, and results are compared with different existing techniques using performance metrics like PSNR-B, MSSIM, and GBIM. Simulation results show significant improvement in the subjective quality of different video frames. The proposed method outperforms state-of-the-art de-blocking methods in terms of PSNR-B with average value lying between (0.7–1.9db) while (35.83–47.7%) reduced average GBIM keeping MSSIM values very close to the original sequence statistically 0.978. Various international standards have widely adopted Block-based Discrete Coding Technique (BDCT) have widely adopted block-based discrete coding technique (BDCT) like motion JPEG, MPEG-1, MPEG-2, and MPEG-4 for moving frames . It is pBlocking artifacts and flickering generally occurs in moving frames with intricate details when processed at low bit rate coding are very annoying artifacts that degrade the frames’ visual quality. It is critical to address and provide an efficient solution to extract and remove such artifacts.The problem associated with the video artifacts significantly degrades the overall subjective quality of the sequences. Flicker removal has been a relevant topic of research with the evolution of video standards from the last decade. However, a concrete solution to this problem has not been found yet. Due to the inadequate response of encoder during video compression, other artifacts occur, commonly called flicker artifacts , 7.At the same time, researchers used a fixed threshold approach in their work . None ofAlthough extensive work is carried out in image and video compression since the last two decades, it causes degradation in the subjective quality of decoded frames. De-blocking algorithms are broadly classified as in-loop processing and post-processing. At a higher compression ratio, it has been observed that the correlation between adjacent pixels decreases. Due to a one-dimensional filtering approach, the in-loop processing method enhances the coding efficiency by reducing blocking artifacts amongst adjoining pixels or frames but is unable to process corner outliers. To alleviate blocking artifacts different post-processing approaches such as frequency domain analysis –31, ProjSome of the literature solutions are based upon filtration done at the decoder side , 38, 39 The rest of the paper is organized as follows: In Section III proposed PP-AFT method is introduced to detect and rectify the issue of blocking artifacts and flicker artifacts as follows, Initially, a threshold-based pre-processing approach is applied, before de-blocking which helps in removing the quantization signal error. In the later stage, four adaptive filters based de-blocking system is implemented, and finally, flicker extraction and removal for frames have been developed. The PP-AFT method helps in improving discontinuities near block boundaries and also helps in removing flicker artifact among successive video frames efficiently. Experimental results signify the subjective performance of the PP-AFT method over other techniques.Our proposed model’s main objective is to develop a system that can detect and remove artifacts like blocking artifacts as well as flickering artifacts in low bit-rate coded mobile standard definition (SD) videos. In this paper, a multi-fold technique is used. Implementing the pre-processing method removes the abrupt variation in signals that are more sensitive to human eyes. Secondly, the region-based adaptive de-blocking filter algorithm is applied along with the removal of ringing artifacts; finally, the flicker detection and removal algorithm are countered and produce high quality decoded video frames. We propose a multi-step de-blocking scheme to resolve the glitches arising due to compression, as shown in The flowchart of the PP-AFT method is as follows.In Block based Transform Coding (BTC) the significant observations are as follows:In smooth regions, the blocking artifacts are more visible than in non-smooth regions.In non-smooth regions, de-blocking filters tend to blur image details, which are more sensitive to human visual systems. In video coding, discontinuities are predominated across block boundaries between consecutive frames when coded at a very low bit rate. The Motion Compensated Prediction (MCP) may propagate blocking artifacts that are more visible to human eyes, specifically more in flat regions than complex regions.The human Visual System (HVS) is more delicate to unexpected changes in signals (pixel estimation) of video outlines, specifical pixels with high or low differentiation esteems than its adjoining pixels. This sudden change diminishes the emotional nature of the casing and eventually upsets the total de-hindering activity. To eliminate such sort of discontinuities, one can apply mean channel. p11) is abrupt signal; then its value should be near to max(S) and the difference is always less than or equal to (Δ) and more significant than N pixels in set (S).Let , where (Td) is the threshold to calculate the dissimilarity that occur between two adjacent frames or pixels. If (p11), then it is observed as a pixel with considerable signal value, and it will be replaced with the mean of all the eight neighbouring pixels to remove undesired noise as in (We consider (N = 8), Δ = 2.5), and  &  pixels across level and vertical square limits as demonstrated in The value of threshold will be large for smooth regions to preserve all edges of blocks and remove the artifacts only in the smooth regions. In contrast, in the non-smooth regions, QF should be small enough to maintain the images’ vagueness effects.i = 173.5*QF (0.05)) is utilized to catch edges of decoded video edges and ’i’ got from ith Pixel Vector. Expecting obstructing curios are even across block limits . Henceforth the vertical obstructing ancient rarities are likewise diminished by just pivoting the picture by 900.A limit is needed to counter discontinuities along block limits. We propose a limit worth to figure PV containing dissimilarities.C) is the inertia limitation set of PV with (jxj) squares to such an extent that Regions categorizes primarily of three types, namely flat, intermediate, and non-smooth in view of the level of perfection. To confirm the attributes of the pixel vector, movement work (Ψ) has been acquainted with measure the nearby variety of PV.s)&(Tc) for the smooth area and complex locales, individually. The area arrangement is addressed in s)&(Tc) tentatively set to ’1’ and ’2’ for an effective differentiation between various districts.Isolation of districts is done with two separate limits  if the activity functions (Ψ) is less than the threshold value (Ts) as shown in Let R is flat region A is introduced.If the above condition is true, calculate the absolute difference between boundary pixels. To verify whether the block is artifact-free or not, another threshold TTNA = 0.035 Ti. In the event that (TFE (TFE = 0.56Ti) is the threshold for smooth edges, where smoothing filters are not suitable.If ent that  isn’t fuIf the above condition is true, PV is altered by .p4′, p5′, p6′, p7′, p8′, p9′, p10′, and p11′ are the eight changed pixels of PV. For a smooth area, every pixel in PV is reliant on different pixels. Henceforth smoothing channel planned in (Else the PV changes by .p4′=p4−anned in .pi(j)′=Similarly, intermediate and non-smooth regions are determined and pixels have been changed. Filtering mode choice is utilized to deal with obstructing relics in the various locales. For the non-smooth locale, a straightforward channel is utilized while for a smooth area solid channel is utilized . Interestingly, the middle channel is liked for the moderate locale, as clarified in In packed pictures or casings, blocks across edges contain outwardly irritating curios most popular as a ringing antique. The ringing relics are delivered because of the deficiency of high-recurrence segments or loss of accuracy in high-recurrence parts. It debases the perceptual nature of picture or video edge somewhat as far as Peak Signal to Noise Ratio, and it is difficult to outperform ringing impact with 1-D channel. We apply a 2-D directional filter, as demonstrated in w is weighted function given in Let f the pixel with a ringing artifact and need to be adjusted. The updated pixel is defined by f′=To extricate flicker in the sequential edges and gauge bending because of gleam ancient rarities, a D-flicker metric is presented. The method to discover gleam block is clarified in Algorithm IALGORITHM-1: PSEUDO-CODE FOR FLICKER DETECTION ALGORITHM:Input: Create data set of videos divided into frames ), where i ∈ [1 : nF];Input: i;  Input video frames VOutput: f;  Extraction of flicker artifacts in video frames Begin for i=2:nF  1    Input: Current frame (IF), Next frame (IFnext), Current2     Processed3     Frame (IFp), Next Processed frame (IFnextP).4     DiffF = IFnext -IF;5     DiffF_P = IFnextP -IFP;6     DFlicker = max {0, abs (DiffF_P -DiffF)};End  Endloss (μ) in de-hindered casings of decoder (MPEG-4). The estimation of (μ) is tentatively set to 0.56 for better execution. The flash expulsion channel calculation is characterized in the accompanying pseudo-code and is appeared in Algorithm II.To eliminate flicker artifacts, we have proposed a 2-D worldly filter that is fit for controlling PSNR-BALGORITHM-II: PSEUDO-CODE FOR FLICKER REMOVAL ALGORITHM:Input: Create data set of videos divided into frames ), where i ∈ [1 : nF];Input:   Input DFlicker video framesFlickerVF; (μ)=0.7;  DOutput: p;  Removal of flicker artifacts in video frames  = VBegin for i=2:nF 1.   Input: Current frame (IF), Next frame (IFnext), 2.   IF = double (IF); IFnext = double (IFnext); 3.   IFn = (μ) .* IF + (1 - μ).* IFnext; 4.   IFfinal = uint8(IFn); End  End To analyses the performance of the proposed method, we consider full-referenced performance metrics. In this paper, we consider the following metrics for performance analysis of de-blocked video sequences.i is the blocking pixels in a particular frame of V video sequence as shown in H and Nv are equal to 8 and no boundary pixel on either side of horizontal and vertical block boundaries is similar to B = 4. The thick lines represent block boundaries. Let Pv are the pixels across the vertical block boundary. Similarly, Ph is the pixels across the horizontal block boundary. On the other hand, PhB and PvB are the set of pixels that are not lying across horizontal and vertical block boundaries, respectively.The first parameter used for performance evaluation is a block sensitive image or video quality metric called Peak Signal to Noise Ratio-Blocking artifacts effect (PSNR-B) . BlockinThe pixel set in d) for boundary pixels as well as pixels which did not lying near to block boundaries(μdB) asAfter defining boundary pixels, we have to calculate mean squared difference  is calculated asGBIM stands for Generalized Block-edge Impairment Metric , 41. We 1,p2……pcWc} where pcj is the jth column of the video frame with Wc as the width of the frame. The interpixel difference across the vertical block boundaries is given byLet us consider a DCT coded frame as shown in Assume that frame is divided into (8x8) block size which is commonly used in standard video compression formats. Let the output of the prosed Human visual system (HVS) with local weights BM to generate Blocking artifact strength (BAS) and is given byv is an inter-pixel difference across vertical block boundaries. Similarly, Horizontal GBIM (GBIMH) is calculated and the final value will be added for each frame.The GBIM metric is calculated by using the HVS parameter as given belowF) of a video is calculated asx and μy are the mean values of pixel element x and y in a frame whereas σx and σy are the standard deviation of x and y respectively and Ca and Cb are stabilizing constant.MSSIM represent mean structural similarity indices , 37, 40 To approve the presentation of the proposed strategy, video guidelines like MPEG-4 procedure were utilized. The PP-AFT strategy’s exhibition is contrasted and existing methods Wang et al.  and MPEGhttps://media.xiph.org/video/derf/), with standard translating calculations like MPEG-4 just as Wang et al. [This work has been executed on various video arrangements like meetings, akiyo, versatile, suzie taken from derf’s assortment accessible without permit for research reason  blocks. These blocks are then pre-processed to eliminate quantization error. Blocking artifact detection and removal is addressed, and finally, a flicker detection and removal algorithm is introduced.Due to the lack of resources, some features on this research work are compromised that can add more strength to its functionality in terms of both time and cost.High-definition videos need to be implemented with hybridization of the proposed algorithm, which will help to find more accurate results for HD contents.One can apply soft computing techniques, which provides efficient approach to remove these artifacts with low complexity.We have proposed a novel adaptive threshold-based pre- and post-processing technique to reduce artifacts in BDCT video frames. The PP-AFT method is based on a 2-D directional filtering approach to remove ringing artifacts while preserving the edges. The Human Visual System is also incorporated to improve the visual quality of the images or frames. The frames’ characteristics across block boundaries are efficiently obtained by removing blocking artifacts using three filters based on three regions . The smooth region filtering has been achieved efficiently using a considerable threshold value. At the same time, the small threshold value is proposed for non-smooth regions, whereas for intermediate regions, two different filters are proposed using different thresholds to maintain a balance between smooth and non-smooth region.Objective and subjective experiments, have been performed to validate the performance of the proposed work. To analyze the proposed technique, PSNR-B is evaluated for an objective, and for human visual perception, MSSIM and DBIM indices have been used. It is observed that the PP-AFT approach provides a significant improvement in the perceptual quality of images or frames and efficiently removes blocking artifacts. MSSIM values obtained from the PP-AFT method is very close to original images.The PP-AFT method is also suitable for real-time applications of frames decompression due to its low computational complexity shortly. Soft computing methods can also optimize the results. This work can be further extended by considering H.265/ HEVC decoding technique for High Definition Video sequences for high-speed internet network like 5G/LTE using DCT and Discrete Sine Transform (DST) with variable block size between (4x4) and (32x32).


As a muscular hydrostat, the tongue undergoes complex deformations during most oralbehaviors, including chewing and drinking. During thesebehaviors, deformations occur inconcert with tongue and jaw movements to position and transport the bolus. Moreover, thevarious parts of the tongue may move and deform at similar timepoints relative to the gapecycle or they may occur at different timepoints, indicating regional biomechanical andfunctional variation. The goal of this study is to quantify tongue deformations duringchewing and drinking in pigs by characterizing intrinsic changes in tongue dimensions across multiple regions simultaneously. Tongue deformations aregenerally larger during chewing cycles compared to drinking cycles. Chewing and drinkingalso differ in the timing, relative to the gape cycle, of regional length and width, butnot total length, deformations. This demonstrates functional differences in the temporaldynamics of localized shape changes, whereas the global properties of jaw–tonguecoordination are maintained. Finally, differences in the trade-off between length andwidth deformations demonstrate that the properties of a muscular hydrostat are observed atthe whole tongue level, but biomechanical variation  at the regional level exists. This study provides new critical insights intothe regional contributions to tongue deformations as a basis for future work onmultidimensional shape changes in soft tissues. Like other muscular hydrostats, the muscular tongue of vertebrates has a complexarrangement of muscles that allows it to deform in myriad ways while maintaining a constantvolume . The musAlthough static interpretations of deformations can be made from the muscle anatomy ofmuscular hydrostats , the dynRelationships between tongue form and function have been studied with magnetic resonanceimaging (MRI) studies of human speech e.g.,  andswalMore invasive sonomicrometry studies in the pig demonstrate regional differences in themagnitude and timing of tongue deformations relative to opening and closing of the jawduring feeding and drinking ,2009. DSusscrofa, Linnaeus 1758). First, we describe overall patterns of tongue deformationduring chewing and drinking gape cycles. Next, we compare the magnitude of these changes totest a series of hypotheses about tongue deformations during chewing and drinking. Wehypothesize that during chewing tongue length and width will deform more than duringdrinking. This hypothesis is based on our previous work demonstrating that tongueprotraction–retraction movements are more pronounced during chewing  and drinking, in the pig  with additional soft tissue markers in thetongue . Followi® calibration cube were imaged in each fluoroscopy view to undistort andcalibrate the videos, respectively, in XMALab  mounted on the output ports of 2 synchronized OEC-9000 fluoroscopes. Radiation technique averaged 100 kVp and 4.3 mAacross trials. Fluoroscopy videos and a separate webcam recording at 30 fps weresynchronized and saved in Qualisys Track Manager motion capture software. Trials of eachbehavior were recorded over the course of a week. Prior to each recording session,perforated metal sheets  and a customLegon XMALab . At the Fluoroscopy videos were processed according to the XROMM workflow in XMALab to (1) track2D points of each marker in the undistorted and calibrated field of view, (2) calculatetheir respective 3D coordinates, and (3) quantify and filter  rigid body transformations of the skull and the jaw . The marinvivo CT scan when the tongue was positioned in a relaxed neutral positioninside the oral cavity  of a3-axis joint coordinate system following The magnitude and timing of regional tongue deformations were determined relative to thegape cycle and gape cycle phases. Gape cycle dynamics were determined by the direction andacceleration of jaw rotation about the cycle and ΔWcycle were calculated from the correspondingregional cycle maximum and minimum lengths and width, respectively  and 90 drinking cycles(40 for Pig 20 and 50 for Pig 21). All statistical analyses were performed in R version3.6.1 . The lmeFinally, we assess whether there are qualitatively observable patterns of compensatorychanges in AP regional deformations and corresponding ML deformations to suggest that theproperties of a muscular hydrostat are preserved at regional levels for both behaviors.For example, we would expect high AP deformation is associated with low ML deformation andlow AP deformation with high ML deformation. Note that this assessment is exploratory innature given that we do not assess the changes over time  andwe do not assess changes in the dorsoventral dimension.Total tongue length changes little within chewing and drinking cycles, neverlengthening >1.1× resting length. Compared to drinking, chewing involves overalllarger total tongue deformations, and even more pronounced regional deformations . During The magnitude of changes in normalized regional widths throughout the gape cycle issimilar during chewing and drinking, staying between 0.5× and 1.5× resting length acrossall regions, whereas behavior-specific patterns in timing are evident . For botcycle) are significantly larger during chewing than drinking due tostatistically higher maximum values (except R2) and lower minimum values  was also significantly higher forchewing than for drinking for all tongue regions, and the ranges of width deformationsalso all occur within 0.5–1.5× resting length  and R4 (27.67% versus 17.12%) . FinallyThe other major limitation is that the distance  betweenmarkers reflects absolute linear distance between consecutive markers. Although this makesa solid approximation of length and width changes, it does not account for the otherpotential deformations that can also influence the distance between markers. For example,a decrease in distance is likely from shortening, but may also be influenced by bending,which may decrease the linear distance without decreasing actual muscle length.Specifically, minimum widths captured in the present dataset may include deformationsother than shortening, such as ML bending or “rolling,” in which the distance betweenmarkers decreases but the actual tongue surface width is constant or may even increase.Our interpretations of the data cannot currently account for these potential confoundingfactors. However, this study does provide new critical insights into the regionalcontributions to tongue deformations and serves as the basis for future work onmultidimensional shape changes during oral behaviors.obab012_Supplementary_DataClick here for additional data file.


Multimodal sentiment analysis (MSA) aims to infer emotions from linguistic, auditory, and visual sequences. Multimodal information representation method and fusion technology are keys to MSA. However, the problem of difficulty in fully obtaining heterogeneous data interactions in MSA usually exists. To solve these problems, a new framework, namely, dynamic invariant-specific representation fusion network (DISRFN), is put forward in this study. Firstly, in order to effectively utilize redundant information, the joint domain separation representations of all modes are obtained through the improved joint domain separation network. Then, the hierarchical graph fusion net (HGFN) is used for dynamically fusing each representation to obtain the interaction of multimodal data for guidance in the sentiment analysis. Moreover, comparative experiments are performed on popular MSA data sets MOSI and MOSEI, and the research on fusion strategy, loss function ablation, and similarity loss function analysis experiments is designed. The experimental results verify the effectiveness of the DISRFN framework and loss function. Multimodal sentiment analysis (MSA), as an emerging field of natural language processing (NLP), aims to infer the speaker's emotion by exploring clues in multimodal information –3. Many For multimodal representation learning methods, since multimodal data is usually a sequence with different feature dimensions, long-short memory neural network (LSTM) is a powerful tool to deal with such problems . TherefoIn this paper, the improved JDSN is adopted to learn the joint representation of modality-invariant and modality-specific of all modes in the common–special subspace. The former aims to map all the modes of discourse to the common subspace to shorten the distance between modes to effectively reduce the extra burden of fusion work. The latter aims to extract special representation from each mode as complementary information. Then, the combination of two representations can fully use the complementary information between modes. In addition, the modal interactions were mostly obtained by feature connection fusion in early work . HoweverA multimodal sentiment analysis framework (DISRFN) is proposed in this study. It can perform the fusion of various representations dynamically while emphasizing learning invariant and specific joint representations of various modes.A new loss function is designed, which can improve the effect of semantic fusion clustering whilst assisting the model in learning the target subspace representation effectively.The performance analysis experiments of MSA tasks is designed on the benchmark data sets MOSI and MOSEI. The results confirm the advancement of the DISRFN model and fusion strategy, the effectiveness of the loss function, and the rationality of similarity loss function selection.In summary, firstly, the applied DSN in this paper is improved in the aspects of the following: (1) The mode of DSN is extended; (2) The orthogonal constraint loss between special representations of different modes is additionally considered See ; (3) AdvThe remainder of this paper consists of the following parts. In In multimodal sentiment analysis, the mainstream multimodal learning methods include multimodal fusion representation and multimodal representation learning, which will be discussed in this section.In recent years, some complex and efficient fusion representation mechanisms have been gradually proposed. Amir Zadeh et al. put forward TFN to obtain the trimodal fusion representations by using the outer product . On thisMultimodal representation learning is mainly divided into two types, namely, common subspace representations and factorised representations. The two types of study on common subspace representations amongst modes are the correlation-based model and adversarial learning-based model. In terms of a correlation-based model, Shu et al. proposed an extensible multilabel canonical correlation analysis (sml-CCA) for cross-modal retrieval . Kaloga In general, the proposed framework is mainly used to study the trimodal data. l ∈ ℝtl×dl, Sv ∈ ℝtv×dv, Sa ∈ ℝta×da. Amongst them, tm and dm  represent the length of discourse and the dimension of the corresponding feature, respectively. Given this data sequence, the study aims to predict the emotional state of the predefined set. This emotional state is a continuous dense variable y ∈ ℝ. In addition, to effectively use multimodal data, linguistic (l), visual (v), and auditory (a) trimodal feature sequences, they should be aligned with emotional state label y.Moreover, the discourse data are divided into N sequences composed of segment S to facilitate detecting emotion in video by using multimodal data. Each segment S includes three low-level feature sequences in linguistic (l), visual (v), and auditory (a) modes. These feature sequences are represented as SThe framework of DISRFN is shown in Sv, Sa) in visual (v) and auditory (a) modes to obtain the underlying features of the sequence. Its output includes the hidden representations of LSTM end state, namely, Fv and Fa, as follows:θvLSTM and θaLSTM refer to the parameters of sLSTM on visual and auditory modes.Firstly, the stacking bidirectional long-short time memory neural network (sLSTM) is used to map the feature sequence (Sl) in linguistic mode, most linguistic features are embedded through Glove [Fl of text is obtained through the pretraining BERT model, as follows:θlBERT refers to the parameter of the BERT model.Secondly, for the text feature sequence  is used to unify mapping these features to Ec is used to learn invariant representation in the common subspace to narrow the gap in the heterogeneity between modes [Emp is used to capture the specific representation in a specific subspace. The process is as follows.In this part, based on the improved JDSN, the unified mapping representation of each mode is factorized into two parts, namely, modality-invariance and modality-specificity. Amongst them, the sharing encoder en modes . The speOm of each mode, the mode-sharing encoder Ec (weight sharing) is used to obtain modality-invariant representation (hmc), and the mode-specific encoder Emp is used to extract modality-specific representation (hmp), as follows:θc refers to a parameter of mode-sharing encoder; θmp refers to a parameter of mode-specific encoder; Ec has the same structure as that of Emp, which is composed of a dense connection layer activated by sigmoid function.Firstly, after obtaining the unified mapping vector hmp and hmc are generated through feedforward propagation of neural network, and the joint domain separation representation is obtained through vector addition “+”, as follows:hm refers to the joint domain separation representation of mode m, and it has the feature representation of shared subspace and specific subspace characteristics.Then, hidden layer vectors After obtaining the joint domain separation representation of each mode, it is necessary to fuse each representation to obtain the interaction information of each mode.al) using the correlation weight between unimodal vectors. Trimodal dynamic layer is constructed through weighting trimodal information vectors  by the correlation weight between unimodal vectors. Finally, three dynamic layers are used for vector connection and fusion to realize the dynamic fusion of multimodal features in HGFN. This hierarchical modeling method is more conducive to exploring the interaction between modes [θHGFN refers to the parameters of HGFN. Then, the predictive neural network (P) is used for prediction, as follows:θPre refers to the parameter of the predictive network. Moreover, the specific parameters of the model are described in the experimental section.As shown in en modes . Therefoα, β, γ, and η refer to weights of the interaction. They determine the contributions of each loss Ldiff, Lsim, Lrecon, and Ltrip to total loss Ltotal. In addition, each loss is analyzed and introduced in the remaining section.A joint loss function is newly set to effectively learn the network model, as follows:c and specific-encoder Emp to perform encoding representation to different aspects, that is, modality-invariant and modality-specific representations. Soft orthogonality constraint is defined as follows.Some studies have shown that a nonredundant effect can be achieved by applying soft orthogonality constraint to two representation vectors , 41. TheHmc and Hmp are set as the two matrices, respectively. The rows of the two matrices correspond to invariant representation hmc and specific representation hmp of mode m in each batch of data, respectively. The orthogonality constraint of the modal vector is calculated as follows [|| · ||F2 refers to squared Frobenius norm.When training a batch of data,  follows :(9)Ldiffsim) used to constrain shared subspace can reduce the difference in the heterogeneity between the shared representations of different modes [Similarity loss  refers to the empirical expectation vector of sample X; Ck(X) refers to the vector of all k-order sample centre moments in the X coordinate. follows :(10)CMDXIn this paper, the similarity loss is calculated by summing the CMD distances of the shared representations of every two modes. Its representation is as follows:Moreover, the reason for selecting CMD as the similarity loss will be discussed in Experimental part 5.4.Dm is used to reconstruct the joint domain separation representation vector hm of mode m, and the output of reconstruction is hm and || · ||22 refers to squared L2-norm.When soft orthogonality constraint is enforced, the risk of specific encoder learning trivial representation exists. However, the reconstruction loss can be added to ensure that the encoder can capture the details of each mode to solve these problems . Initial follows :(12)Lrectrip, and the distance between different discourse segments is maximized [In the fusion representation of joint domain separation representation vector, to ensure the high-level relationship of the similarity between all projects, the representation distance of discourse segments with similar semantics between different modes is minimized through cosine triplet-margin loss Laximized .hl, hv+, hv−) is established. Amongst them, visual representation hv+ is positively correlated with linguistic representation hl in semantics. At the same time, visual representation hv− is the contrary. Therefore, the cosine triplet-margin loss of linguistic mode is shown as follows [hm+, hm− refers to the joint domain separation representation vector of mode m; “margin = 1” is a boundary parameter.For example, in linguistic and visual modes, a triple representation ( follows :(13)LtriIn the same way, the cosine triplet-margin loss of visual mode and auditory mode can be described as follows:Based on formulas –15), th, th15), Nb discourse data in one batch, this loss calculation is as follows:yi refers to the actual emotional label; The mean square error (MSE) is used as the task loss of the network to predict continuous dense variables. For In this section, the required data sets, evaluation index, and experimental details  are described.The data set is introduced in this section. This data set includes two parts, namely, CMU-MOSI and CMU-MOSEI.CMU-MOSI data set: this data set is a collection of monologues on YouTube, including videos with 93 comments from different speakers. These common videos consist of 2199 subjective discourses. These discourses are manually marked with continuous opinion scores in the range of −3 to 3. Amongst them, −3/+3 represents strong negative/positive emotions. A total of 1283 segment samples are used for training, 229 segments are used for verification, and 686 segments are used for testing.CMU-MOSEI data set: it is an improved version of MOSI; it includes 23453 annotated discourse segments, which are from 5000 videos, 1000 different speakers, and 250 different topics. A total of 1283 segment samples are still used for training, 229 segments are used for verification, and 686 segments are used for testing.1 in many studies [The problems on multimodal signal  acquisition and modal data pretreatment are solved based on CMU-Multimodal SDK studies . This toThis experiment is a regressive task. Therefore, the mean absolute error (MAE) and Pearson correlation coefficient (Corr) are adopted to measure the test results. In addition, the classification index is considered in the experiment, including five-classification accuracy (Acc-5) in affection domain , two-classification accuracy (Acc-2) including positive and negative emotion (p/g), and F1score (F1-Score).α∈{0.3, 0.4}, β∈{0.7, 0.8, 0.9, 1.0}, γ∈{0.1, 0.2, 0.3, 0.4, 0.5}, η∈{0.01, 0.1} and drop∈{0, 0.1, 0.2, 0.3, 0.4}; the hidden layer sizes of the representation and predictive network can be reviewed from the following: Hid∈{128, 256}, P_h∈{50, 64}.This method is tested on Pytorch in this section. The grid searching of hyperparameter is performed in a data verification set to identify appropriate hyperparameter, and the best model and hyperparameter are saved. In grid searching, limited option sets for setting hyperparameters are as follows: = 20, batch_size = 16, and learning rate of 0.0001 are used to train the network. The grid searching results of all data sets are shown in In the iterative optimization process, Adam optimizer with max epoch 2 and COVAREP [dv of the visual feature are 47 and 35, respectively, and the dimension da of the auditory feature is 74.Manual feature extraction of video and audio: for CMU-MOSI and CMU-MOSEI, Facet COVAREP  are usedl with dimension of 768 [v and Fa are obtained based on sLSTM.Discourse-level feature extraction: for linguistic mode, because the BERT model has text embedding and representation functions, the pretraining model of BERT is directly used to extract linguistic features. Its discourse-level feature is represented as feature representation Fn of 768 . And thel, Fv, and Fa to an output Om with the unified dimension size.Unified representation mapping: MLP is adopted to map linguistic, visual, and auditory representation vectors FOm is input to sharing encoder and specific encoder to obtain hidden layer representation hmc, hmp. And then, an improved joint domain separation representation hm is obtained through vector addition (hmp+hmc).Improved joint domain separation representation: Fusion inference: the joint domain separation representation vector is sent to the HGFN to perform fusion and prediction tasks.Calculating loss function and training: loss function is calculated to train the neural network and make cyclic iteration.This section mainly introduces the experimental process, the specific experimental steps are as follows:Model comparison experiments, research on fusion strategy, research on loss function ablation, and research on similarity loss selection are designed in this section. All experiments are discussed by combining visualization and quantitative analysis.In the comparison experiment, some classical models  are reproduced. In addition, some derived fusion model based on LSTHM  is desigTables Moreover, it can be seen from the “CPU Clock” items in Tables -derived fusion model, various evaluation indexes are improved significantly by the model using BERT (DISRFN and MISA). It proves that the application of the BERT method is reasonable. Moreover, compared with the MISA model using BERT, the proposed model still has a slight advantage. The difference is probably caused by different fusion strategies. The comparative experiment is carried out in the next section to further discuss the effectiveness of the fusion strategy of this model.In In this section, a fusion strategy comparison experiment is designed in the MOSI data set to verify the effectiveness of the HGFN fusion strategy. The improved JDSN component remains unchanged in the experiment, and the fusion component is replaced with Multi-Attention Fusion (AttFusion), vector concatenation fusion (Concate), dynamic fusion net (DFN), and other strategies. Then, the results are concluded, as shown in The results shown in As shown in The loss functions of various components discussed in α, β, λ, η) of other components except for the basic task loss Ltask, and the best performance model parameters are saved. Moreover, to intuitively observe the effects of various loss functions on the model results, the fusion representation of MOSI test samples is visualized by T-SNE, as shown in An ablation experiment is designed in this section. The network is retrained after obtaining a zero setting of the loss weights  , maximumThe reasons are summarised in the following points: (i) CMD can directly perform exact matching of the high-order moment without expensive distance and kernel matrix calculation; (ii) compared with CMD, DANN obtains modal similarity through minimax game using discriminator and shared encoder. However, in adversarial training, additional parameters are added, and fluctuations may be encountered in training. Moreover, through the observation of joint form (the last three columns), the effect of similarity loss with CMD is better than that of the loss without CMD but worse than that of single CMD loss. This finding indicates that the increase in computation cost reduces the efficiency of network learning and further verifies the rationality of selecting CMD as similarity loss.This paper studies multimodal emotion analysis. In the research, we have the following findings: (1) feature representation with more comprehensive information can reduce the burden of fusion network; (2) the redundant information of each mode can be used more effectively by jointing modality-invariance and modality-specificity representations of each mode; (3) simple dynamic fusion mechanism can obtain the interaction between modes more efficiently. Thus, this study puts forward a multimodal sentiment analysis framework consisting of two parts, namely, improved JDSN and HGFN. Firstly, modal invariant-specific joint representation of each mode is obtained through an improved JDSN module to effectively utilize the complementary information amongst modes and reduce the heterogeneity gap between modes. Then, the joint representation of each mode is input to the HGFN for fusion to provide input for the prediction network. Moreover, a new combined loss function is designed to encourage the DISRFN model to learn the representation of expectation. Finally, the performance analysis experiment is carried out on MOSI and MOSEI data sets, obtaining acceptable results. In practice, the multimodal data usually have an unbalanced phenomenon, which will lead to the task bottleneck of the model. However, the study does not consider this issue. Therefore, we plan to study the problems of multimodal imbalance in the future.


Color vision tests use estimative of threshold color discrimination or number of correct responses to evaluate performance in chromatic discrimination tasks. Both approaches have advantages and disadvantages. In the present investigation, we compared the number of errors during color discrimination task in normal trichromats and participants with color vision deficiency (CVD) using pseudoisochromatic stimuli at fixed saturation levels. We recruited 28 normal trichromats and eight participants with CVD. Cambridge Color Test was used to categorize their color vision phenotype, and those with a phenotype suggestive of color vision deficiency had their L- and M-opsin genes genotyped. Pseudoisochromatic stimuli were shown with target chromaticity in 20 vectors radiating from the background chromaticity and saturation of 0.06, 0.04, 0.03, 0.02, 0.01, and 0.005 u’v' units. Each stimulus condition appeared in four trials. The number of errors for each stimulus condition was considered an indicator of the participant's performance. At high chromatic saturation, there were fewer errors from both phenotypes. The errors of the normal trichromats had no systematic variation for high saturated stimuli, but below 0.02 u’v' units, there was a discrete prevalence of tritan errors. For participants with CVD, the errors happened mainly in red-green chromatic vectors. A three-way ANOVA showed that all factors  had statistically significant effects on the number of errors and that stimulus saturation was the most important main effect. ROC analysis indicated that the performance of the fixed saturation levels to identify CVD was better between 0.02 and 0.06 u’v’ units reaching 100%, while saturation of 0.01 and 0.005 u’v’ units decreased the accuracy of the screening of the test. We concluded that the color discrimination task using high saturated stimuli separated normal trichromats and participants with red-green color vision deficiencies with high performance, which can be considered a promising method for new color vision tests based in frequency of errors. CCT and CAD are color vision tests that implemented methods to estimate color discrimination thresholds along different chromatic vectors in chromaticity color spaces7. Both tests apply a forced-choice method in the psychophysical task to feed a staircase that controls the chromatic distance between the target and background chromaticity coordinates. The staircase procedure can be very time-consuming since it requires several reversals or trials to complete. Nevertheless, the completion of the staircase procedure results in the estimation of color discrimination thresholds at different chromatic vectors, which can be taken as indicators of the color vision performance for the perceptual task. Fast approaches such as the trivector test from CCT, which the chromatic discrimination task occurs only in the chromatic axis of the color confusion lines. CCT trivector protocol has been used to shorten the test duration. The disadvantage of these short approaches is the reduction in the number of chromatic axes to be tested, especially in case of acquired color vision deficiency (CVD) evaluation.Different research groups have been developing computerized tests for a psychophysical color vision evaluation, such as the Cambridge Colour Test (CCT) and the Colour Assessment Diagnosis (CAD), applying different testing strategies to quantify congenital and acquired color vision deficiencies8. The authors used a computerized test showing pseudoisochromatic stimuli with suprathreshold chromatic contrast for normal trichromats. The observers were asked to identify the shape of a target or its absence. They used eighty combinations of lightness, saturation, and hue and recorded the number of errors during the test to measure the individual color vision performance. In that investigation, the chromatic discrimination was applied to compare diabetic patients with healthy trichromats. Instead of estimating thresholds, the authors investigated the frequency of errors in target identification, and they observed that the performance of the diabetic patients was worse than healthy trichromats and that the color vision loss was consistent with a diffuse acquired color vision loss.While CCT and CAD were designed to quantify the color discrimination thresholds of a subject, another interesting approach to evaluate color vision was suggested by Shin et al.8, named SNU color test, is similar to many printed color vision tests. In the same test, they used stimulus with different combination of lightness , chromaticity  and saturation . For a given saturation level, the chromatic vector size was equal distance from the central white point in the HSV color space. The performance of the participant was based in the number of errors of the observer.The computerized proposal of Shin et al.8 approach could be applied using fixed saturation levels and evaluated the performance of each saturation level to identify the presence of CVD. The comparison of the performance of different fixed saturation stimulus levels to screen the color vision deficiency can be useful to design fast color vision tests using only the smaller fixed saturation level with a desired criterion of screening. In addition, the use of C-gap as target enables the execution of a simple positional localization task which is more related to retinal and primary visual cortex processing than the target form discrimination that is processed in higher visual areas9. Testing participants with congenital CVD is very important to test the specificity of the test to identify predictable error during testing condition using colors that confuse participants with color vision deficiencies.In the present study, we intended to explore this approach using more controlled stimulus conditions , and testing participants with congenital CVD in fixed saturation levels. We consider that these additions would improve the comprehension of this approach in order to design other color vision tests based in the estimating of frequency of errors. We think that the Shin et al.An informed written consent was obtained from all the participants after being informed about the objectives and all steps of the investigation. This study was carried out following the recommendations of National Health Resolutions, the National System of Research Ethics of Brazil, conducted under the Declaration of Helsinki 1964 and posterior updates. Furthermore, we received the approval of the Ethical Committee for Research in Humans of the Núcleo de Medicina Tropical, Universidade Federal do Pará (report # 2.274.540), Brazil.Our sample comprised 28 normal trichromats  and eight participants with congenital red-green CVD . All participants had no history of systemic or neurological disease that could impair any visual function and no medication affecting the central nervous system. In addition, all had normal or best-corrected visual acuity to 20/20.10 to confirm that the CVD has a congenital background. The method amplifies a 300 bp fragment containing the exon 5 of both visual opsins genes, using polymerase chain reactions (PCR) and the primers 5’TCCAACCCCCGACTCACTATC and 5’ACGGTATTTTGATGTGGATCTGCT, followed by a restriction endonuclease reaction. DNA samples were collected and extracted from a buccal brush and purified using the Gentra Puregene Buccal Cell Kit , following the manufacturer’s protocol. PCRs were carried out using High Fidelity Platinum® Taq Polymerase, 10 × High Fidelity Buffer, MgCl2, 10 mM GeneAmp dNTPs , and 20 mM primers in 50 µl reactions, and PCR conditions were described elsewhere11. The amplified fragments were incubated with the restriction endonuclease Rsa I , which recognizes and cleaves the exon 5 of the L-opsin gene, resulting in two smaller fragments with about 100 bp and 200 bp, but do not cleave the M-opsin gene, resulting in a single ~ 300 bp fragment. The resulting PCR products were visualized in 1.0% agarose gel electrophoresis. Trichromat individuals, with both L and M genes, were identified by the presence of three bands . This method identifies the absence of M-opsin or L-opsin gene, it cannot separate when the individual is dichromat or anomalous trichromat. Deuteranope and deuteranomalous individuals (no M-opsin gene) display the two smaller bands, while protanope and protanomalous individuals, with no L-opsin gene, display only one ~ 300 bp band. The DNA sample from a known trichromat was used as a control.First, all participants were evaluated using CCT  in the 20 chromatic vectors protocol to estimate color discrimination thresholds. For all subjects whose ellipse’s major axis was lengthier than the upper limit of the normative data, we performed the genotyping procedure to identify the genomic presence of the M- and the L-opsin genes, based on the method proposed by Neitz and Neitz2. Each mosaic had a C-shaped subset of circles in the mosaic center whose chromaticity differed from the remaining mosaic, forming a target-background arrangement , using a graphic library of the ViSaGe system . The stimuli were 6.5° circular pseudoisochromatic mosaic composed of 450 circles ranging 10 radii from 0.07° to 0.23° of visual angle. The luminance of the circles was randomly defined with one of six equally spaced values between 8 and 18 cd/cmWe used CIE 1976 color space to define the stimuli with parameters similar to those used in CCT. During the test, target chromaticity varied along 20 vectors in the CIE 1976 color diagram, radially distributed around the background chromaticity coordinate, and equally spaced in 18°. Six saturation conditions (chromatic vector) were tested . The saturation values represent the distances from the coordinate of background chromaticity. The target was shown during 2 s and followed by a dark screen. The subject’s task was to inform the orientation of the C-gap  by pressing the corresponding button in a 4-button box (CRS). Each target chromaticity was shown in 4 trials, totaling 480 trials . The target orientation and the saturation condition of the target were randomly chosen. The indicator of the participant’s performance was the number of errors. The greater the number of errors, the worse the performance.The chromatic discrimination across stimuli conditions was estimated by the number of errors. A three-way ANOVA with Geisser-Greenhouse correction was conducted to compare the main effects of color vision phenotype , chromatic saturation (six levels), chromatic vector (20 levels), and their interaction in the number of errors. Partial eta squared was calculated the effect size of the main effects. We considered a confidence level of 95% for all the statistical procedures.We applied a Receiver-Operating Characteristic (ROC) analysis to evaluate the accuracy of each fixed saturation level to screen the presence of color vision deficiency. For that, we considered the cumulative distribution of correct responses from normal trichromats as “correct responses” and the cumulative distribution of the correct responses from the color vision deficient participants was considered the “false alarms”. The area under the curve (AUC) was considered the quantification of the accuracy of the classifier to screen the presence of color vision deficiency. A MATLAB  routine was written to proceed with the statistics.Figure Figure In general, we observed that at high chromatic saturation conditions, there were fewer errors. The number of errors of the sample increased as the chromatic saturation of the target decreased, following a negative power trend, as shown in Fig. 12. Then, it is reasonable that trichromats’ errors show first in chromatic axis close to the tritan color confusion line.Figure For participants with CVD, the distribution of the errors across the chromatic vectors occurred mainly in red-green chromatic vectors (0°–342° and 162°–180°) . The results of the analysis of variance are shown in Table The results of the ROC analysis are shown in Table The present study demonstrated that the proposed protocol for psychophysical chromatic discrimination at fixed saturation levels could distinguish the data obtained from subjects with normal trichromacy and people with congenital CVD. Furthermore, stimuli with saturation ranging between 0.06 and 0.02 u’v’ units showed high accuracy for the identification of CVD allowing their classification in either normal or deviant color vision.4. Additionally, we have successfully used these stimuli designs to evaluate color vision changes in patients with genetic defects such as Duchenne Muscular Dystrophy13, Leber's Hereditary Optic Neuropathy15, and patients with acquired defects such as diabetes16, retinal toxicity by hydroxichloroquine17, and mercury vapor18 in studies that validated this methodology.Pseudoisochromatic stimuli have been used extensively in color vision tests because its design, including luminance and spatial noise, is very efficient in masking the chromaticity differences between the target and background20, leading to a reduction of their sensitivity and reliability.Many color vision tests use number of errors in the identification of the target in pseudoisochromatic plates such as the Ishihara and HRR plate tests, but those tests show several limitations: (1) They investigate the chromatic discrimination in few chromatic axes; (2) Since they are printed, the precision control of color and saturation is limited in short color ranges; (3) They are fixed in plates making it easy to memorize after several presentations; (4) The psychophysical task is a pass-fail procedure generating a nominal classification, and; (5) They require the identification of specific shapes, including letters, numbers, geometric forms, or pathways, which carry greater difficulty in execution compared to the C-gap orientation task used in the present study. Many investigations have reported the misreading of the Ishihara plates even in normal trichromats8 proposed a computerized approach to evaluate color vision using a test that quantified number of errors to identify targets of pseudoisochromatic stimulus in the screen. In their test, there were 80 pseudoisochromatic stimuli that combined different lightness, saturation and hues values. We understood that the use of the number of errors during the test could shorten the test duration, but variability across the stimulus conditions difficulted the characterization of the results or the association of the result with the characteristic of the stimulus. Then in the present study, we proposed to investigate the performance of fixed saturation level to evaluate congenital CVD.Shin et al.We identified the phenotype of the participant’s color vision using a well-known commercial color vision test, CCT. In the participants who had suggestive color vision loss in the CCT, we evaluated the genetics of the L-cone and M-cone genes to confirm the hereditary background of the color vision loss. We understand that anomaloscopy is the gold-standard to classify CVD. As our sample size of participants with CVD is small and our purpose was not to classify the CVD in deutan or protan defects, but to identify the presence of CVD, this limitation did not interfere in the final results of the present investigation.19 and the different combination of saturation, lightness and hue present in the plates can potentially explain these differences in their efficiency in screening CVD. Eight participants with CVD were evaluated in the present study. We performed a ROC analysis to calculate the accuracy of the fixed saturation levels to identify the presence of red-green CVD. We observed that using saturation conditions equal and above of 0.02 u’v’ units, the accuracies were close 100%. Even using saturation of 0.005 or 0.01 u’v’ units, the accuracies were higher than 80%. Several studies have investigated the performance of other color vision tests to identify or classify the congenital color vision deficiencies. Data from 401 people with red-green CVD showed that the performance to identify the presence of CVD varied depending on the type of plates from Ishihara test or HRR test21. For Ishihara test, the combination of transformation and vanishing plates had 95% of accuracy to screen the CVD, while hidden digits plates had identified 50% of the participants with CVD. The protan/deutan classification plates had fails to classify 18% of the protans and 3% of the deutans. Similarly, HRR plates had screening accuracy of 100% for dichromats and 96.4% for anomalous trichromats. Protan/deutan classification was correct for 95% of the participants. As far as we know, there is no data of accuracy for CVD screening or classification of CCT ellipse and trivector protocols.Although it is tempting to compare the test offered here with the variety of existing tests, the present study did not experimentally assess this issue. Our main contribution and novelty was to estimate the accuracy from different saturation levels to screen CVD. Our results add to the literature suggesting that the use of different stimulus conditions has potential to interfere in the accuracy of the test. For example, there are several reports about the efficiency of different plates in the Ishihara to evoke correct responses from trichromats or people with CVDThere is a trade-off relationship between number of chromatic axes to be evaluated and duration of the color vision test. The duration of the testing using CCT trivector protocol is similar to the duration of our proposal using 20 chromatic vectors. Considering purposes of diagnostic and classification of the CVD, the current proposal could be applied using only three vectors in the color confusion lines and the testing would be faster than the estimation of thresholds during CCT trivector protocols.22.We are aware of some caveats and limitations to be overcome in future studies. The main limitation of the present study was the number of participants with CVD. Still, the results obtained from those eight participants were consistent, and we believe there is no qualitative loss of information or misinterpretation of the results shown here. Methodological studies regarding the computer-generated pseudoisochromatic stimuli reliability showed very consistent individual results, even if the procedure were replicated several timesIn conclusion, we suggest that the accuracy of pseudoisochromatic stimulus in chromatic discrimination test to screen CVD can be influenced by the saturation levels present in the target of the stimulus. The combination of high and low saturation levels in the same color vision test can impair the accuracy of the test. Our results can contribute to the design of new color vision tests.