Histone lysine methyltransferases (HKMTs) are key regulators of many cellular processes. By definition, HKMTs catalyse the methylation of lysine residues in histone proteins. The enzymatic activities of HKMTs are under precise control, with their allosteric regulation emerging as a prevalent paradigm. We review the molecular mechanisms of allosteric regulation of HKMTs using well-studied histone H3  methyltransferases as examples. We discuss the current advances and future potential in targeting allosteric sites of HKMTs for drug development. Lysine  residue ,8. Targe residue ,8  another subunit in the same enzymatic complex can act as an allosteric effector or (2) can host the allosteric regulatory site that binds another effector. These two mechanisms are not mutually exclusive and can appear simultaneously or intermittently in the same complex or in closely related complexes. In this review, we discuss how HKMTs are allosterically regulated. In particular, we discuss four distinct types of SET domain-containing HKMTs with clear evidence to support their allosteric regulation: EZH2, Clr4/SUV39, a few members of H3K36 methyltransferases, and the MLL family.Polycomb repressive complexes (PRCs) are histone modifier complexes formed by members of the polycomb-group (PcG) protein family . The polThe core PRC2 complex comprises four subunits: one of the catalytic subunits EZH2 or EZH1, the WD40-containing regulatory protein EED, one of the two WD40 proteins RBBP4 or RBBP7, and the scaffold protein SUZ12  Figure . In addiThe isolated SET domain of EZH2 is in an autoinhibited state . The SETin vitro and reduce the global H3K27me2/3 levels in vivo [Mechanistically, the regulatory subunit EED is the ‘reader', which includes a binding site for the H3K27me3 effector. Upon binding to EED, the H3K27me3 effector triggers allosteric activation of EZH2, which serves as the ‘writer'. The mechanism for H3K27me3-induced allosteric activation of PRC2 has been identified by multiple independent structural and functional works . These s in vivo .de novo, without a dependency on a pre-existing H3K27me3 mark for allosteric activation [H3K27me2/3 is not the only effector of PRC2: PRC2 can also be subjected to the allosteric stimulation by the di- or tri-methyl lysine 116 of the PRC2.2-specific subunit JARID2 (JARID2-K116me2/3) . JARID2-tivation . By dointivation .The physiological relevance of JARID2-induced allosteric activation of PRC2 in the context of embryonic development is yet to be determined. However, some indication that this molecular mechanism is fundamental can be obtained from evolutionary perspectives. First, JARID2 K116 is conserved from fly to human . Second,RNA also plays a role in the regulation of PRC2. PRC2 binds to RNA promiscuously , with a ‘Crosstalk' between histone modifications is common in the regulation of histone methyltransferases. It occurs when a histone mark is deposited by one histone modifier and then regulates another modifier. PRC2 is regulated by various histone marks. For instance, H3K36me2/3 or H3K4me3 inhibit the HMTase activity of PRC2 . A mechaOverall, the mechanism of allosteric regulation is well established in PRC2: once the subunits EED and SUZ12 associate with EZH2, the autoinhibition state is released. Following that release, PRC2 is in its basal state, with the complete substrate-binding site being formed. From that point, the SET domain of EZH2 can participate in methyl transfer. Yet, in its basal state, EZH2 has a relatively low catalytic efficiency. Once the H3K27me2/3 product is produced it is recognised by the regulatory subunit EED, which then triggers a series of conformational changes in EZH2 to stabilise its stimulated state. PRC2 can be subjected to allosteric activation even in the absence of the H3K27me3 mark: subunit-induced allosteric activation of PRC2 occurs when a di- or tri-methyl-lysine within the accessory subunit PALI1/2 or JARID2 leads to the allosteric activation of PRC2.1 or PRC2.2, respectively. The various effectors and mechanisms for the allosteric regulation of PRC2 enable its regulation in a context-specific manner and open paths for drug development (more below).The di- and tri- methylation of H3K9  are a hallmark of constitutive heterochromatin. H3K9me2/3 serves as a platform to recruit heterochromatin protein 1 (HP1) family proteins to mediate gene silencing ,61. In fN. crassa, the catalytic site is occupied by the K9 substrate lysine  is inserted halfway into the active site and blocking it   and Co-IP revealed similar ASH1L-associating proteins in human cells . The humCrystal structures of truncated ASH1L in complex with the MRG domain of MRG15 revealed how MRG15-induced allosteric stimulation of ASHL takes place ,89: a coSubunit-induced structural rearrangement of ASH1L is the first case of allosteric regulation identified in H3K36 methyltransferases . EvidencCollectively, two mechanisms for the allosteric activation were proposed to relieve the autoinhibitory state of H3K36 methyltransferase: (1) subunit-induced allosteric regulation and (2) nucleosome binding-induced conformational change of the autoinhibitory loop. Both mechanisms might safeguard H3K36 methyltransferases by preventing their activity in the absence of an assembled complex or away from chromatin.The methylation of H3K4 is generally recognised as a mark of gene activation . H3K4 meAll six members of the MLL family in humans are large multi-domain proteins, with the highly conserved catalytic SET domain in their C-terminal region. Contrarily, the highly divergent N-terminal regions of MLLs function in their recruitment and modulating their enzymatic activity ,103 Fig. A commoCrystal structures of the isolated SET domain from MLL1, MLL3 and MLL4, in a complex with S-adenosylhomocysteine (SAH), revealed either an open or a closed conformation ,103,105 AS and MLL1AS, respectively, are loop regions critical for the activation of MLL1 [SET). Consequently, the SET-I region is locked into a closed active conformation with respect to its inactive apo-form  modulates the activity of the SET domain of the yeast Set1 by contacting the SET-I region. This regulatory loop is referred to as SIM (SET-I interacting motif) or SMART (Set1 Methyltransferase Activity Regulator) motif ,108. HowAS . Insteadeen them . The SETthem [AS . It suggtivities .SET is associated with WRAD, it only displays weak mono-methyltransferase activity on nucleosomes [AS are absent from MLL3, leading to a different RBBP5–WDR5–MLL3 interface compared with MLL1  between the catalytic subunit EZH2 and the regulatory subunit EED. This was done by a stabilised α-helix peptide, designed to mimic an EED-interacting helix from EZH2 to disrupt its interactions with EED. Accordingly, the peptide destabilises PRC2 and selectively inhibits PRC2 in cancer cells . FollowiFurther studies to reveal how allosteric regulation of SET-domain histone lysine methyltransferases occurs will expand the panel of druggable epigenetic modifiers in disease.More than 100 lysine methyltransferases have been reported in humans . AllosteAt this point in time, some HKMTs are incompletely characterised, including their substrate specificity and their regulation of catalysis . AdvanceHistone lysine methyltransferases (HKMT) are commonly subjected to allosteric regulation, allowing for their regulation in a context-specific manner.in vitro and in situ will likely reveal the full variety of mechanisms for their allosteric regulation.Advances in structural biology allow revealing how allosteric regulation of some HKMTs takes place at the molecular level. Future structural studies of intact HKMTs complexes together with their chromatin substrates The consequences of the allosteric regulation of HKMTs in development and disease remains a key knowledge gap. Advances in genome modification techniques and structural biology will likely allow this knowledge gap to be closed, and open a path to targeting allosteric sites and effectors of HKMTs in disease.


Active deformation mechanisms have been identified and compared for two different deformation modes—uniaxial drawing and plane-strain compression. The initially amorphous PLLA was capable of significant deformation in both tension and plane-strain compression. In contrast, the samples of crystallized PLLA were found brittle in tensile, whereas they proved to be ductile and capable of high-strain deformation when deformed in plane-strain compression. The main deformation mechanism identified in amorphous PLLA was the orientation of chains due to plastic flow, followed by strain-induced crystallization occurring at the true strain above e = 0.5. The oriented chains in amorphous phase were then transformed into oriented mesophase and/or oriented crystals. An upper temperature limit for mesophase formation was found below Td = 90 °C. The amount of mesophase formed in this process did not exceed 5 wt.%. An additional mesophase fraction was generated at high strains from crystals damaged by severe deformation. After the formation of the crystalline phase, further deformation followed the mechanisms characteristic for the semicrystalline polymer. Interlamellar slip supported by crystallographic chain slip has been identified as the major deformation mechanism in semicrystalline PLLA. It was found that the contribution of crystallographic slip increased notably with the increase in the deformation temperature. The most probable active crystallographic slip systems were (010)[001], (100)[001] or (110)[001] slip systems operating along the chain direction. At high temperatures (Td = 115–140 °C), the α→β crystal transformation was additionally observed, leading to the formation of a small fraction of β crystals.The ability of PLLA, either amorphous or semicrystalline, to plastic deformation to large strain was investigated in a wide temperature range (T This polymer can be synthesized from monomers obtained from annually renewable natural resources, e.g., corn, and is biodegradable under industrial composting conditions , (100)[001], or (110)[001] slip systems, all operating along the chain direction [001].The deformation mechanism of amorphous samples compressed at a low temperature (Td < 90 °C, the crystal transformation α→β was also observed, but at high deformation temperature Td = 115–140 °C and high strains. This transformation produces only a small fraction of β crystals in plane-strain compression.Apart from the mesophase, which is produced at high strains from residues of the damaged or destroyed crystalline phase at T


This study aimed to identify the 15-year trends of handgrip strength-to-weight (relative HS) and assess the association between relative HS and hypertension among Chinese adults aged 20–69.Using a complex, stratified, multistage probability cluster sampling, we analyzed data collected from 712,442 adults aged 20–69 years in four successive national surveys (2000–2014). We used a handheld dynamometer to measure strength and divided by body weight to calculate the relative HS. Blood pressure was recorded with a sphygmomanometer and hypertension was defined as resting systolic blood pressure at least 140 mmHg or diastolic at least 90 mmHg. The Mann-Kendall trend test examined trends in relative HS over time. We also computed odd ratios (OR) and 95% confidence intervals (95% CI) by tertile of relative HS and examined the association between relative HS and hypertension.ptrend <0.001). In each of four surveys, one interquartile decrease in relative HS was associated with an increased risk of hypertension by 44% , 58% , 48% , 43% , respectively.The relative HS level decreased with the increase of age in both male and females (In the Chinese adult population, the relative HS level decreased from 2000 to 2014 across all ages in both males and females. A lower relative HS was associated with a higher risk of hypertension. The findings provided evidence for the association between muscle strength and hypertension in large-scaled population. Hypertension is a common health problem worldwide . The ChiRegular exercise is an essential component of a lifestyle intervention for the primary prevention of hypertension . AbsolutThe evidence regarding the association between absolute HS and hypertension is inconclusive. A meta-analysis summarized that an increase of absolute HS exercise was associated with a lower risk of hypertension in adults, but these findings had limitations given the small number of low-quality randomized controlled trials . Two croThe above studies had one common feature: they used absolute HS, rather than relative HS (handgrip strength to body weight or BMI ratio), which might be more comparable among people with different body mass , 14. WheIn the present study, we analyzed the Chinese National Survey on Adults’ Fitness in 2000, 2005, 2010, and 2014. The study aimed to identify the 15-year trends of relative HS in different age and sex groups from 2000 to 2014 and assess the association between relative HS and hypertension among Chinese adults aged 20–69.We used the data from 2000, 2005, 2010, and 2014 Chinese National Survey on Adults’ Fitness, the largest nationally representative survey of civilians in China, using a complex, stratified, multistage probability cluster sampling design. The detail of the recruitment has been described elsewhere .In brief, the 31 provinces, autonomous regions, and municipalities in mainland China were covered in the first stage. At the second stage, subject to their economic positions weighted by GDP assessment, three sub-provincial or prefectural-level cities  were randomly selected from each province. At the third stage, three urban districts  of each city were picked out. At the fourth stage, three city streets  were chosen. At the fifth stage, two street community societies (or villages) were selected. At the final stage, at each street community society (or village), eligible participants living there for more than three years were selected by systematic sampling method.The study protocol had approval from the ethical review committee of the China Institute of Sport Science. Each enrolled participant signed an informed consent form.Through the questionnaire survey and standard physical and physiological measurement, the trained investigators collected the data. We investigated career, education, urban and rural place, nationality (Han or minority) with the questionnaire. For exercise time, the participants were asked to recall the number of days and the exercise duration in the usual past week during the last year. The questionnaire was validated among Chinese adults  .Participants’ weight and height were measured using a calibrated digital scale, stadiometer , Before the measurement, participants with lightweight clothes took off their shoes. The measured weight was rounded up to 0.1kg and height was rounded up to 0.1cm, respectively. The blood pressure measurement was conducted by the auscultatory method by qualified medical staff. The Riva-Rocci sphygmomanometer, and a medical stethoscope  were used for the measurement. The systolic pressure was defined as the first sound heard after releasing the occluded artery. The diastolic pressure corresponded to the final sound heard . SystoliThe subjects were asked not to engage in vigorous physical activity within 1–2 hours before the test.The subjects were asked to sit on the chair in silence for 10–15 minutes before the test, and relax.The mercury column of the sphygmomanometer was checked before measurement. If the column was not at the “0” position, the position was corrected. Furthermore, the mercury column was inspected for the presence of bubbles, and if there were bubbles, it was not used.When measuring blood pressure, it was ensured that the upper arm was not pressed tightly by the sleeve of the jacket. It was ensured that the lower edge of the cuff was located at least 2.5 cm above the cubital fossa.When re-testing was needed (the first test was failed by some reasons), it was ensured that the mercury column of the sphygmomanometer dropped to zero before re-testing. Blood pressure re-testing was carried out 10–15 minutes rest on chair after the initial test. Hypertension was defined as resting systolic blood pressure at least 140 mmHg or diastolic at least 90 mmHg according to the common standard .The following measures were taken before and during the data collection:We used the handgrip meter  to measure the dominant hand handgrip strength. Before the measurement, the subjects held the grip handles with dominant hand and adjusted the grip width to the appropriate force grip. During the measurement, the participants’ body was upright, feet were naturally separated at shoulder width, while arms were inclined and drooping. After three consecutive tests, with adequate rest interval in between, we recorded each participant’s best score as the final result. The grip strength test value is in kg, rounded up to 0.1kg.We used the Skinfold Thickness Meter  to measure the skinfold thickness from three different body sites . We measured three times for each body position and took the median value or two of the same value as the final result. We added three different skinfold thickness measurements to reflect the body fat (skinfold thickness).We described the median  of relative HS (HS to body weight ratio) in different survey years since relative HS was not normally distributed. In the study, the bootstrap percentile method (the number of bootstrap replicates is 500 times) was used to calculate confidence intervals for medians , 19. We To explore the relationship between relative HS and hypertension, we used the tertiles to separate the subjects into three groups  according to the different sex, survey year, and age group, then calculated the prevalences of hypertension in different relative HS groups. The Mann-Kendall trend test was used to examine the trend of hypertension prevalences in different years among different tertiles of relative HS.We used the general linear mixed model to estimate the effect value and odd ratios. Relative HS was analyzed both as a continuous variable and a categorical variable . We presented the results of two statistical models, in which the province of each participant was used as the random effect. We only put exposure and outcome varibales in the crude model. In model 1, we adjusted with age and sex as covariates; In model 2, we further adjusted for the region , the inner-province socio-economic status , the nationality , the education level , career, and exercise level  as covariates.2 for underweight group, between 18.5 kg/m2 and 23.9 kg/m2 for normal group, above 24.0 kg/m2 for overweight and obesity group) [In the sensitivity analyses , 22, we p-value <0.05 was considered as statistical significance.All the statistical analyses were performed with R software, version 3.6.1. A two-sided We included 712,442 participants , while 8,838 participants were excluded for missing main outcome and exposures. The general characteristics and HS levels of the study population in each of the four national surveys were shown in p trend <0.001, p <0.001). It also decreased in urban and rural areas from 2000 to 2014 . In addition, the prevalence of hypertension increased with increasing age groups (p trend <0.001) and was higher in men than in women in different survey years  and this is a strength of our study. Finally, relative HS significantly decreased during the past 15 years in both men and women in China. A similar trend was seen in different age groups, in both urban and rural areas.The finding of a 43–58% reduction of risk in hypertension with one interquartile decrease in relative HS is of public health significance. High blood pressure is the leading modifiable risk factor for cardiovascular disease and premature death worldwide . In our The decline of HS, which may reflect the change in whole body strength , has beeOur findings suggested that relative HS is inversely associated with the risk of hypertension. Furthermore, in the sensitive analysis, stratified BMI groups’ analysis showed that the body mass index did not affect the negative association between relative HS and hypertension, which was in agreement with a rescent study . One posWe found that relative HS has been decreasing among Chinese male and female adults in both urban and rural areas since 2000. The results extended findings from the previous study, which reported the decreasing trend in absolute HS in these successive national surveys . In receThis is the first national study to display the difference of relative HS between male and female in China, and its result, higher relative HS in male adult was consistent with the studies of relative HS in other populations , 46. MorThis study had notable strengths, identifying epidemiological characteristics of relative HS with a sizeable nationally-representative sample size in China and repeated measures during the past 15 years. However, the study had several limitations that should be noted. First, the cross-sectional nature of the national representative data precluded causal inference. Longitudinal studies should be conducted to demonstrate the relationship between relative HS and hypertension using a stronger experimental design. Second, we did not collect hypertension-related risk factors such as energy intake, sodium and potassium intake. It was infeasible to collect comprehensive diet information in large-scale national surveys. However, the consistent association results was found in different survey years and stratified analyses by gender, urban / rural, normal-weight /overweight or obese, indicating the potential bias due to unadjusted factors might be minor. Third, we used body weight and BMI to calculate relative HS and hence we were unable to distinguish between lean and fat mass. Although the sensitive analysis with adjusting for skinfold thickness has confirmed the main results, future studies should include precise measurements such as dual-energy X-ray absorptiometry to study the association between relative HS  and hypertension.The relative HS level decreased during the last 15 years in urban and rural areas across different sex- and age-groups of the Chinese adult population. Lower relative HS was associated with a higher risk of hypertension. Considering the relative strength reflected the overall strength level, the findings provided evidence for the association between muscle strength and hypertension in large-scaled population.S1 Table(DOCX)Click here for additional data file.S2 Table(DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.S4 Table(DOCX)Click here for additional data file.S5 Table(DOCX)Click here for additional data file.S6 Table(DOCX)Click here for additional data file.S7 Table(DOCX)Click here for additional data file.S8 Table(DOCX)Click here for additional data file.S9 Table(DOCX)Click here for additional data file.S10 Table(DOCX)Click here for additional data file.S11 Table(DOCX)Click here for additional data file.


However, one obstacle to practicality lies in the variability between subjects and sessions. Although several studies have adopted domain adaptation (DA) approaches to tackle this problem, most of them treat multiple EEG data from different subjects and sessions together as a single source domain for transfer, which either fails to satisfy the assumption of domain adaptation that the source has a certain marginal distribution, or increases the difficulty of adaptation. We therefore propose the multi-source marginal distribution adaptation (MS-MDA) for EEG emotion recognition, which takes both domain-invariant and domain-specific features into consideration. First, we assume that different EEG data share the same low-level features, then we construct independent branches for multiple EEG data source domains to adopt one-to-one domain adaptation and extract domain-specific features. Finally, the inference is made by multiple branches. We evaluate our method on SEED and SEED-IV for recognizing three and four emotions, respectively. Experimental results show that the MS-MDA outperforms the comparison methods and state-of-the-art models in cross-session and cross-subject transfer scenarios in our settings. Codes at  Emotion as physiological information, unlike widely studied logical intelligence, is central to the quality and range of daily human communications Dolan, . In the Due to the non-stationary between individual sessions and subjects of EEG signals , and the order of whether first concatenating multiple sources or normalizing each session individually. To our knowledge, we are the first to investigate the normalization methods for EEG data, which we believe can be taken as a guide for other future works, and be applied to all data in EEG-based datasets and EEG-related domains.In the remainder of this paper, we first review related works on domain adaptation in the field of EEG-based emotion recognition in section 2. Section 3 introduces the materials, including the diagram of EEG-based affective BCI with transfer scenarios, datasets and pre-processing methods. The details of MS-MDA are given in section 4, whereas section 5 demonstrates the settings, results, and additional experiments. Section 7 discusses the results of the experiment and our findings, as well as problems and solutions. Finally, section 7 concludes the work and outlines the future extension.In recent years, the research of affective computing has become one of the trends of machine learning, neural systems, and rehabilitation study. Among those works, emotions are usually characterized into two types of emotion model: discrete categories , which transfers multiple source domains to the target domain separately, thus avoiding the destruction of the marginal distribution of the multiple EEG source domains; and also takes the domain-invariant features into consideration. Due to the sensitivity of the EEG data and intuition, we do not adopt complex networks, but just a combination of few multi-layer perceptrons (MLPs) .Feature extraction. In this step, features of the pre-processed signals are extracted in various ways. Most of the current research works are to extract features in the time or frequency domain.Pattern recognition. The use of machine learning techniques to classify or regress data according to specific application scenarios.Cross-subject transfer. In one session, new EEG data from a new subject is taken as the target domain, and the rest of existing EEG data from other subjects are taken as the source domains for DA. (2) Cross-session transfer. For one subject, data collected in the previous sessions can be used as the source domain for DA, and data collected in the new session are taken as the target domain.Considering the sensitivity of the EEG, domain adaptation in emotion recognition can be divided into several cases: (1) In our work, since the datasets we evaluate on contains 3 session and 15 subjects (refer to section 3.3 for details), we take the first 2 session data from one subject as the source domains for cross-session transfer, and take the first 14 subjects data from one session as the source domains for cross-subject transfer. The results of cross-session scenarios are averaged over 15 subjects, and the results of cross-subject are averaged over 3 sessions. Standard deviations are also calculated.SEED  from 15 subjects  with 3 sessions each. The signals are recorded by a 62-channel ESI neuroscan system.The SEED-IV is an evolution of SEED, which contains 3 sessions, each has 15 subjects and 24 film clips. Comparing to the SEED with EEG signals only, this database also includes eye movement features recorded by SMI eye-tracking glasses.After collecting EEG raw data, pre-processing on signals and feature extractions will be adopted. For both SEED and SEED-IV, to increase the SNR, the raw EEG signals are first down-sampled to a 200 Hz sampling rate, then been processed with a band-pass filter between 1 Hz to 75 Hz. After that, features are then being extracted.Recent works extract features from EEG data on the time domain, frequency domain, and time-frequency domain. Among them, Differential Entropy (DE) as in (1), has the ability to distinguish patterns from different bands  × trial  × band (5), we then merge the channel with the band, and the form becomes trial × 310 (62 × 5). For SEED, 15 trials contain 3394 samples in total for each session. For SEED-IV, 24 trials contain 851/832/822 samples for three sessions, respectively. In the end, all data are formed into 3394 × 310 (SEED), or 851/832/822 × 310 (SEED-IV) with corresponding generated label vectors in the form of 3,394 × 1, or 851/832/822 × 1.For simplicity of demonstration, we list the symbols and their definition in Given a set of pre-existing EEG data and a newly collected EEG data, our goal is to learn a model ϕ that is trained on these multiple independent source domain data using DA, and thus has a better prediction on the newly collected data than simply combining the existed data into one source domain. The architecture of the proposed method is illustrated in N independent source domain data XT}, and then these data are fed into a common feature extractor module to get the domain-invariance features QT}. Then for each domain-specific feature extractor, extracted common features QT} and get their domain-specific features: As shown in the Common feature extractor in the MS-MDA is used to map the source and target domain data from the original feature spaces to a common sharing latent space, and then common representations of all domains are extracted. This module can help to extract some low-level domain-invariant features.N single fully connected layers to correspond to N source domains. For each pair of source and target domain, we map the data to a unique latent space via the corresponding Domain-specific Feature Extractor (DSFE), respectively, and then obtain the domain-specific features in each branch. To apply DA and bring the two domains close in the latent space, we choose the MMD to estimate the distance between these two domains. MMD is widely used in the DA and can be formulated in (2). In the process of training, MMD loss is decreased to narrow the source domain and the target domain in the feature space, which helps make better predictions for the target domain. This module aims to learn multiple domain-specific features.Domain-specific feature extractor follows the Common Feature Extractor (CFE). After obtaining the features of all domains, we set up N single softmax classifiers that correspond to each source domain. For each classifier training, we choose cross-entropy to estimate the classification loss, as shown in (3). Besides, since there are N classifiers in this module, and these N classifiers are trained on N source domains, if their predictions are simply averaged as the final result, the variance will be high, especially when the target domain samples are at the decision boundary, which will have a significant negative impact on the results. To reduce this variance, a metric called discrepancy loss is introduced to make the predictions of the N classifiers converge, which is shown in (4). The average of the predictions of the N classifiers is taken as the final result.Domain-specific classifier uses the features extracted from the DSFE to predict the result. In Domain-specific Classifier (DSC), there are N source domain EEG data and one target domain EEG data, and then includes a common feature extractor to get N source domain features and one target domain feature. Next, N domain-specific feature extractors are used to pairwise compute the MMD loss of one individual source with the target domain and extract their domain-specific features. Finally, a domain-specific classifier is used to do the classification task, which also calculates the classification loss of the N classifiers using the features, with the discrepancy loss of the N classifiers for the features of the target domain data after the previous N feature extractors.In summary, MS-MDA accepts The training is based on the (5) and following the algorithm as shown in Algorithm 1. For the three losses, minimizing MMD loss can get domain-invariant features for each pair of the source and target domains; minimizing classification loss will bring more accurate classifiers for predicting the source domain data; minimizing discrepancy loss will get more convergent multiple classifiers. The setting of α is illustrated in section 5.1 and we also investigate different settings of β in section 5.5.1.In this section, we describe experiments settings and results in classifying of emotions on two datasets SEED and SEED-IV, with the normalization study with three types and two orders to the EEG data for domain adaptation. Besides, we also conduct some exploratory experiments in addition to the evaluation of our proposed methods and comparison methods.i. e. EEG signals) but the extracted DE features as vectors, complex deep models such as deep convolutional neural networks are not suitable for this module, thus we choose 3-layer MLP for simplicity which reduces feature dimensions from 310-dimension  to 64-D. In CFE, every linear layer is followed by a LeakyReLU , since we do not take raw data  into 3-D matrices [session × trial × (channel*band)], for each session, there is a 2-D matrix of trial × 310. Following the common machine learning normalization approaches and the prior knowledge and intuition of EEG data , the normalization methods to these 2-D matrices can be categorized into three, as shown in During the experiments, we also find that different normalization to data can significantly impact the outcomes, and also the order of whether first concatenating multiple sources or first normalize each session individually. Thus we design diagrams and conduct extensive experiments to investigate the effects of different normalization strategies on the input data, 1 with DAN2, the results indicate that the first normalization order of normalizing the data first and then concatenating them is better. In the third set of MS-MDA, we find that all the results of four normalization types are better than those in the first and second sets, and the improvement is significant. Row w/o normalization in MS-MDA, for example, has a top of 47% improvement, which also indicates the generalization of our proposed method in different normalization types, and the positive effects of taking multiple sources as individual branches for DA.We evaluate three normalization methods and two normalization orders on SEED and SEED-IV with our proposed method MS-MDA and representative domain adaptation model DAN  in For a better understanding of the effect of our proposed method, we randomly pick 100 EEG samples from each subject (domain) in the scenario of cross-subject to visualize with t-SNE , the Strategic Priority Research Program of CAS (XDB32040000), the Zhejiang Provincial Natural Science Foundation of China (LQ20F030013), and the Ningbo Public Service Technology Foundation, China (202002N3181).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Level of Difficulty: Intermediate.)Persistent left superior vena cava (PLSVC) is a known arrhythmogenesis site in patients with atrial fibrillation. However, the optimal PLSVC isolation approach has remained unclear because of the potential risk of complications. The current study reports 2 cases of successful electrical PLSVC isolation using pulsed field ablation. (  Herein, 2 cases of successful electrical isolation of PLSVC with PFA are reported.A 73-year-old male patient with a history of hypertension presented with worsening dyspnea and lower-extremity edema. The electrocardiogram indicated AF. A transesophageal echocardiogram excluded structural heart disease and showed a dilated ostium of the coronary sinus; a patent foramen ovale of 5 mm; and atresia of the right superior vena cava, which indicated a PLSVC. The patient was admitted to undergo an AF ablation after appropriate clinical management of heart failure. The procedure was planned with 3-dimensional (3D) mapping-system guidance . The strategy was switched to PFA, using a 35-mm FARAWAVE with a single transseptal approach because of the difficult transseptal puncture after PLSVC angiography . The sucA 78-year-old female patient with previously diagnosed sick sinus syndrome type 3 subsequent to paroxysmal AF was admitted for worsening palpitation. Catheter ablation was indicated after excluding other structural heart diseases. A transesophageal echocardiogram showed a dilated ostium of the coronary sinus and a common ostium of lateral PVs. The procedure was started with a selective angiography of PVs and PLSVC  under flLeft atrium (LA)-PLSVC disconnection was confirmed with the entrance block based on real-time isolation of PLSVC potential in both cases. However, neither diaphragmatic dysfunction nor ST-segment change on 12-lead electrogram was observed.,,This study is believed to be the first description of the successful PLSVC isolation with PFA. The major thoracic veins, with their specific electrical properties, have an established role in AF genesis and maintenance.,,The biophysical concept of PFA offers nonthermal lesion formation through the mechanism of irreversible electroporation and has recently been applied to catheter ablation.LA-PLSVC disconnection was observed only at the distal portion of the PLSVC in both cases. This may be explained by the enlarged proximal part of PLSVC. The area of proximal PLSVC adhering to the myocardial sleeve may be too large to be isolated with lateral part of the catheter in the starfish-like pose. Moreover, electrical connection between PLSVC and LA was present only along one-half of its circumference after the middle part.PLSCV ablation succeeded without any procedural relevant complications in the 2 current cases. No routine coronary angiogram was performed in the absence of ST change. However, a correlation between PFA and coronary vasospasm was suggested in a single case report by Gunawardene et al.Two cases of successful PLSVC isolation were reported in patients with AF using the pulsed field ablation device without observing signs of procedure relevant complications, which suggested the possibility of PFA as an alternative approach to PLSVC ablation, reducing the risk of complications.All patients in this report provided written informed consent before undergoing the ablation procedure. This report was approved by the Institutional Review Board. The authors have reported that they have no relationships relevant to the contents of this paper to disclose.


Roasted yellow split pea (YSP) flours were used to substitute wheat flour, at 10–20% (flour basis) in wheat bread formulations. Rheometry showed that roasted YSP flour addition increased elasticity and resistance to deformation and flow of the composite doughs, particularly at 20% substitution; instead, at 10% addition (either raw or roasted YSP flour), there were no effects on dough rheology and bread textural properties. Breads fortified with roasted YSP flour at levels >10% exhibited lower loaf-specific volume and harder crumb compared to control (bread without YSP flour). Moreover, only breads with 20% roasted YSP flour displayed a significantly higher staling extent and rate, compared to control, as assessed by large deformation mechanical testing and calorimetry (starch retrogradation) of crumb preparations. This formulation also showed a large increase in β-sheets and β-turns at the expense of α-helix and random coil conformations in protein secondary structure as assessed by FTIR spectroscopy. Roasting of YSP effectively masked the “beany” and “grass-like” off-flavors of raw YSP flour at 10% substitution. Overall, roasted YSP flour at the 10% level was successfully incorporated into wheat bread formulations without adversely affecting dough rheology, bread texture, and shelf-life, resulting in final products with a pleasant flavor profile. Wheat is one of the most important crops in terms of global production and use in various food items. More specifically, the increased consumption of wheat-based products has been associated with the adoption of a “western lifestyle” . The whoDry pulses mainly consist of carbohydrates (60–70% on dry weight basis), and they are superior sources of both dietary fibers and lysine-rich proteins ; apparenConsidering the potential role of pulses consumption in preventing chronic diseases, and in order to improve the nutritional quality of grain foods, there have been many attempts to incorporate legumes into bread and other widely consumed cereal-based products, such as biscuits, pasta, snacks, etc., with limited success, mostly leading to products of inferior technological and sensory characteristics . IncorpoAs expected, the modifications in dough rheological behavior greatly influence bread volume and crumb textural characteristics, which are important indicators of bread quality attributes, from the consumers’ point of view. Legume flour addition at levels higher than 10%, in most cases, adversely affects specific volume and crumb hardness, as it has been observed with the use of various legume flours in breadmaking, such as chickpea ,24,25, lPretreatment of legumes prior to milling might have an important impact on flavor improvement of the end products; e.g., bread made with roasted peas was found to have a less intense aroma and pulse flavor compared to that made from untreated peas . VolatilFinally, bread staling is an important determinant of bakery products shelf-life, since unfavorable changes in texture during storage often precede microbial spoilage. The increase in protein and fiber content in the fortified products might reduce water availability to gluten, but it might also limit the water migration from the crumb to the crust and/or decrease amylopectin retrogradation. For example, an anti-staling effect of lupin protein isolates as evidenced by delaying bread firming during storage for up to 48 h has been reported ; this ouAdditional research is needed to understand the physicochemical properties of legume flours, in order to promote their incorporation into different types of baked items, especially in bread with improved nutritional attributes, but also with acceptable sensorial characteristics and adequate shelf-life. In the present study, roasting of yellow split pea seed before milling was explored as a quick and cost-effective process for eliminating the undesirable legume off-flavors in the final product and thereby creating products with acceptable sensorial attributes to the consumers. Roasted yellow split pea, usually containing 25–30% of protein, was chosen for partial substitution of wheat flour in bread formulation due to its high nutritional value, aiming at providing a comprehensive investigation of its effect on dough thermal and rheological properties as well as physicochemical, sensory, and nutritional characteristics of the end-product, including the staling events. To our knowledge, for the first time, a multi-instrumental analytical approach including calorimetry, FTIR spectroscopy, and texture analysis was employed for the investigation of quality parameters of both fresh and staled products in an attempt to shed some light on the mechanisms of staling kinetics of pulse flour-supplemented wheat doughs and bread; this approach was also combined with an assessment of the potential nutritional benefits and sensorial attributes, which are important complementary aspects of product acceptance by consumers, besides the shelf-life.Wheat flour (WF) with extraction rate of 55% (Type 55), and 11 and 0.70% protein and ash content, respectively, was gifted by Flourmills Thrakis I. Ouzounopoulos, SA , whereas yellow split pea  and dry instant yeast  were purchased from the local market. 50 (median diameter) of 220, 230, and 95 μm, respectively (Yellow split peas (YSP) were roasted in a dry nut roaster  at 150 °C for 25 min in 3 batches of 5 kg each, under continuous mixing to ensure uniform thermal treatment, and then, the roasted YPS seeds from all batches were combined; the color and moisture content of the roasted legumes were determined as described below, and the obtained values from each batch were used as indicators to ensure the repeatability of the roasting treatment. Raw and roasted yellow split pea were milled into flour using a household stone mill , stored at −18 °C until used and designated as yellow split pea flour (YSPF) and roasted yellow split pea flour (RYSPF), respectively. Particle size distribution of the obtained flours and of WF was determined by sieve analysis, according to Sereti et al. , using 2ectively .The RYSPF preparation was incorporated into breads at 10, 15, and 20% substitution levels of the wheat flour. The following flour samples were used for breadmaking : CON  were used to determine the optimum water level of the different dough formulations following the ICC method ; other tDough rheological properties  were also determined with a rotational Physica MCR 300 rheometer , using a Paar Physica circulating bath  and a controlled peltier system  for temperature control. The doughs were prepared as described above, using the same formulations and procedure with breadmaking, but omitting the yeast addition. After mixing, the doughs were wrapped with a plastic membrane to avoid moisture loss and left to rest for 20 min at room temperature, before any rheological measurement. The oscillatory measurements and creep-recovery tests were performed using a parallel plate geometry , with a solvent trap to avoid moisture loss during measurements. After dough loading on the rheometer, the specimen was left to rest for 15 min prior to any measurement. The temperature was regulated at 25 (±0.1) °C. The doughs for these measurements were tested in triplicate.Frequency sweep (oscillatory) measurements  were performed from 0.1 to 20 Hz under a constant strain (0.1%), which fulfilled the linear viscoelasticity requirements as shown by preliminary strain sweep tests. Storage modulus, G′, loss modulus, G″, complex viscosity, η*, and damping factor, tan δ, (G″/ G′) were recorded. Creep-recovery tests were carried out by applying a constant stress (50 Pa) for 60 s on the dough and allowing strain recovery for 180 s after removal of load. The compliance curve data from the creep-recovery tests were analyzed using the supporting software  of the rheometer and fitted to the Burgers model as described elsewhere .The moisture of raw materials and final products (bread crumb and crust) and protein content were determined according to the American Association of Cereal Chemists International official methods 44-15.02  and 46-3Total dietary fiber assays were carried out based on the AACC method 32-05  and the Digestible and resistant starch of the breads was determined using the respective assay procedure of Megazyme (K-DSTRS), which is based on a modified method of Englyst et al. . Accordiab) and Chroma (C*ab) parameters were calculated as described elsewhere [Color parameters  of both bread crumb and crust were measured using a Chromameter  calibrated with a white tile . Additionally, the hue angle . Three bread samples were measured from each breadmaking batch.A texture Analyzer TA.XT plus from Stable Micro Systems  was used for crumb and crust texture analysis. For crumb analysis, a circular shape cutter was employed to obtain cylindrical bread crumb specimens (40 mm diameter × 30 mm height). The samples were submitted to texture profile analysis (TPA) using a 75 mm diameter plate probe. The test was performed at 60% applied deformation, 0.8 mm/s test speed, and 5 s delay time between first and second compression cycles. The TPA parameters, namely, hardness, cohesiveness, springiness, resilience, and chewiness, were calculated according to Armero and Collar . A puncto), the peak temperature (Tp), and the melting enthalpy (ΔHgel) of starch gelatinization (endothermic transition) were determined from the respective thermograms. For estimation of starch retrogradation, samples of about 12 mg of lyophilized crumb from breads stored for 0, 1, 2, and 4 days at 25 °C were mixed with distilled water (flour: water 30:70) and conditioned as described above. Then, the crucibles were heated from 5 to 120 °C (heating rate 5 °C/min). The onset temperature (Toret), the peak temperature (Tpret), and the melting enthalpy (ΔHret) of the retrograded amylopectin were determined from the respective thermograms (endothermic transition). Moreover, the retrogradation index (RI), which represents the percentage of retrograded amylopectin in relation to that of granular starch undergoing gelatinization, was calculated according to Correa and Ferrero [Differential scanning calorimetry (DSC) was performed using the DSC 3 calorimeter . For starch gelatinization, about 12 mg of flour samples were mixed with distilled water (flour: water 30:70), and the slurries were hermetically sealed into DSC stainless steel crucibles and conditioned (proper hydration of flour particles) for 2 h at room temperature; then, the crucibles were heated from 25 to 120 °C (heating rate 5 °C/min). The onset temperature  with a 3-Reflection Diamond/ZnSe Performance Crystal Plate, and to attain good contact between flour particles and cell surface, a constant pressure was applied by the pressure tool. Spectra were obtained in triplicate. To avoid significant interference to the spectra signal due to the water high FTIR signal, freeze-dried bread samples were instead used for the FTIR analyses [2, H2O, and ATR corrections were performed in this order with the aid of Spectramanager v.2.15.15, JASCO. The corrected spectra were then subjected to Savitsky–Golay smoothing  and baseline correction  by the software SpectraGryph v.1.2.13 . The Amide I region between 1580 and 1700 cm−1 [−1 have been assigned to the β-sheet structure, peaks centered in the region at 1660–1685 to β-turn structure, peaks centered in the region at 1652–1660 cm−1 to α-helix conformation, and peaks centered in the region of 1644–1652 cm−1 to random coil structures [Fresh bread samples (2 h after baking) and those obtained from breads following storage for 1, 2, and 4 days were freeze-dried and used for FTIR analysis to evaluate the effect of storage on protein secondary structure and starch chain reordering. Wheat and raw and roasted YSP flours, as well as freeze-dried crumb of all bread samples, were used to obtain their absorption spectra (32 scans) by a FTIR spectrometer  at a resolution of 4.0 cmanalyses . The CO2700 cm−1  was furtructures ,46. The In vitro starch enzymatic digestibility of the bread crumb was assessed according to Lazaridou et al. [3 and placed in 35 mL of sodium phosphate buffer . The pH of the suspension was adjusted to 1.5 (with HCl) and first digested with pepsin  (575 units/g starch) for 30 min at 37 °C. The pH of the mixture was subsequently re-adjusted to pH 6.9 (with NaOH), the volume of the liquid was made up to 50 mL with a solution of sodium phosphate buffer , and porcine pancreatic α-amylase (Megazyme) (110 units/g starch) was added. The suspension was transferred to a dialysis tube , which was placed in a screw cap glass bottle containing 450 mL of sodium phosphate buffer  for 5 h at 37 °C under mild stirring in a water bath shaker . Aliquots (1 mL) of dialysates were taken at 20, 60, 120, 180, 240, and 300 min in duplicate. The aliquots were incubated with amyloglucosidase , and the released glucose was measured using the GOPOD reagent. For evaluation of in vitro starch digestibility, the area under the curve (AUC) of released glucose over 300 min of digestion was calculated. The analysis was performed in triplicate.u et al. . BrieflyA preliminary sensory analysis was carried out at the beginning of the present study to figure out whether the sensory characteristics of the fortified breads with flour from raw or roasted YSP were acceptable by the panel. Thus, 20 individuals were selected to perform a paired preference test in order to choose the most preferable sample between the fortified with 10% raw YSP flour and that with 10% roasted YSP flour. The assessors were selected as potential consumers of the product, based on their responses in a previously distributed questionnaire, indicating that they daily consume wheat bread. Approximately 75% (data not given) of the assessors preferred the formulation with the addition of 10% roasted YSP flour, and therefore, this type of flour was chosen for studying its impact on sensory characteristics of the fortified products at higher levels of supplementation (up to 20%). Quantitative descriptive analysis of the final products was carried out by a trained panel of 18 assessors, focusing on flavor characteristics, such as “wheat bread”, beany, green (grass-like), earthy, roasted, and over-roasted (burnt). The process included several meetings in order first to define the perceived flavors and followed by training to familiarize the panelists with the sought flavor notes. The mean values of all analyzed parameters were compared using Tukey’s test at α = 0.05 significance level.p) was at 63.76 °C (p < 0.05) onset (To) and peak (Tp) gelatinization temperatures, compared to wheat flour alone. Higher gelatinization temperatures have been previously found for starch of different legume species than that of cereal species, attributed to the higher amylose content of starch, i.e., ~30–40% for the former than ~20–25% for the latter [gel) of YSPF or RYSPF was not significantly different compared to that of WF  was higher than the loss modulus (G″), while both moduli slightly increased and complex viscosity (η*) sharply decreased with frequency a. As shop < 0.05) lower maximum creep compliance (Jmax) and viscoelastic compliance (Jm) in the creep phase. The YSP10 and RYSP20 were also characterized by significantly lower maximum creep strain compared to control. The RYSP20 dough showed the highest zero shear viscosity (ηo), which along with the lowest Jmax, describes a dough with a greater resistance to flow and deformation, in agreement with the highest η* and G’ values of this sample, respectively. In previous studies, shifting of the maximum creep strain to lower values and ηo to higher values have been related to stronger doughs [m) compared to control. The rheological properties of dough inevitably affect the textural characteristics of the final product, influencing bread quality, since the dough must be sufficiently elastic to allow the formation of the three-dimensional deformable network and strong enough to maintain the bubble structures formed during proofing and baking. Lupin fiber inclusion in a wheat dough has been found to exhibit a similar pattern, since at increased concentration (15% on flour basis), there was a significant decrease in maximum deformation, compared to lower levels of substitution [Regarding the creep-recovery test, the obtained rheological responses were typical of viscoelastic behavior, as expected for wheat-based doughs, showing the creep compliance data to slightly decrease with increasing roasted legume flour level b. All rhr doughs . It is wtitution . Overalltitution ,17, whilThe appearance of the fortified wheat breads with yellow split pea flours was evaluated by loaf-specific volume measurements and crust and crumb color parameters. Loaf-specific volume was negatively affected by the higher level of legume flour addition, as the RYSP15 and RYSP20 exhibited significantly lower specific volume values than the CON sample . Additiow values in crumb, ~0.97 [The crust and crumb color parameters of the examined breads are summarized in b, ~0.97 . Similarb, ~0.97 . Consumeb, ~0.97 . p < 0.05) hardening rate compared to YSP10 and RYSP10, indicating a quicker staling process. The increased crumb firmness of breads with the highest yellow split pea level is consistent with its reduced loaf volumes and rather compact crumb structure  and peak (Tpret) retrogradation temperatures did not show any differences among the tested samples, probably because the substitution levels were relatively low to have an impact. The ΔHret, corresponding to the melting enthalpy of retrograded amylopectin, reflects the extent of retrogradation upon storage  [−1 is characteristic of the presence of protein molecules and is attributed to the Amide I  and Amide II  bond vibrations [g modes) . As was brations ,62. The −1, assigned to the β-sheet structure, peaks centered at 1668, 1677, and 1683 cm−1, assigned to the β-turn structure, peaks centered at 1653 and 1660 cm−1, assigned to α-helix structure, and a peak centered at 1646 cm−1, assigned to random coil structures. The RYSP20 sample revealed peaks centered at 1622, 1629, and 1693 cm−1 for β-sheet structure, peaks centered at 1671, 1678, and 1686 cm−1 for β-turn structure, peaks centered at 1656 and 1664 cm−1 for α-helix structure, and a peak centered at 1647 cm−1 for random coil structure [Deconvolution of the Amide I peak of the flour spectra yielded several peaks attributed to different secondary structures of the protein components b–e. CON,tructure ,45,46. Ttructure ,64. Raw tructure ,65. Roasp < 0.05) increased estimates of β-turns. As reported by other investigators, for protein structures in a complex system, such as bread, conformational stabilities are very sensitive to interactions with the water molecules and presence of polysaccharides, mainly due to gluten dehydration upon kneading and baking [The evolution of the CON, YSP10, RYSP10, and RYSP20 protein secondary structures was also monitored with FTIR spectra through storage time . The prod baking ,45,72. Sd baking , suggest−1 are attributed to the carbohydrate region and are associated with starch. The peaks around 1010–1020, 1080, and 1150 cm−1 have been attributed to the coupled C-O and C-C stretching vibrations of the polysaccharide molecules [−1 have been previously assigned to short-range molecularly ordered or crystalline structures and amorphous forms of starch, respectively [1047/1022) of the intensities at 1047 and 1022 cm−1 has been adopted as an indicator of the relative level of starch chain ordering [1047/1022 values for the flours used in the present study. Crystallinity calculated by this index was not significantly (p > 0.05) different between wheat and raw YSP flours being 0.455 for both samples; this value is lower than those reported for wheat and waxy maize starches  [p < 0.05) lower R1047/1022 ratio (0.432) compared to YSPF probably due to the thermal processing, which may have caused some disordering of organized chains in starch granules. The spectral features of flour and bread samples a around olecules . The higectively ,74, and ordering . Table 7ctively)  due to t1047/1022 has been proposed as a relative index for monitoring starch retrogradation during bread storage and, thus, product staling [−1 band) and the loss of amorphous chain domains (corresponding to 1022 cm−1). As expected, the increase in starch molecular ordering and the decrease in the amorphous starch structure, due to retrogradation events, were evidenced by the increased R1047/1022 values calculated for the crumb during storage, although such an increase was pronounced only for the RYSP20 bread  reduced “wheat bread” flavor, as that typically recorded for the CON bread preparation. Sample YSP10, which contained the untreated (raw) YSP flour, had higher scores for “green, grass-like” and “beany” flavors, while flour from roasted YSP, when added at 10%, seemed to effectively reduce the aforementioned attributes, and at the same time, a characteristic, pleasant, “roasted” flavor was developed. Inclusion of 20% of flour from YSP increased the “roasted” perception and the “beany flavor” as well, suggesting that the roasting pretreatment of YSP seeds could mask completely, only the “green, grass-like” in the final product when this legume was added to the bread formulations at relatively high levels. Low volume, compact crumb structure, and “beany” and “grass-like” off-flavors are the major defects of legume flour inclusion into bread formulations. The flavor profile analysis, as presented in It has been suggested that the use of gluten or carboxymethylcellulose can significantly improve the volume and crumb characteristics of breads fortified with high levels of legumes, providing breads with accepted textural characteristics . On the Addition of YSP flours improved the nutritional quality of breads, to a certain degree, depending on the fortification level. Protein content adequately increased, only in the case of RYSP20, while all legume-fortified breads had higher total dietary fiber (TDF) content compared to control ; all thein vitro enzymatic starch digestion of breads. Inclusion of roasted YSP at 20% level into the bread formulation significantly decreased the areas under the curves (AUC), calculated over 300 min of digestion, compared to control bread  flour into bread formulations at 10, 15, and 20% substitution level of wheat flour has been investigated in this study. Multi-instrumental analysis of doughs and breads as well as sensory evaluation of the final products were employed for the assessment of dough rheological properties and textural, nutritional, and sensorial attributes of breads as well as for evaluation of bread staling events. Rheometry showed that the addition of flour from roasted YSP at a 20% level exhibited a significant effect on dough rheological parameters by increasing storage modulus and zero shear viscosity as well as decreasing maximum creep compliance, indicating a more elastic dough with a greater resistance to flow and deformation. Parameters derived from farinographs showed that both higher levels (15 and 20%) of legume flour inclusion significantly increased development time and decreased stability of the dough. Moreover, extensographs showed that stretch energy and ratio of resistance to extension/extensibility of dough fortified with 20% roasted legume flour was reduced compared to control formulation made solely by wheat flour, implying a poor baking performance for the composite flours. Inclusion of 10% raw or roasted YSP flour into bread formulations did not have any significant effect on loaf specific volume or on crumb textural properties as evaluated by texture profile analysis. Instead, breads with 15 and 20% roasted YSP exhibited significantly lower specific volume and higher crumb hardness than control, while a product with 20% YSP addition level also had higher chewiness and lower resilience of bread crumb, higher crust moisture content, and lower moisture loss rate of crumb upon bread storage, presumably due to presence of higher dietary fiber and protein concentrations. Therefore, it appeared that the high resistance to deformation and flow of dough fortified with 20% YSP flour resulted in low loaf-specific volume (18% lower than control), which subsequently led to a harder bread crumb and a more compact macrostructure. Moreover, for the latter bread sample, a disruption of the gluten network was revealed by the FTIR analysis of the protein secondary structure, as shown by the large increase in β-sheets in the crumb of this formulation, associated with dehydration of the gluten network, probably due to the presence of high amounts of dietary fiber and legume globulins into the composite dough system; migration of water from gluten to other bread components including starch could also contribute to observed increases in crumb hardening and starch retrogradation rates upon bread storage. Breads formulated with 20% roasted YSP flour exhibited the highest changes in these properties. in vitro testing. Overall, substitution of wheat flour with roasted YSP flour at 10% level was the most successful treatment, since it did not adversely affect dough rheology, bread texture, and staling kinetics, as well as the product sensory attributes. It is obvious that flours from roasted YSP can be effectively incorporated into wheat bread formulations in order to improve quality and nutritional characteristics of the final product when introduced at relatively low levels of fortification. Sensory evaluation using a trained panel revealed that roasting of YSP seeds before milling can effectively reduce the “beany” and “grass-like” off-flavors often detected for the fortified breads with raw YSP flour; however, both attributes were eliminated only in the case of 10% roasted YSP addition. Therefore, breads with 10% roasted YSP could be appealing to a wider diet-conscious consumer group. Finally, inclusion of YSP flours into wheat bread formulation enhances the nutritional quality of the product by increasing the protein and the total dietary fiber content, as well as reducing starch digestibility rates as monitored by


The Domain Name Service (DNS) is a central point in the functioning of the internet. Just as organizations use domain names to enable the access to their computational services, malicious actors make use of domain names to point to the services under their control. Distinguishing between non-malicious and malicious domain names is extremely important, as it allows to grant or block the access to external services, maximizing the security of the organization and users. Nowadays there are many DNS firewall solutions. Most of these are based on known malicious domain lists that are being constantly updated. However, in this way, it is only possible to block known malicious communications, leaving out many others that can be malicious but are not known. Adopting machine learning to classify domains contributes to the detection of domains that are not yet on the block list. The dataset described in this manuscript is meant for supervised machine learning-based analysis of malicious and non-malicious domain names. The dataset was created from scratch, using publicly DNS logs of both malicious and non-malicious domain names. Using the domain name as input, 34 features were obtained. Features like the domain name entropy, number of strange characters and domain name length were obtained directly from the domain name. Other features like, domain name creation date, Internet Protocol (IP), open ports, geolocation were obtained from data enrichment processes (e.g. Open Source Intelligence (OSINT)). The class was determined considering the data source . The dataset consists of data from approximately 90000 domain names and it is balanced between 50% non-malicious and 50% of malicious domain names. Develop and apply Machine Learning (ML) algorithms for this purpose it is very promising;•The data could be used and applied by the scientific community to study and improve the accuracy of detection of malicious domains names in a timely manner towards the development of a real-time DNS firewall;•The data provides the identification and valuation of two classes of domains, malicious and non-malicious, a topic valuable to computer and data science investigations;•The data shows patterns to each class giving the possibility to compare and analyze it in a multi-feature study;•Contribution to the lack of datasets on malicious and non-malicious domains based on DNS logs which are especially important to the field.1The data consists of a Comma-separated Values (CSV) file providing thirty-four features for each DNS domain. The features are presented in The enumerations data type identified in The dataset is targeted for Supervised ML Classification. It is a binary classification predictive modeling, since according the features presented in The total number of complete entries without “null” values in the dataset is 11547 . There exist 78453  rows where at least one of the features it is “null”. The “null” values should be considered in the data preparation phase allowing the researcher to choose the best approach to handle the “null” values.The features related to the DNS response are illustrated in the As expected the feature IP has a large number of different values. In A simple plot between the DomainInAlexaDB and CommonPorts, illustrated in The CountryCode and RegisteredCountry features reveal geographic information about the domain or the IP associated with the domain. These features are illustrated in The domain registration/creation date and last update date uses an enumeration to compress the range of dates. In The Autonomous System Number (ASN) distribution is illustrated in The Hypertext Transfer Protocol (HTTP) / Hypertext Transfer Protocol Secure (HTTPS) response code enumeration is described in The registered organization feature results from querying the WHOIS service. In The subdomain feature  allows cAnother features present in the dataset are the domain entropy and the mean entropy value obtained from the entropy of each sub domain. The result was rounded to integer values and the distribution by the class label is illustrated in the The Top Level Domain (TLD) feature per class is illustrated in The IP and domain reputation feature per class is illustrated in the The distribution of the ratio of vowels, consonants, numeric and special characters in the domain are illustrated in The domain length per class is illustrated in 2To create the dataset we started from lists of already classified malicious and non-malicious domains (data sources). All started with a simple domain name and for each domain a DNS query was performed . The resIn a real scenario the data is obtained from the DNS server and not from structured public available files. To mimics it we replicated the DNS queries. To do so we configured a Local DNS Bind server Presenting the details of the Python modules is not the objective of this article, although we leave an example of a Python module that allows to obtain the IP address for a given domain. This •main_create_datasets.py - The main file to create the dataset and order all the steps starting in the data collection to information gathering.•data/ - Inside the folder there are two sub folders (input and output). The logs collected must be inside the input folder categorized by the DNS record type and by the class. The output sub folder will be the path for the final dataset.•lib/ - Contains the functions and modules to support the information gathering process.•utils/ - Contains utility functions in Python Scripts, such as the constants to be used in run-time. It contains a database sub folder with the AlexaDB The full Python code used to create the dataset is available in •AlexaDB •geoip2 •pydnsbl •sublist3r •tld •IPWhois The third-party libraries/data used to collect information from a domain are:The result of the information gathering process is stored in a CSV file. A small excerpt of the result is illustrated in The work did not involve any human subject or animal experiments. The values of the IP column which are considered personal data under the General Data Protection Regulation (GDPR) rules were anonymized.Cláudio Marques: Investigation, Software, Data curation, Writing – original draft, Visualization. Silvestre Malta: Formal analysis, Writing – review & editing. João Paulo Magalhães: Conceptualization, Methodology, Formal analysis, Writing – review & editing.The authors declare that they have no known competing financial interests or personal relationships which have, or could be perceived to have, influenced the work reported in this article.


Introduced in the late 1980s for generalization purposes, pruning has now become a staple for compressing deep neural networks. Despite many innovations in recent decades, pruning approaches still face core issues that hinder their performance or scalability. Drawing inspiration from early work in the field, and especially the use of weight decay to achieve sparsity, we introduce Selective Weight Decay (SWD), which carries out efficient, continuous pruning throughout training. Our approach, theoretically grounded on Lagrangian smoothing, is versatile and can be applied to multiple tasks, networks, and pruning structures. We show that SWD compares favorably to state-of-the-art approaches, in terms of performance-to-parameters ratio, on the CIFAR-10, Cora, and ImageNet ILSVRC2012 datasets. In recent decades, deep neural networks have become the reference for many machine learning tasks, especially computer vision. Their popularity quickly grew once deep convolutional networks managed to outclass classical methods on benchmark tasks, such as image classification on the ImageNet dataset . Since tThis is a crucial issue for multiple reasons. Indeed, many applications, such as autonomous vehicles, require networks that can provide adequate, real-time responses on energy-efficient hardware: for such tasks, one cannot afford to have either an accurate network that is too slow to run or one that performs quickly but crudely. Additionally, research on deep learning relies heavily on iterative experiments that require a lot of computation time and power: lightening the networks would help to speed up the whole process.Many approaches have been proposed to tackle this issue. These include techniques such as distillation ,11, quanOur work focuses on pruning. The basis of most pruning methods is to train a network and, according to a certain criterion, to identify which parts of it contribute the least to its performance. These parts are then removed and the network is fine-tuned to recover the incurred loss in performance ,17.Multiple decades of innovation in the field have uncovered many issues at stake when pruning networks, such as structure , scalabiOur contribution aims to solve these key problems in a more straightforward and efficient way that avoids human intervention in the training process as much as possible. Our method, Selective Weight Decay (SWD), is a pruning method for deep neural networks that is based on Lagrangian smoothing. It consists in a regularization which, at each step during the training process, penalizes the weights that would be pruned according to a given criterion. The penalization grows in the course of training until the magnitude of the targeted parameters is so close to zero that pruning them induces no drop in performance. This method has many desired properties, including avoiding any discontinuity, since pruned weights are progressively nullified. The weight removal is, itself, learned, which reduces the manual aspects of the pruning process. Moreover, since the penalized weights are not completely removed before the very end of training, the subset of the targeted parameters can be adjusted during training, depending on the current distribution of the weight magnitudes. The dependencies between weights can, thus, be better taken into account.Our experiments show that SWD works well for both light-weight and large-scale datasets and networks with various pruning structures. Our method shines especially for aggressive pruning rates (few remaining parameter targets) and manages to achieve great results with targets for which classical methods experience a large drop in performance.using standardized benchmark datasets, we prove that SWD performs significantly better on aggressive pruning targets than standard methods;we show that SWD needs fewer hyperparameters, introduces no discontinuity, needs no fine-tuning, and can be applied to any pruning structure with any pruning criterion.Therefore, about SWD, which prunes deep neural networks continuously during training, we have the following claims:In the following sections, we will review in detail the field of network pruning, describe our method, present our experiments and their results, and then discuss our observations.We now review the main pruning methods and attempt to organize them into sub-families.N pairs of input/groundtruth pairs We first recall the standard optimization problem with weight decay. Let icient μ ,23. The Although network size correlates with performance, the fundamental observation that motivates pruning is that not all of a trained network’s parts seem to be useful. Unnecessary parts may be removed without penalizing performance.At the end of the 1980s and at the beginning of the 1990s, the field of pruning quickly expanded from a few seminal studies ,25,26. AThis field of research seems to have almost completely vanished during the ensuing decade, only to be resurrected by Han et al. . Since tT. The method of Han et al. [t until T is reached. In particular, pruning is achieved here by reducing to and then maintaining at zero a proportion of the parameters of the whole network whose absolute magnitude is the smallest. Though it is still possible to prune and fine-tune the model only once, doing so can be viewed as a particular case of the method.Assume we want to train and prune a given model with a target pruning rate n et al. , which iThe literature that followed the work of Han et al.  has highOne crucial prerequisite to pruning networks is to have a good criterion to define which parameters to prune. Many pruning criteria have been tested ,31,32,33The early branch of sensitivity calculation methods, birthed by the studies of Le Cun  and MozeThe other main branch identified by Reed  revolvedSince weight decay is required for weight-magnitude pruning, which is the favored criterion among several of the best implementations, it seems that sparsity-inducing regularizations are worth exploring further.One may object by noting that removing weights, even those that seem the least important, may damage the network in such a way that no fine-tuning could ever allow it to recover. Indeed, doing so severely disrupts the training process, for example, by removing parts of the network while it is trying to learn to solve a problem. The work of Le Cun, Bengio, and Hinton  tends toFor example, there is no guarantee that weights which seemed unimportant at first could not become crucial again in the new context of the pruned network. That is the reason why many efforts have focused on allowing weights to regrow in one way or another. Different approaches have been proposed ,45,46 tosparse training. Sparse training was first introduced by Mocanu et al. [The principle of regrowing weights is central to the family of methods that could be called u et al.  and thenu et al. ,49,50. ISuch a family of methods provided a promising way to work around the problem of falsely unimportant weights, while limiting the impact of increasing sparsity all throughout training.lottery ticket hypothesis [learning rate rewinding, which proposes replacing the fine-tuning step with a full retraining stage that uses the weights of the trained and pruned network as a new initialization.Of course, sparse training is not the only family of methods to tackle this issue. Indeed, this technique belongs to a vast trend in the literature: discovering the importance of sparsity during training to achieve better results with shrunken networks. Pruning networks early, so that they start training with their definitive sparsity from the beginning, is the whole point of a whole range of work ,52,53,54pothesis ,58,59,60pothesis , while sAnother distinct branch of methods involves work that aims to induce sparsity in a more continuous way throughout the training process, possibly avoiding any fine-tuning or retraining. One intuition that motivates these methods is that delegating the care of sparsity to the gradient descent is a sensible way to not disrupt training too much.One sub-family of this branch focuses on finding a way to learn a pruning mask during training ,62,63,64However, this whole family of methods tends not to be among the simplest to implement and adapt to various kinds of tasks, datasets, or structures. Adapting variational dropout  to strucUnfortunately, one problem that encompasses the whole field, and about which Blalock et al.  raise thAs pointed out by both Anwar et al.  and Li efilters) [Because of the predominance of convolutional neural networks in the literature, the most widespread type of structure to be considered by the field is convolutional channels (or filters) ,72,73,74unstructured) and convolutional filter-wise (structured).Other types of structures have been experimented on, such as kernels or intra-kernel strided structures ,71 or thWe now present our contribution, illustrated in a being a coefficient which determines the importance of SWD relative to the rest of the optimization problem. Algorithm 1: Summary of SWDSelective Weight Decay (SWD) is a regularization which induces sparsity continuously on any type of structure: at each training step, a certain penalization is applied to the parameters to be pruned at this very step according to a certain criterion and a certain structure. The criterion we chose is weight magnitude, or variants of it according to the chosen structure. The penalized optimization problem can be viewed as: a is designed to be exponential and, according to two bounds s, is defined as such:a can reach large final values without penalizing training too much throughout the training process. In addition, setting The evolution of  as such:a(s)=amina of the SWD term in Equation  is removed.In order to eliminate all unwanted variables, each series of experiments was run under the same conditions, except when explicitly stated, with the same initialization and same seed for the random number generators of the various used libraries. We used no pre-trained networks and we trained all of them in a very standard way.https://github.com/HugoTessier-lab/SWD, accessed on 26 February 2022.The training conditions were as follows: all our networks were trained using the Pytorch framework ; using Unstructured pruning: Han et al. The networks trained with this method were pruned and fine-tuned for five iterations. At each step, the pruning target is a fraction of the final one: for example, when first pruned, only a fifth of the final pruning target is actually removed. The pruned weights are those of least magnitude.Structured pruning: Liu et al. The network is only pruned once and fine-tuned. In accordance with the paper, a smooth-LR-Rewinding: Renda et al When networks are trained following this method, the post-removal fine-tuning is replaced by a retraining which consists in repeating the pre-removal training process exactly, with the same learning rate values and the same number of epochs. This method updates and significantly improves the train, prune, and fine-tune framework that serves as a basis for both previous methods.SWDWhether on unstructured or structured pruning, when trained with SWD, the network is not fine-tuned at all and only pruned once at the end. The values Overall methodologyAll the unstructured pruning methods use weight magnitude as their criterion;All the structured pruning methods are applied to batch normalization layers;Structured LR-Rewinding also applies the smooth-u et al. ;The hyper-parameters specific to the aforementioned methods, namely, the number of iterations and the values of the smooth-In order to isolate the respective gain of each method:SWD does not apply any fine-tuning;Unstructured LR-Rewinding only re-trains the network once ;SWD does not apply a smooth-Here are the only notable differences:Since these results do not come from identical networks on the same datasets, trained in the same conditions, and pruned at the same rate, the comparisons have to be interpreted with caution. However, The results of the experiments on ImageNet ILSVRC2012 are shown in For models trained on ImageNet ILSVRC2012, the standard weight decay , which k being the kernel size, h being the height (in pixels) of the input feature maps, and w being its width:convolution layer: batch normalization layer: dense layer: Exact results are reported in We make no distinction between multiplications and additions in our count.To show the influence of the values of The LeNet-5 models were trained for 200 epochs, with a learning rate of 0.1 and no weight decay , is reported in In order to tease apart the sensitivity of SWD from variations of the model or of the dataset, we provide additional grid searches in Additionally, as highlighted by both In order to verify that SWD can be applied to tasks that are not image classification (such as CIFAR-10/100 or ImageNet ILSVRC2012), we ran experiments on a Graph Convolutional Network (GCN) based on Kipf and Welling  on the CPruning models introduced severe instabilities when training with the original number of epochs per training, set to 200, which is why we trained models for 2000 epochs instead. For SWD, we set We ran this ablation test using ResNet-20, with an initial embedding of 64 feature maps, on CIFAR-10, with unstructured pruning, and under the same conditions as stated in Therefore, we can deduce that (1) SWD is a more efficient removal method than the manual nullification of small weights and (2) the selectivity of SWD is necessary.We measured the additional computing time caused by using SWD. Results are presented in The experiments on CIFAR-10 have shown that SWD performs on par with standard methods for low pruning targets and greatly outperforms them on high ones. Our method allows for much higher targets on the same accuracy. We think that the multiple desirable properties brought by SWD over standard pruning methods are responsible for a much more efficient identification and removal of the unnecessary parts of networks. Indeed, dramatic degradations of performance, which could come from removing a necessary parameter or filter, by error are limited by two things: (1) the continuity of SWD, which lets the other parameters compensate progressively for the loss, and (2) its ability to adapt its targeted parameters, so that the weights that are the most relevant to remove are penalized at a more appropriate time.We compared SWD to multiple methods, described in Because of the low granularity of filter-wise structured pruning, there is always the risk of pruning all filters of a single layer and, then, breaking the network irremediably. This likely explains the sudden drops in performance that can be observed for reference methods in Multiple observations can be drawn from the grid searches displayed in a , shows a better performance/parameters trade-off when compared with reference methods from the literature. We have shown that our method performs better while removing the need for any fine-tuning after the network is pruned. One great advantage of SWD is that it can be combined with virtually any pruning criterion on any pruning structure, which opens up many possibilities. The hyperparameter