CEBPA mutations and differential activity of the S-adenosylmethionine-dependent DNA methylation pathway. In summary, integration across multiple studies can help to identify novel molecular disease subtypes and generate insight into disease biology.Existing large gene expression data repositories hold enormous potential to elucidate disease mechanisms, characterize changes in cellular pathways, and to stratify patients based on molecular profiles. To achieve this goal, integrative resources and tools are needed that allow comparison of results across datasets and data types. We propose an intuitive approach for data-driven stratifications of molecular profiles and benchmark our methodology using the dimensionality reduction algorithm t-distributed stochastic neighbor embedding (t-SNE) with multi-study and multi-platform data on hematological malignancies. Our approach enables assessing the contribution of biological versus technical variation to sample clustering, direct incorporation of additional datasets to the same low dimensional representation, comparison of molecular disease subtypes identified from separate t-SNE representations, and characterization of the obtained clusters based on pathway databases and additional data. In this manner, we performed an integrative analysis across multi-omics acute myeloid leukemia studies. Our approach indicated new molecular subtypes with differential survival and drug responsiveness among samples lacking fusion genes, including a novel myelodysplastic syndrome-like cluster and a cluster characterized with  Gene expression profiling represents the most common genome-wide method for studying cells in healthy and diseased states. As a result, large repositories for data sharing across studies exist . HoweverOne common approach to discover different cellular states and disease types based on gene expression is to use unsupervised methods that require no prior knowledge on sample groups within a dataset . DimensiAcute myeloid leukemia (AML) multi-omics profiling  represenWe propose here a framework aimed to identify and overcome key challenges in analysis of multi-study and multi-platform data and benchmark the methodology with integrative analysis of hematologic malignancies.http://hemap.uta.fi. These data represent microarray data from the commonly used hgu133Plus2 platform that were jointly processed using a typical data analysis pipeline, including the RMA probe summarization algorithm  database , comprislgorithm  to servelgorithm   can distinguish known biological subtypes  based on NMI between cluster assignment and class labels (maximized).eNMI: Similar as above, NMI was calculated between cluster (X) and experiment  identifiers. The data series differences represent mainly technical  differences between samples and therefore this measure was minimized.cNMI: In general, we would like to maximize the separation by phenotype (pNMI) and at the same time minimize the separation by data series (eNMI). We combine these two metrics to one by (pNMI + (1 - eNMI)) / 2. Higher values of cNMI indicate better biological separation compared to technical separation and thus this measure was maximized.To account for both biological and technical differences that are characteristic of sample sets generated by different studies, we propose new metrics based on normalized mutual information  (NMI) toThe utility of the new metrics was demonstrated by comparing the parameters for t-SNE that uses by default Principal Component Analysis (PCA)  for initNew samples were mapped to an existing t-SNE space by using a modified version of the BH-SNE  implemenin silico experiments, a subset of samples assigned to specific clusters were removed and then re-mapped. In this case, the original cluster assignments were used as class labels. The 15% most variable genes, t-SNE map and mean-shift clustering were re-calculated for each sample subset, in order to simulate situations when the original sample set represented the molecular subtype (cluster) well, or poorly.To assign the new samples into existing clusters, distance to the closest centroid was used in cluster assignment. The cluster annotation was determined based on majority voting . Accuracy was calculated based on the proportion of correct phenotype assignments based on clinical annotation data. In the To compare remapping success to a supervised approach, a random forest classifier was trained using 15% most variable genes (randomForest R-package version 4.6–14) by setting the number of trees grown to 20000 while the rest of the parameters were left unchanged. Remapped (removed) samples were used as test data and prediction accuracy was measured. Notice that the labels for training and test data were obtained from clusters found from the full dataset (unlike for re-mapping simulation), and only the 15% most variable genes was re-calculated for each subset to train the classifier.P-value < 0.05) of gene expression level and a given cluster label to assess the robustness of the clustering and correspondence between t-SNE maps.Similarity in sample clustering between t-SNE maps was evaluated in a data-driven manner using gene set variation analysis (GSVA)  enrichmeP-value was computed using 1000 random permutation of genes within the gene set. Estimation of significance was limited to a range of gene set sizes  to adequately account for differences in gene set size distribution. The observed pathway score was compared with the random permutations of a corresponding gene set size and empirical P-value computed as the number of higher/lower scores in the permuted set divided by the total number of permutations. Enrichment of significant scores in a specific cluster was computed using a hypergeometric test. For separate t-SNE maps, consistent enrichment results were obtained by requiring the same directionality in the correlation (to corresponding cluster) with a nominal P-value < 0.05 for the smaller data set and adjusted P-values < 0.001 for the larger dataset.The gene lists for the characterization of sample clusters were obtained from MsigDB v5.0 , WikipatCategorical or binary annotation features , continuous/discrete numeric expression values and continuous/discrete numeric molecular data  were collected for each sample. Missing values were marked as NA. Each t-SNE map cluster and combinations of clusters (binary feature) was tested against all other features and the statistical significance evaluated using Spearman correlation between a binary cluster feature and a numeric feature. With binary-binary pairs Fisher's exact test was used. Multiple hypothesis testing correction was performed using the Benjamini-Hochberg (BH) method.For BeatAML  drug anaP-value > 100) resulting in 151 genes . Both gene sets were evaluated using GSEA in the TCGA cluster 4 matched AML samples versus remaining AML samples in all three datasets .Myelodysplastic syndrome (MDS) signature was generated using Hemap dataset by identifying genes that were most specific to MDS  to reproduce the analysis presented, refer to the User Guide .All custom code for analysis are accessible via GitHub Methods). The dataset was jointly processed using a typical data analysis pipeline that included established normalization , we found that selection of genes based on variable expression results in a much better sample grouping that does not reflect data origin  were all assigned to the same cluster as their replicates, as shown on the Hemap AML t-SNE map in Figure Next, we developed a remapping algorithm that allows additional samples to be included to the existing map. In this manner, the obtained clustering of disease (sub-)types on the t-SNE maps could be used to characterize new samples. Since many new studies currently use RNA-seq, the remapping algorithm was extended to incorporate alternative types of data by revising the similarity metric (see Methods). As a first benchmark, the left-out samples (Methods), even when only 10 samples were included in the initial data (accuracy 0.87), while the accuracy for cluster 4 varied from 0.93 to 0.66. In this case the supervised classifier trained with the original clusters was more robust, however it should be noted that the clusters were re-calculated from each subset only for the re-mapping  and cluster 4 (including CEBPA-mutated samples) that was more challenging to distinguish see . The re--mapping . Finally-mapping .In conclusion, our re-mapping method readily extends t-SNE results beyond the initial sample set when new samples matching the represented biological subtypes are analyzed, and shows robustness for different technologies and matching underrepresented subtypes.Methods, refer to Comparisons between studies would allow to determine whether the same molecular subtypes segregate in a reproducible manner. Towards this end, we developed methodology for examining the correspondence between clusters on separate t-SNE maps. We chose the TCGA AML  RNA-seq, Ross ALNPM1, CEBPA and RUNX1 mutations were characteristic for these non-fusion samples in Hemap . Based on correlation of cluster assignment with multi-omics TCGA data (NPM1 mutations were associated with two clusters (TCGA clusters 3 and 6), whereas CEBPA mutations characterized TCGA cluster 5  confirmed that the TCGA and Hemap patients from the matched clusters share molecular characteristics . Because all DNA methyltransferases use SAM, we quantified TCGA DNA methylation data to test whether global changes in methylation patterns exist between clusters (see Methods). Accordingly, we observed a significant elevation in DNA methylation level in the TCGA cluster 5 compared to other samples .Next, we detected significant associations between the cluster category and pathway activity. The gene set enrichment analysis   Figure  comparin) Figure  and F, w) Figure . In comp) Figure  and F bu) Figure , further) Figure . The Aur) Figure .A large number of biological conditions have been characterized at genome-wide level since the introduction of microarray and deep sequencing technology , HoweverCurrently, it is relatively easy to compile multi-study datasets by prioritizing relevant data sets based on gene-level queries . HoweverMotivated by recent advances in unsupervised applications, we selected the t-SNE algorithm for further development, with the aim to provide solutions suitable for heterogeneous data. Evaluating the dimensionality reduction alone  was fouBased on these data integrative solutions, we performed a comprehensive analysis with integration of additional data types from two multi-omics AML studies to further characterize the identified sample groups. Our approach considers gene expression levels as the main phenotype that gene set and pathway analysis can further characterize. Other data types such as mutations and methylation levels were included by correlating them with the clusters observed in this phenotype (gene expression) space. Alternatively, multi-view methods designed to preserve structures between several layers of data could be utilized ,41. WhilCEBPA, NPM1, RUNX1 and TP53 mutation positive samples. These molecular phenotypes that we found to distinguish clusters of fusion gene-negative samples could be identified from each dataset, demonstrating that the molecular subtypes found are robust, comparable to the fusion gene clusters distinguished in previous classifications , clinical annotations of prior MDS status and the MDS-like gene expression signature indicate that MDS-like AML cases have a unique molecular phenotype associated with poor survival. Furthermore, the analysis of cluster-associated mutations distinguished a patient cluster with CEBPA mutations. Using pathway analysis, we could demonstrate that TCGA and Hemap patients from the matched clusters had elevated expression of genes involved in SAM-dependent methylation activity. Previous studies have found contradicting results, reporting both specific (Our analysis comparing data-driven identification of AML molecular subtypes from TCGA, Hemap and BeatAML datasets revealed a distinct clustering of ications ,30. The utations . The enrspecific  or broadspecific  methylatIn conclusion, we present new data integration approaches for multi-study and multi-omics datasets that allow researching disease subgroups across studies. In future, this analysis framework can be adopted to support the utilization of genome-wide data across different biological systems and disease contexts.gkz281_Supplemental_FilesClick here for additional data file.


Traditional supervised learning classifier needs a lot of labeled samples to achieve good performance, however in many biological datasets there is only a small size of labeled samples and the remaining samples are unlabeled. Labeling these unlabeled samples manually is difficult or expensive. Technologies such as active learning and semi-supervised learning have been proposed to utilize the unlabeled samples for improving the model performance. However in active learning the model suffers from being short-sighted or biased and some manual workload is still needed. The semi-supervised learning methods are easy to be affected by the noisy samples. In this paper we propose a novel logistic regression model based on complementarity of active learning and semi-supervised learning, for utilizing the unlabeled samples with least cost to improve the disease classification accuracy. In addition to that, an update pseudo-labeled samples mechanism is designed to reduce the false pseudo-labeled samples. The experiment results show that this new model can achieve better performances compared the widely used semi-supervised learning and active learning methods in disease classification and gene selection. However these supervised learning methods need a lot of labeled samples to achieve satisfactory results. Nevertheless in many biological datasets there is only a small size labeled data and remaining samples are unlabeled. Labeling these unlabeled samples manually is difficult or expensive; hence many unlabeled samples are left in the dataset. On the other hand, the proportion of small size labeled samples may not represent the real data distribution, which makes the classifier difficult to get the expected accuracy. Trying to improve the classification performance, many incrementally learning technologies such as semi-supervised learning (SSL)3 and active learning (AL)4 have been designed which utilize the unlabeled samples.Identifying disease related genes and classifying the disease type using gene expression data is a very hot topic in machine learning. Many different models such as logistic regression model5 is the most popular AL strategy in practice because it does not require significant overhead to use. However one problem is that using uncertainty sampling may make the model to be short-sighted or biased6. What is more, though AL reduces the manpower work, manually labeling the selected samples by AL in biological experiments still cost much.AL tries to train an accurate prediction model with minimum cost of labeling the unlabeled samples manually. It selects most uncertain or informative unlabeled samples and annotates them by human experts. These labeled samples are included to the training dataset to improve the model performance. Uncertainty sampling7, graph-based methods8, co-training9, self-training10 and so on. However11 pointed out that the pseudo-labeled samples are annotated based on the labeled samples in the dataset, and they are easy to be affected by the high noisy samples. That is why SSL may not achieve satisfactory accuracy in some places.In another way, SSL uses unlabeled data together with labeled data in the training process without any manual labeling. Many different SSL methods have been designed in machine learning including transductive support vector machines12, the most informative samples which were selected by AL-SVM were annotated by experts and then the classifier was retrained using SSL technology by the new dataset13. used a SSL technology to help AL select the query points more efficiently and further reducing the workload of manual classification. In14 a SVM classifier was proposed to manually label the most uncertain samples and at the same time the other unlabeled samples were labeled by SSL, thus a faster convergence result was gained. The recent study15 proposed by Lin designed a new active self-paced learning mechanism which combines the AL and SSL for face recognition.Many researchers found the complementarity between AL and SSL. Song combined the AL and SSL to extract protein interaction sentencesHowever, most attention of the methods combing SSL and AL are paid to the SVM model. The logistic regression model which widely used for disease classification is seldom mentioned. And also in these existing methods, the most informative samples selected by AL are manually annotated, this work maybe very expensive or time consuming in disease classification. Hence we design a new logistic regression model combining AL and SSL which meets the following requirements:The new model should be easily understood and applied. Our method should not require significant engineering overhead to use.15. By the complementarity of uncertainty sampling and self-training, it is easy to build a select-retrain circulation mechanism based on the samples’ probabilities estimated by the logistic classifier.In this new logistic regression model, we use uncertainty sampling to select the most informative samples in AL. Uncertainty sampling is fairly easily generalized to probabilistic structure prediction models. For logistic regression model, the sample probability closed to the decision boundary (probability ≈ 0.5) will suffice. In the new logistic regression model, self-training is used as a complement to AL. Self-training is one of the popular technologies used in SSL because of its fast speed and simplicity, and this method is a good way to solve the short-sighted problem in AL. In self-training the classifier is first trained by using the small size labeled samples, and then the obtained classifier will be used to label the high confidence samples in the unlabeled samples pool. These selected samples will be included into the training set and the classifier will be retrained. The cycle repeats until all the unlabeled samples have been used. In the logistic regression model, the samples which the probability closed to 0 or 1 can be seen as the high confidence samples. In our model, uncertainty sampling is used for avoiding the classifier being misled by high noisy samples, and self-training can avoid the model to be short-sighted or biased because of the high confidence samples’ compactness and consistency in the feature spaceThe new model can achieve a satisfactory accuracy while labeling the samples automatically without manual labeling.Sometimes labeling the disease samples manually is difficult, expensive or time consuming. In our model the uncertain samples selected by AL are labeled by the last classifier automatically, it significantly reduces the burden of manual labeling. However how to ensure the correctness of these uncertain samples? The most uncertain samples mean the false pseudo-labeled samples are easy to be generated. On the other hand the most uncertain samples can be seen as the most informative samples in the logistic model, and the misclassified samples will degenerate the model performance obviously. Considering these samples are not removed or corrected in the standard AL and SSL methods, we design an update mechanism for the pseudo-labeled samples which makes the misclassified samples have chances to be corrected based on the new classifiers which generated in later training interactions.n samples, which includes n1 labeled samples and n2 unlabeled samples, n = n1 + n2. And this dataset contains p genes. Y and gene expression X. yi is disease type, ci represents the sample is labeled or unlabeled. The basic logistic regression model can be expressed as:Supposing the biological dataset has The log-likelihood of the logistic regression method can be expressed as:λ is the tuning parameter.Trying to identify disease related genes in the gene expression data, L1-norm regularization (Lasso) is added in the model:v is the weight parameter of the unlabeled samples, and the v, a is the control parameter. The selected unlabeled samples will be labeled manually and then included into the training dataset. The In the active learning part of our method, we use uncertainty sampling to select samples in the unlabeled dataset. In the logistic regression model the sample which probability close to the decision boundary (probability ≈ 0.5) can be seen as the most uncertain sample in AL. Hence an AL logistic regression model can be expressed as:w is the weight parameter of the unlabeled samples, the γ is the control parameter. The In the logistic regression model the sample probability closest to 0 or 1 can be seen as the high confidence sample. It is easy to find that the difference between the self-training and uncertainty sampling is that the selection criteria of identifying the used unlabeled samples. Hence the self-training logistic regression model is shown as:w is the weight parameter of the unlabeled samples given by SSL, and the v is the weight parameter of the unlabeled samples obtained by AL.In this paper we propose a novel logistic regression model combining SSL and AL with an update mechanism. The high confidence unlabeled samples selected by self-training can avoid the classifier to be short-sighted. The low confidence samples selected by uncertainty sampling prevent the classifier to be misled by high noisy samples which are offered by self-training. The model can be expressed as:If the sample is selected by SSL and the label has been changed by the classifier, this sample will be returned to the unlabeled sample pool and wait to be selected again.If the sample is selected by AL and the label has been changed, we revise the label of this sample and it will be put into the training dataset directly.Different from the ordinary AL methods, the unlabeled samples selected in our model are labeled by the learned classifier automatically. Considering the uncertainty of classified samples, the misclassified samples should have the chances to be revised in latter training iterations. The update mechanism is described below:Step 1: Firstly the labeled data will be used to learn an initial logistic regression model.Step 2: The logistic regression model will be used to label the unlabeled samples and the high value samples which are selected by SSL or AL will be included into the training dataset.Step 3: Update the logistic regression model using the new training dataset.Step 4: Identify the false pseudo-labeled samples. If they are selected by SSL, return them to the unlabeled sample pool. Otherwise, change their labels and put them into the training dataset directly.Step 5: The cycle will continue until all the unlabeled samples have been labeled or the run time exceeds the maximum number of iteration.The work flow of our proposed logistic regression model is show in Fig. The algorithm of our proposed logistic regression model combining SSL and AL is given in below:C is computed based on the step size SZ which is the selection range for identifying high value samples based on the pseudo-labeled samples’ probabilities. The probi is defined as the probability of the ith pseudo-labeled sample which is estimated by the logistic model. Here we give an example to discuss the convergence of this model: if the SZ is set 0.2, the C is 5 (SZ *C = 1). In first iteration only the pseudo-labeled samples meeting the following conditions will be used: 0 < probi < 0.05 and 0.95 < probi < 1 (selected by SSL) & 0.45 < probi < 0.55 (selected by AL), here the initial probability range is 0.2; in the second iteration the range will be increased to 0 < probi < 0.1 and 0.9 < probi < 1 (selected by SSL) & 0.4 < probi < 0.6 (selected by AL), the probability range is increased to 0.4, SZ = 0.2 means in every iteration the range of probability will increase by 0.2. And while C = 5, the probability range is increased to 1, it means all the pseudo-labeled samples will be used. The commonly C is set 10 (SZ = 0.1) or 20 (SZ = 0.05). Sometimes before the iteration reaches the maximum iteration C, all the pseudo-labeled samples have been selected, especially while the SZ is set very small. For saving the computing time and cost, the program will be terminated early.The maximum iteration n samples, and the number of the genes is 4000. In these 4000 genes we set 10 disease related genes, and the coefficients of the remaining 3990 genes are set zero.Step 1: Supposing the dataset has p is set 0.3.  are generated independently from standard normal distributionStep 2: The correlation coefficient β0 is the intercept and ε is the randomly generated Gauss white noise.Step 3: The sample is generated as: n1 labeled samples and n2 unlabeled samples, where n = n1 + n2. In Group A we suppose n1 = 100, n2 = 200; and in Group B n1 = 150, n2=300. We recorded the  is the number of identified disease related genes, false positive (FP) is the number of selected unrelated genes, false negative (FN) is the number of disease related genes which are missed, and true negative (TN) is the number of the unrelated genes that are abandon by different models. The sensitivity and specificity can be expressed as:In this paper we compare six different methods: the single logistic model with Lasso, the AL logistic model with Lasso (AL-lo), the self-training logistic model with Lasso (SSL-lo), the logistic model combining with AL and SSL which needs manual labeling (ASSL-lo), the auto logistic model with Lasso combining with AL and SSL without manual labeling and update mechanism (Auto-ASSL(A)), and the logistic model with Lasso combining with AL and SSL without manual labeling but using update mechanism (Auto-ASSL(B)). In AL-lo and ASSL-lo, about 40% unlabeled samples are labeled manually. The classification accuracy of the unlabeled data is used to evaluate the classification performances of different models. The number of selected correct genes (NC), the number of selected genes (NS), specificity obtained by single logistic regression model is highest than any other methods, it means it doesn’t select too many unrelated genes. However the lowest sensitivity shows single logistic regression model selects the least disease related genes. The AL-lo achieves a closed specificity value compared to single logistic model, but it identifies more disease related genes. The SSL-lo selects more disease related genes than single logistic model, and meanwhile many unrelated genes are also selected. Through combining the AL and SSL, the ASSL-lo identifies most disease related genes, but the problem is that it also selects more disease unrelated genes than SSL-lo. Auto-ASSL(A) selects less correct genes compared to ASSL, and the numbers of selected unrelated genes are closed. Compared to the Auto-ASSL(A), the gene selection performance obtained by Auto-ASSL(B) is obviously improved. The sensitivity obtained by Auto-ASSL(B) is only less than ASSL-lo but higher than any other methods, and the specificity is even more than the ASSL-lo. It shows that the Auto-ASSL(B) can achieve a balance between the sensitivity and specificity, and it has a strong ability to identify the disease related genes meanwhile eliminates the interference of unrelated genes.The gene selection performances of different methods in simulation experiments are shown in Table The values of classification accuracy obtained by different methods in the unlabeled data are shown in Fig. It works without any manual intervention. This saves much cost and the results can be quickly obtained.It can achieve accuracy above 90% in disease classification. The experiments show our method can achieve a better accuracy than the AL and SSL logistic regression models.It can identify more disease related genes and at the same time less unrelated genes will be selected. This further saves the researchers’ time and cost.Hence the new logistic regression model combing SSL and AL can be seen as a very efficient method because it implements the following functions:16, Prostate cancer dataset17, GSE2105018 and GSE3260319. In these four datasets about 2/3 samples are treated as the unlabeled samples for evaluating the classification accuracy of unlabeled samples. The labeled samples and unlabeled samples are randomly selected in every runs of the program. More details of the datasets used in the experiments are shown in Table In real data experiments six methods are applied on four real gene expression datasets: Diffuse large B-cell lymphoma (DLBCL) datasetThe values of classification accuracy obtained by different methods in real datasets are shown in Table The numbers of genes selected by different methods in real dataset are shown in Fig. SELENOP, HPN, MTHFD2 and ROR2 are the ones which are selected by all the methods in the same datasets. The SELENOP in DLBCL can be seen as an extracellular antioxidant, and it may be potential non-invasive diagnostic markers for cancer. Some researches show that selenium could be seen as an anticancer therapy by affecting SELENOP20. The research has proved that expression of the encoded protein of HPN is related to the growth and progression of cancers, particularly prostate cancer. It may be associated with susceptibility to prostate cancer21. The MTHFD2 in GSE21050 is seen as a prognostic factor and a potential therapeutic target for future cancer treatments.22. The ROR2 in GSE32603 is reported that it can significantly reduce cell proliferation and induced apoptosis23.In order to further assess the correctness of the selected genes by different methods, the top-10 ranked genes selected by different methods in real datasets are listed in Tables On the other hand, our method also identified some special genes which other methods did not select.MDM4 in DLBCL plays a very important role in the proliferation of the cancer cells, and it is crucial for the establishment and progression of tumors24. JUNB plays a specific role in cancer cell proliferation, survival and drug resistance25. Single nucleotide polymorphism of TIPARP in Prostate has been proved to be related with cancer26. In27ENO2 is reported to be a risk factor for bone metastases in cancer. The TPD52L2 in GSE32603 encodes a member of the tumor protein D52-like family, and contributes to proliferation of cancer cells28. These genes mentioned in the literatures demonstrate that our new logistic regression model has a strong ability in gene selection.These genes are shown in bold in the Tables In this paper we have designed a novel method which does not require significant engineering overhead to use and meanwhile achieves satisfying results by utilizing the unlabeled gene expression samples in disease classification. The novel logistic regression model is designed based on the complementarity of semi-supervised learning and active learning. In addition to that an update pseudo-labeled samples mechanism is embedded in this method to reduce the false pseudo-labeled samples. In conclusion, our method can achieve more accuracy results compared widely used SSL and AL logistic models, and it also has a good performance in identifying the disease related genes. In addition to that, this model can work without any manual labeling for saving much time and cost. We believe it will be an efficient tool to make contributions for disease classification and gene selection because of its high reliability and stability against noise and outliers.


The thermoanalytical technique differential scanning calorimetry (DSC) has been applied to characterize protein denaturation patterns (thermograms) in blood plasma samples and relate these to a subject’s health status. The analysis and classification of thermograms is challenging because of the high-dimensionality of the dataset. There are various methods for group classification using high-dimensional data sets; however, the impact of using high-dimensional data sets for cancer classification has been poorly understood. In the present article, we proposed a statistical approach for data reduction and a parametric method (PM) for modeling of high-dimensional data sets for two- and three- group classification using DSC and demographic data. We compared the PM to the non-parametric classification method K-nearest neighbors (KNN) and the semi-parametric classification method KNN with dynamic time warping (DTW). We evaluated the performance of these methods for multiple two-group classifications: (i) normal versus cervical cancer, (ii) normal versus lung cancer, (iii) normal versus cancer , (iv) lung cancer versus cervical cancer as well as for three-group classification: normal versus cervical cancer versus lung cancer. In general, performance for two-group classification was high whereas three-group classification was more challenging, with all three methods predicting normal samples more accurately than cancer samples. Moreover, specificity of the PM method was mostly higher or the same as KNN and DTW-KNN with lower sensitivity. The performance of KNN and DTW-KNN decreased with the inclusion of demographic data, whereas similar performance was observed for the PM which could be explained by the fact that the PM uses fewer parameters as compared to KNN and DTW-KNN methods and is thus less susceptible to the risk of overfitting. More importantly the accuracy of the PM can be increased by using a greater number of quantile data points and by the inclusion of additional demographic and clinical data, providing a substantial advantage over KNN and DTW-KNN methods. Differential scanning calorimetry (DSC) analysis of human blood plasma has been used as a method to detect disease related changes in the plasma proteome –6. DSC aWe have previously described statistical approaches for the classification of plasma thermograms according to disease status, an essential step in the development of the clinical utility of DSC thermograms . In thisPlasma samples from 100 healthy individuals with demographic characteristics were purchased from Innovative Research . Cervical cancer specimens were obtained from women attending the clinics of the Division of Gynecologic Oncology with invasive cervical carcinoma. Lung cancer specimens were obtained from patients attending the clinics of the Division of Thoracic Oncology. The study protocol and patient consent procedures were approved by the University of Louisville Institutional Review Board . All patients gave written informed consent for their blood and tissues to be entered into a tissue repository  and utilized for research purposes. The IRB specifically approved the use of plasma specimens from the bio repository for use in this study without the need for further consent . All specimens collected for the study were deidentified. Associated demographic and clinical information was collected by clinical trials office personnel and securely stored on the bio repository computer. The bio repository was approved by the University of Louisville Institutional Review Board  and was fully HIPAA compliant. Specimens provided for DSC studies were coded by bio repository collection number. In this form, specimens were deidentified and blinded for demographic and pathologic disease status for unbiased data collection. Demographic and clinical status was subsequently provided for data analysis. Blood was drawn into 6 mL green top  vacutainers. Tubes were gently mixed by inversion 8–10 times immediately after blood collection to evenly distribute the anticoagulant additive followed by centrifugation at 3200 rpm for 10 min (BD-Clay Adams Compact II centrifuge). Separated plasma was carefully aspirated to avoid hemolysis or contamination of the separated blood phases, aliquoted and immediately stored at −80 °C until analysis. All handling of specimens and specimen waste was in accordance with OSHA bloodborne pathogen procedures.2PO4, 8.3 mM K2HPO4, 150 mM NaCl, 15 mM sodium citrate, pH 7.5) for 24 h at 4 °C to achieve normalization of buffer conditions for all samples obtained in different plasma anticoagulants. Samples were diluted 25-fold with dialysate to obtain a suitable concentration for DSC analysis. DSC data were collected using a VP-Capillary automated DSC instrument . Samples and dialysate were stored in 96-well plates and transferred to the instrument autosampler thermostated at 5 °C until being loaded into the calorimeter by the robotic attachment. DSC scans were recorded from 20 °C to 110 °C at 1 °C/min using the mid feedback mode, a filtering period of 2 s and a prescan thermostat of 15 min. Total protein concentrations of plasma samples for normalization of DSC data were measured using the bicinchoninic acid protein assay kit microplate procedure , with minor modifications to the manufacturer’s protocol. DSC data were processed using Origin version 7 . The process was as follows: 1) correct for the instrument baseline by subtracting an appropriate buffer reference scan, 2) normalize for the total gram concentration of protein in each sample, 3) fit for nonzero sample baselines by applying a linear baseline function. Final thermogram data were the average of duplicate measurements and plotted as excess specific heat capacity  versus temperature (°C). Examination of thermogram data revealed that the temperature range 45–90°C spanned the denaturation profile for all samples and scans were truncated to this range for subsequent analyses. Three control samples were flagged as poor quality data and removed prior to analysis.Plasma samples were prepared for DSC analysis according to our previously published method , 21. BriC0) [97 samples], (ii) cervical cancer (CC) [35 samples], (iii) lung cancer (LC) [54 samples]. There are 451 data points per thermogram with each sample having duplicate measurements. We take the mean of the duplicate measurements to get a single measurement at each of these data points for each sample (The DSC thermogram data set is comprised of heat capacity (HC) values at 0.1°C intervals over the temperature range 45°C-90°C for a total of 186 samples from three clinical groups: (i) normal/ control  for all samples.C0 and CC groups with a higher age range associated with the LC group. The observations for the CC and LC groups are consistent with the demographic characteristics of the patient populations seen at the James Graham Brown Cancer Center clinics. There are two levels in the variables, ethnicity [White (Non-Hispanic or Latino) or others ] and gender [male or female].We have incorporated demographic data  of the individuals for the development of the classification method . DiffereC0 and CC, C0 and LC, CC and LC) . We evaluated classification performance using only DSC data as well as combining DSC data and demographic data and compared the performance of our developed method with the semi-parametric method DTW-KNN and the non-parametric method KNN. We calculated accuracy measures for each classification [The three classification approaches were applied to evaluate the discrimination of various two group combinations: (i) fication . To assefication   and log transformed data (H1) (CC and LC) in case of the log transformed (H1) data. Therefore, we have selected the most common transformation, i.e., the logarithmic transformation (H1) for further analysis.The result given in ata (H1) . We founH1 = log(H), T1 = (T − mean(T))/sd(T), G is the group indicator  for both Eqs ..5).H1=αboth Eqs . We diviCC + LC) samples in the training dataset. The remaining 29 normal, 11 cervical cancer, 16 lung cancer samples and 27 cancer (CC + LC) samples have been used as the testing dataset.We have used 70% of the samples as the training dataset for the two group and three group classification methods. We have 68 normal, 24 cervical cancer, 38 lung cancer samples and 62 cancer  was the optimal transformation based on the normality test at each temperature point  is lower than KNN and DTW-KNN . Inclusion of demographic data decreased the performance of KNN and DTW-KNN but performance of PM was similar thus making all three methods comparable. Overall, the class-wise classification results obtained from the three-group classification method shows that the normal samples are predicted more accurately as compared to cervical cancer and lung cancer samples.One disadvantage of the KNN and DTW-KNN methods are the risk of the overfitting which can be observed from our results when the inclusion of demographic data decreased accuracy of both methods. This shows the KNN and DTW-KNN methods are sensitive to inclusion of additional covariates. The use of fewer parameters with the PM resulted in lower performance for models with no demographic data but higher performance when demographic data was included. This is of great interest in the further development of the PM where classification performance can be increased by the inclusion of additional demographic or clinical parameters as well as by using a greater number of quantile points and provides a substantial advantage over methods such as KNN and DTW-KNN. Alternative approaches such as support vector machine (SVM) methods –34 are oIn the present article, we have compared three methods  using DSC and demographic data for classifying normal and cancer samples. We have developed and implemented the various steps of two-group and three-group classification methods using three different approaches using the statistical analysis program R. Future work will address the development of the PM for the classification of DSC datasets in other disease settings and the development of a classification prediction algorithm using DSC reference datasets.S1 FigH) and (B) log transformed data (H1).Principal component analysis biplot using (A) the original data ((TIF)Click here for additional data file.S1 FileThis file contains the DSC data and demographic / clinical data for all subjects within each clinical group . The file contains one worksheet for the DSC data and one worksheet for the demographic / clinical data for each of the three groups . The first row of the DSC data worksheet contains the column labels. The first column is labeled “Temperature” and contains the temperature values (in degrees Celsius) for each heat capacity measurement . Subsequent columns are labeled with a unique identifier for each subject and contain heat capacity data at each temperature point for a given subject. Subject identifiers starting with the letter L indicate lung cancer patients, the letter C indicates cervical cancer patients and N indicate normal / control subjects. The first row of the demographic / clinical worksheets contains the column labels indicating the sample ID (unique subject identifier) and relevant demographic / clinical variables for each group. The subsequent rows in the worksheet contain demographic / clinical data for a unique subject.(XLSX)Click here for additional data file.S2 File(DOCX)Click here for additional data file.S3 FileFlowchart of the dynamic time warping method .(DOCX)Click here for additional data file.S4 FileResults of the normality test showing p-values at different temperature points using different data transformations for Normal vs. Cervical Cancer (Table A). Results of the normality test showing p-values at different temperature points using different transformations for Normal vs. Lung Cancer (Table B). Results of the normality test showing p-values at different temperature points using different transformations for Cervical Cancer vs. Lung Cancer (Table C).(DOCX)Click here for additional data file.S5 FileSummary of age for the normal group using a median cut-off of 36 years (Table A). The composite line plot and error bar plot . (1) Composite line plot of HC values at each temperature data point for 49 normal samples with age ≤ 36 years (green) and 48 normal samples with age > 36 years (orange). (2) Composite error bar plot of HC values at each temperature data point for the two groups: normal samples with age ≤ 36 years (green) and normal samples with age > 36 years (orange). The circles represent mean values and the error bars represent the 95% confidence interval.(DOCX)Click here for additional data file.S6 FileUnzip/extract the file to get two R script files: “Classification_DSC.R” and “source.DSC.R”. Use “Classification_DSC.R” to obtain various outputs.(ZIP)Click here for additional data file.


Systemic lupus erythematosus (SLE) is a heterogeneous autoimmune disease that is difficult to treat. There is currently no optimal stratification of patients with SLE, and thus, responses to available treatments are unpredictable. Here, we developed a new stratification scheme for patients with SLE, based on the computational analysis of patients’ whole‐blood transcriptomes.We applied machine learning approaches to RNA‐sequencing (RNA‐seq) data sets to stratify patients with SLE into four distinct clusters based on their gene expression profiles. A meta‐analysis on three recently published whole‐blood RNA‐seq data sets was carried out, and an additional similar data set of 30 patients with SLE and 29 healthy donors was incorporated in this study; a total of 161 patients with SLE and 57 healthy donors were analysed.Examination of SLE clusters, as opposed to unstratified SLE patients, revealed underappreciated differences in the pattern of expression of disease‐related genes relative to clinical presentation. Moreover, gene signatures correlated with flare activity were successfully identified.Given that SLE disease heterogeneity is a key challenge hindering the design of optimal clinical trials and the adequate management of patients, our approach opens a new possible avenue addressing this limitation via a greater understanding of SLE heterogeneity in humans. Stratification of patients based on gene expression signatures may be a valuable strategy allowing the identification of separate molecular mechanisms underpinning disease in SLE. Further, this approach may have a use in understanding the variability in responsiveness to therapeutics, thereby improving the design of clinical trials and advancing personalised therapy. We examined whole‐blood RNA‐seq data from a total of 161 patients with systemic lupus erythematosus and 57 healthy donors, uncovering four distinct subtypes of patients with distinct gene expression patterns. Our study provides an innovative patient stratification framework, which is likely to be useful in future studies looking at different responses to treatments, potentially enabling improved therapeutic decisions and more reliable prediction of response to therapy. Systemic lupus erythematosus (SLE) is a debilitating chronic autoimmune condition characterised by the activation of inflammatory immune cells and the production of proinflammatory autoantibodies responsible for pathology in multiple organs.Numerous large‐scale clinical trials for SLE treatments have been carried out, with an improvement over standard of care as the expected outcome of these studies. Disappointingly, the vast majority of tested therapies failed their primary endpoints,Here, we have applied machine learning approaches to stratify patients with SLE based on gene expression patterns derived from whole‐blood transcriptomic data. We demonstrated that this approach identified disease‐linked gene expression patterns not previously visible through conventional data analysis of unstratified patients.We examined a cohort of 30 patients with SLE and 29 healthy donors for differentially expressed genes by RNA‐seq, alongside three publicly available independent data sets   is transcriptionally the most similar to healthy donors, compared to C2‐C4  gene signature was reduced in C4. M1 and M2 macrophage gene signatures were not significantly altered  criteria  classifier using the three independent Data sets (1 + 3 + 4), which we then used to classify the patients in Data set 2 (Supplementary figure TNFSF13B) overexpression is well established as a driver of autoimmunity,TNFSF10 mRNA  expression is also upregulated in SLE,CFLAR), previously reported to be upregulated in blood B cells of patients with SLE, and correlating with disease severity.CFLAR overexpression in C3 and C4  is a TLR3‐inducible negative regulator of noncanonical NF‐κB, and the expression of PELI1 was negatively correlated with disease severity.PELI1 was not significantly underexpressed in any SLE clusters, but was upregulated in C3 and C4, possibly induced for NF‐κB regulation  was identified as a negative regulator of B cells, and lack of GILZ drives autoimmune disease .We examined numerous SLE‐associated genes previously identified by GWAS.RETN gene, encoding the proinflammatory adipokine resistin, was upregulated in patients with active flares only , which suggests increased neutrophil‐mediated antibacterial activity; NGAL is also a biomarker of kidney injury.In Data set 2, 6 of the 30 patients with SLE had flares, who diverged further from healthy donors when visualised by PCA Figure a. While y Figure c. ResistA universally effective and safe treatment for SLE remains an unmet need because of the heterogeneity of clinical presentations, associated with unpredictable responses to current treatments.Our stratification method differentiates patient subgroups with four different gene expression profiles C1–C4), using whole‐blood transcriptomics to obtain a snapshot of the immune system, and we examined three study populations. This stratification may possibly have a use in improving the design of clinical trials, by more appropriately targeting specific clusters of patients with SLE who possibly express pathology‐relevant genes more homogeneously, suggesting a more consistent mechanism of action underpinning disease in each cluster Figures b and 4. , using wet al.et al.Similar to us, previous studies using microarrays have described distinct clusters of SLE patients in whole‐blood transcriptomic data.RETN gene, encoding the proinflammatory adipokine resistin . RNA libraries were prepared for sequencing using standard Illumina protocols. RNA‐sequencing (RNA‐seq) was performed on an Illumina HiSeq 2500 platform ; 100‐bp single‐end, stranded reads were analysed with the bcl2fastq 1.8.4 pipeline. Sequence read data are available on Gene Expression Omnibus (GSE112087). Sequencing of the same sample in two lanes showed comparable results (Supplementary figure featureCounts function of the Subread software package (v1.6.1);Publicly available data sets used in this study are listed in Table 2‐transformed with an offset of 1, and samples in each data set were computed as the log2 fold‐change (log2fc) against the matching healthy control group mean. These processing steps were useful to reduce the distracting effects of extreme values and skewness typically found in RNA‐seq data.Read counts were normalised by the upper‐quartile method, to correct for differences in sequencing depth between samples, using edgeR.k‐means function . The number of clusters was chosen based on unsupervised hierarchical clustering with MATLAB. ECOC classifiers, which contain several support vector machines for multiclass identification, were generated using MATLAB. Random forest classifiers were generated using MUVR.Principal components analysis and PLSDA were performed using the mixOmics R package voom (a function of the limma package).The limma/edgeR workflow was used for differential expression analysis, considering each data set as a batch.high CD11b+, CD49d−) by flow cytometry. Whole‐blood samples were stained for 15 min at room temperature before being fixed with BD lysing solution (BD) and acquired on a MACSQuant 10  with subsequent analysis done with FlowJo software .Whole‐blood samples collected into lithium heparin tubes (BD) were examined for frequency of circulating neutrophils  were obtained using StringTie (v1.3.4) and Ballgown (v2.12.0) R packages.P < 0.05 (*), P < 0.01 (**), P < 0.001 (***), P < 0.0001 (****) or not significant (n.s.).The mixOmics and MUVR R packages were used for multivariate analysis using count data.WAF conducted the analysis, wrote source code, produced the figures and wrote the manuscript. FM, EFM, KM, MA, MN and NJW reviewed the manuscript. KM, MN, MA, EFM, NJW, AYH and EFM generated Data set 2.KM, MN, MA, EM and NJW are employees of CSL Ltd.Written informed consent was obtained from all subjects.Ethics approval was obtained from the Human Research Ethics Committee at Monash Health. Click here for additional data file.


When genomics researchers design a high-throughput study to test for differential expression, some biological systems and research questions provide opportunities to use paired samples from subjects, and researchers can plan for a certain proportion of subjects to have paired samples. We consider the effect of this paired samples proportion on the statistical power of the study, using characteristics of both count (RNA-Seq) and continuous (microarray) expression data from a colorectal cancer study.We demonstrate that a higher proportion of subjects with paired samples yields higher statistical power, for various total numbers of samples, and for various strengths of subject-level confounding factors. In the design scenarios considered, the statistical power in a fully-paired design is substantially  greater than in an unpaired design.For the many biological systems and research questions where paired samples are feasible and relevant, substantial statistical power gains can be achieved at the study design stage when genomics researchers plan on using paired samples from the largest possible proportion of subjects. Any cost savings in a study design with unpaired samples are likely accompanied by underpowered and possibly biased results.The online version of this article (10.1186/s12864-018-5236-2) contains supplementary material, which is available to authorized users. When a genomics research team is planning a study that will involve testing multiple biomarkers (or features) for differential expression, many important decisions must be made at this study design stage. Previous literature has addressed replication and statistical analysis plans . OfteAt an extreme, normal tissue sample expression from a database  could beOver the past several years, the genomics literature has included attention to principles of good experimental design for a variety of genomics platforms , 16, 17.Statistical power in this context refers to the probability that truly differentially expressed features will be called statistically significant. Higher statistical power can be achieved through a combination of good study design and appropriate statistical analysis plan , 16–18. The purpose of this manuscript is to draw attention to (and to quantify) the statistical value of paired samples in genomics studies where such paired samples are feasible. We use a hypothetical  scenario of a genomics research team that, with the reality of limited resources, has determined that they can afford to run N samples for a given study. In designing this study, the team can collect paired samples on some percentage of subjects in the study; for the sake of conciseness, we assume the remaining subjects with unpaired samples will be evenly split between conditions , for a total of N samples. We evaluate the statistical power in tests for differential expression, under a variety of scenarios (number of samples N and percent of subjects with paired samples), for both count (such as RNA-Seq) and continuous (such as microarray-based) expression data. The power evaluations are made using the characteristics of microarray-based miRNA and RNA-Seq-based gene expression data from a large colorectal cancer (CRC) study  that included a majority of subjects having paired samples . This colorectal cancer study and the power evaluation approaches are described in the Methods section below.We highlight the role of possibly confounding subject-level factors, which are subject characteristics that affect expression values in both normal and tumor samples. Examples of such possible confounding factors include dietary or lifestyle factors. Additionally, tissue samples may vary between hospitals in how they are handled during processing as well as pathological observations, introducing additional sources of potential bias in non-paired data. We demonstrate that in the presence of such confounding factors, a higher proportion of subjects with paired samples leads to a considerable gain in statistical power. In addition, using our CRC data we demonstrate the substantial power gain that we observe when comparing an analysis of paired samples to an analysis of unpaired samples.Figure While the overall power in Figs. Figures Figures The power approximations in Figs. p-values). In particular, whereas the paired design resulted in 8856  features called significantly differentially expressed at FDR .01, unpaired designs 1 and 2 resulted in 6865 and 7076 significant features, respectively. Accounting for covariates did mitigate the loss of power somewhat in unpaired design 1, resulting in 7192 significant features; but accounting for covariates in unpaired design 2 actually resulted in only 6755 significant features.Figure Figure The only exception to the “higher proportion of subjects with paired samples yields higher statistical power” result seen in Figs. If fewer than about 10 % of subjects have paired samples and there is very weak subject-level confounding, then from a statistical power perspective it may be tempting to drop one of each subject’s paired samples, and run an analysis on fully unpaired data. However, this will lower the effective sample size, further reducing statistical power. Without clear a priori knowledge of the magnitude of subject-level confounding in a planned study, the best strategy would be to plan for as high a proportion as possible of subjects with paired samples, and use all valid samples (even if unpaired) that are available.p-values in the paired design analysis tended to be systematically smaller than in the unpaired designs’ analyses   or GenomN = 20 should suffice. Figure Figures When subject-level confounding , making it less likely that those genomics researchers will be able to detect truly differentially expressed features with meaningful fold changes. Any cost savings in a study design with unpaired samples are likely accompanied by biased and underpowered results. The greatest statistical power is achieved by having paired samples on Study participants include all incident colon and rectal cancers between 30 and 79 years of age, who were living along the Wasatch Front in Utah or were members of the Kaiser Permanente Medical Care Program (KPMCP) in Northern California, and with primary adenocarcinoma diagnosed between October 1991 and September 1994 for colon, and between June 1997 and May 2001 for rectal. The study was approved by the Institutional Review Board of the University of Utah and at KPMCP, and all participants provided written informed consent. Numerous demographic, dietary, lifestyle, and tumor characteristic covariates were recorded for study participants, as reported previously , 23.miRNA from both carcinoma and adjacent normal samples for study participants were processed  as described previously , 24, witA subset of colon cancer study participants were chosen for gene expression measurement using RNA-Seq, as described previously . RNA-Seqij) of mRNA fragments mapping to the biomarker can be parameterized as follows:A log-linear negative binomial model is often used for biomarker count data, such as from RNA-Seq , 18, 26.ij is the total number of counts  from subject j under treatment i, and is used as an offset  to account for different total amounts of genomic material collected from different samples. The parameter pij represents the probability that any given mRNA fragment in the sample from subject j under treatment i would map to the biomarker. μ is an intercept term, TRTi is the effect due to treatment level i, and Subjj is the effect due to subject j.Here, Ncritical, and could result from a combination of dietary, lifestyle, genetic, and other unknown factors, all with biomarker-specific confounding effects.) This factor is confounded with the Subject effect in the model, so the model can be rewritten asEach subject j has a possibly confounding factor  that will affect the value of both their normal and their tumor expression values. , and Subj is a random effect . The presence of both fixed and random effects makes this a “mixed” model, and the non-normal distribution of F with a linear model for the expected value of Y (after log link function transformation) makes this a “generalized linear” mixed model.j are assumed to be independent and identically distributed Normal for those subjects in Eq. Because Subj is a random effect, the values of Subjmentclass2pt{minimsee lme4  and GLIMsee lme4  documentA negative binomial model for biomarker count data can be fit by popular R packages such as DESeq2  and edgeIn these data, many thousands of genes were significantly differentially expressed (even after multiplicity corrections), but those are not of greatest interest in this manuscript. Instead, it is most relevant to note that even for very small TRT effects, statistical significance was noted. Based on the results for our CRC data, a range of mentclass2pt{minimThe probability distribution method  is a verThe probability distribution method was chosen after attempts at the simulation method for power approximation  for counA linear mixed model can be used for continuous expression data, such as for cDNA , mRNA 3–33, or mi = 1,2 , and j = 1, …, #subjects. This model assumes that the data have been appropriately preprocessed and normalized. As in the model for count data above, each subject j has a possibly confounding factor  that will affect the expression of both their normal and their tumor expression values. This factor is confounded with the Subject effect in the model, so the model can be rewritten asHere, 1 = TRT2 = 0. The random subject (or confounding factor) effects are assumed to follow a Normal distribution.If the biomarker is not differentially expressed, then TRTσ2 were both estimated, along with the treatment effect. The estimated σ2 was roughly linearly associated with Using the lmer function of the R package lme4 , model  [For demonstration purposes of the practical effect of full vs. partial (or no) pairing, we randomly selected 100 subjects (from 169) with paired samples in the RNA-Seq data, and fit model (1) on each of the 17,462 protein-coding genes. The test for differential expression (of the TRT effect) in this fully paired design  yielded a te (FDR) .p-values were obtained for each of the 17,462 protein-coding genes.Unpaired design 1 consisted of using the same data as in this paired design, for the same 100 subjects. A variation of model (1) was fit, excluding the subject random effect, which is equivalent to ignoring the pairing. Again FDR-adjusted Unpaired design 2 used data for the same 100 subjects, but with 50 of those subjects randomly selected to be tumor-only and the other 50 to be normal-only . Then of the subjects not among those 100, 50 others were randomly selected  to be tumor-only and another 50  to be normal-only. Data for these 200 total samples were used to fit a variation of model (1), excluding the subject random effect , and again FDR-adjusted p-values were obtained for each of 17,462 protein-coding genes.One way to look at these three designs is as follows. The paired design is intended to represent the ideal, with every subject providing paired samples and the analysis accounting for the pairing. Unpaired design 1 is intended to represent partially lost potential, with every subject providing paired samples but the analysis not accounting for the pairing. Unpaired design 2 is intended to represent fully lost potential, with no paired samples for any subject.To demonstrate the potential impact of accounting for subject-level covariates in an unpaired design, we again ran a variation of model 1) on both unpaired design 1 and unpaired design 2, but instead of the subject random effect we included a few covariates as fixed effects (or continuous covariate in the case of age at diagnosis). The covariates included tumor site , tumor SEER summary stage , age at diagnosis, MSI tumor status, and sex. These covariates were chosen only for demonstrative purposes as potential subject-level confounding factors that have previously been shown to be associated with expression levels in colorectal cancer . Table 1 on both Additional file 1:SAS code to implement the probability distribution method for calculating approximate power with count expression (such as RNA-Seq) data (SAS 9 kb)Additional file 2:SAS code to implement the probability distribution method for calculating approximate power with continuous expression (such as miRNA) data (SAS 9 kb)Additional file 3:Approximate power contours for continuous data, such as miRNA, based on the  simulation method. Contours are given for the average and median power across 100 simulations. In an effort to quantify the amount of variability across the 100 simulations, contours are also given for the standard deviation (SD) and range (max minus min) of power across the 100 simulations. (PDF 68 kb)Additional file 4:Approximate false discovery rate contours for continuous data, such as miRNA, based on the  simulation method. Contours are given for the average and median FDR across 100 simulations. In an effort to quantify the amount of variability across the 100 simulations, contours are also given for the standard deviation (SD) and range (max minus min) of the FDR across the 100 simulations. (PDF 101 kb)


M and values) are estimated neglecting pixel-level uncertainties, which may reflect noise and systematic artifacts. For example, intensity values from two-color microarray data are estimated by taking the mean or median of the pixel intensities within the spot and then subjected to a within-slide normalization by LOWESS. Thus, focusing on estimation and normalization of gene expression profiles, we propose a spot quantification method that takes into account pixel-level variability. Also, to preserve relevant variation that may be removed in LOWESS normalization with poorly chosen parameters, we propose a parameter selection method that is parsimonious and considers intrinsic characteristics of microarray data, such as heteroskedasticity. The usefulness of the proposed methods is illustrated by an application to real intestinal metaplasia data. Compared with the conventional approaches, the analysis is more robust and conservative, identifying fewer but more reliable differentially expressed genes. Also, the variability preservation allowed the identification of new differentially expressed genes. Using the proposed approach, we have identified differentially expressed genes involved in pathways in cancer and confirmed some molecular markers already reported in the literature.Faced with the lack of reliability and reproducibility in omics studies, more careful and robust methods are needed to overcome the existing challenges in the multi-omics analysis. In conventional omics data analysis, signal intensity values (denoted by  The growing number of omics datasets  and the recent advances in multi-omics integration approaches have contributed to the better understanding of biological mechanisms and also the emergence of the personalized medicine. However, the lack of reliability and reproducibility in omics studies stands as one of the biggest obstacles in bridging the gap between research and practice of personalized medicine . ConsideImage processing and fluorescence analysis are the preferred approaches for data quantification in microarray technologies. Although microarrays have been predominantly used since the end of the nineties to measure gene expression levels, they remain widely used to detect other omics data types, including microRNA expression, DNA methylation, single-nucleotide polymorphisms (SNPs), and copy number variants (CNVs) . After hThe continuance of the microarray technology can be mainly explained by the availability of many datasets in public repositories, such as the Gene Expression Omnibus (GEO)  and ArraSeveral preprocessing and normalization procedures have been proposed to remove biases due to the inhomogeneity of the background and the different fluorescence properties of the dyes. However, biases introduced in the image analysis step, which includes spot segmentation and signal extraction, have not received the same attention, and those may partially explain the existing reliability and reproducibility problems in omics studies. Particularly, several factors, including image resolution, scanner settings, effectiveness of the segmentation algorithm, and unexpected behaviors during hybridization, may lead to errors in spot localization and classification of the pixels . Thus, spot intensities are usually noisy and that high pixel–level variability leads to uncertainty in microarray quantification and correlates with variability between replicate spots on duplicate slides .Given that even state-of-art image processing tools are susceptible to errors that significantly influence the variability of the data derived from microarray images , new segDespite the common use of pixel-level variability measures as data quality criteria for filtering purpose, the conventional microarray analysis is solely based on statistical measures of location of the spot intensities . To imprPixel-level uncertainties have been taken into account by many spot quantification algorithms in the literature, but requiring all pixel values to be available. Some of them are interested in improving the log-ratio estimator. Particularly, the method proposed by  is a logM) and average log intensity (A) of a microarray spot that accounts for pixel-level variance and covariance between channels. For a spot t, these values are denoted by tM and tA, respectively   is one oAlthough all these methods have shown superiority over LOWESS normalization with a fixed arbitrarily chosen smoothing parameter, they lack in taking into account any heteroskedasticity in the data. In addition, they usually suffer from a poor bias–variance trade-off, tending to choose small smoothing values, which yield unnecessarily complicated (with high variance) LOWESS fits.The second contribution of this paper is a data-driven method for selecting the smoothing parameter of the LOWESS normalization process. Inspired by the previous proposed methods, we choose the optimal smoothing value by minimizing a mean squared error criterion. However, our selection method also takes into account heteroskedasticity of the microarray data and offers a better bias–variance trade-off by selecting from among the low-MSE fits the one that is the most parsimonious. The parameter selection is obtained by solving a discrete optimization problem and is based on conventionally accepted ideas for analysis of M-plots—a graphical tool showing the curve of the MSE against the effective degrees of freedom of the estimate .Given that the primary application of DNA microarrays has been to measure gene expression levels, we focus in this paper on variation-preserving estimation and normalization methods for gene expression levels from two-channel (or two-color) microarrays. However, it is straightforward to adapt the same ideas to improve analysis of other types of microarray data, even from single-channel technologies.tM and tA values were compared with the conventional estimators that neglect the pixel-level variability. In addition, we compared the proposed method for selecting the LOWESS smoothing parameter with OLIN, as it is conceptually similar to the other existing methods and can be applied even to microarray experiments with few or no replicates. Results show that a more robust and conservative analysis is performed when the LOWESS smoothing parameter is selected by our method, potentially reducing the number of false-positive differential expressions. Besides, both the pixel-level variabilities incorporated by the proposed estimators for the tM and tA values and the variability preserved by our more parsimonious normalization method contributed to the identification of new differentially expressed genes. Thus, the proposed methods may also reduce the false-negative rate.The proposed methods were evaluated by a differential gene expression analysis from real intestinal metaplasia and normal microarray samples. The proposed estimators for the tM and tA values).Two procedures that critically affect the adequacy of microarray data analysis are the spot quantification, which extracts summarized quantitative measures of the pixel intensities within each spot of the microarray slide, and the within-slide normalization, which removes dye-specific biases and other systematic noises simultaneously from all logged spot intensities .In the section Intestinal Metaplasia Database, we describe a gene expression dataset used to illustrate the application of our proposed methods. In the section Improved Estimators for the Due to a chronic inflammatory process, the normal squamous mucosa of the stomach may be replaced by columnar intestinal-type epithelium, characterizing a disease called intestinal metaplasia of the stomach. Since adenocarcinoma of the stomach and inflamed intestinal mucosa are strongly associated , intestiWe analyzed data from a two-color microarray experiment with tissues samples from 90 different subjects, being 35 from tissues representing type II intestinal metaplasia and 55 from tissues representing the normal condition, obtained from the Tumor Bank at A.C. Camargo Cancer Center/Antonio Prudente Foundation.Agilent Feature Extraction software, version 9.5, where statistics  of the foreground and local background pixels were computed for each spot, in both test and reference channels. Each microarray spot contains about 60 foreground pixels.It was used the standard reference design , in whicThis study was carried out in accordance with the recommendations of the international guidelines for investigations involving human beings with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Institutional Committee of the A.C. Camargo Cancer Center (process number 1023/07).G (green), following this usual notation, denoted by tjR and by tjG, the intensity value of the jth pixel within the th spot, respectively, in the test and reference channel. The relative expression of pixel j within spot is denoted by tjM and defined as follows:Usually, in microarray analysis, the test channel is denoted by (red), and the reference channel is denoted by tjA and defined as follows:The average expression of pixel within spot is denoted by Usually, image analysis software does not provide all pixel intensity values within each spot. Nonetheless, it provides several descriptive statistics of the foreground and background pixel intensities, including sample estimates for the mean, median, variance, and covariance between the two channels.tjM and tjA by using the multivariate delta method  and Var (tjG), which are assumed to be independent of the background correction. Note that mean and variance estimates are calculated across observed foreground pixel intensities within the spot at the respective channel.Let We can derive improved estimators for tjM and tjA values, given byNote that the conventional estimators for the tjM and tjA. Thus, the conventional estimators ignore the known measures of pixel-variability, which represent uncertainties in the gene expression measurements.are approximations of, respectively, Intestinal Metaplasia Database. Since these estimators may suffer from numerical instability if the corrected foreground signals, normexp method  and (2) that define, respectively, tjM and tjA, as shown in Appendix 5. The variance estimators for tjM and tjA, for pixels j within spot t are:Since we have also available the sample covariance between tjM and represent pixel-level uncertainties of the th spot. They can be used, for instance, for assessing the quality of the th spot or for constructing confidence intervals for the parameters The variances of tM and tA values.To simplify the notation, we will denote the estimates for tjM intensity values the dependent dye-specific biases and other systematic errors by using some within-slide normalization method.It is necessary to remove from these tA observed value to a smoothed tM value, tM value to obtain a residual value representing, presumably, the biologically relevant gene expression level.In the LOWESS within-slide normalization method, one estimates for each microarray slide a smoothing function f, which, in the nearest neighbor method, is a number between and indicating the proportion of data used in each local fit, affects the bias and the variance of the fit.An appropriate LOWESS estimation depends on the choice of its parameters. According to loader , the weiSpecifically, the higher the degree of the local polynomial (related to the complexity of the model), the lower the bias of the fit . However, the additional parameters of this more complex model increase the variance of the fitted values, yielding a poor generalization ability . Thus, to avoid unstable LOWESS estimates, several references as  recommenf on the bias and variance of the fit are opposite to those of the degree of the local polynomials. Since the f parameter indicates the number of observations that will be used in the local polynomial estimation, when f value is large, a simple polynomial may not fit well to all observations in the neighborhood, distorting or ignoring essential features. In other words, the estimation of the smoothing function can be significantly biased. On the other hand, when a low f value is chosen, the number of observations may be insufficient to capture the general behavior of the data, resulting in a very noisy (large variance) fitness function.The effects of the smoothing parameter f parameter, focusing on microarray data normalization. Our method takes into account the intrinsic characteristics of the bias and variance of the fit as well as of gene expression data.In the next section, we propose a method for selecting a value for the f parameter value.For microarray data normalization, the ideal LOWESS fitted curve captures only trends and effects from systematic errors, retaining all biological variation. However, it critically depends on the choice of the f values varying from 0.05 to 0.9. The improved estimation method was used to obtain the tM and tA values, that is, the Figure 2 illustrates the MA plot of the microarray slide shown in Figure 1B, with different LOWESS fits yielded by The quality of a LOWESS estimator can be assessed by the MSE, which measures how close the estimator Since the real curve μ is unknown, we need a criterion to evaluate the MSE. Under the assumption of heteroskedasticity, Cleveland and Devlin  propose A to the tA observed points:Considering T × T matrix L which maps the observed to the fitted values:Consider the Two commons definitions of the effective degrees of freedom of tM, across T spots of a microarray slide, is constant and equals to σ2, the Mallows’ Cp for local fitting is defined as:Supposing that the variance of 2 can be estimated as follows:T × T diagonal matrix Σ, whose th diagonal element is given by a non-homogeneous variance tM, a robust MSE estimation can be achieved by using the HRCp criterion, defined as:When heteroskedasticity is present, Mallows’ Cp criterion is not an appropriate MSE estimator. Considering the f = 0.1. Since the local variance estimates can be very noisy, it may be appropriate to smooth them using a gamma kernel.According to f value which minimizes a measure of error of the LOWESS fit, such as the MSE criterion  are shown in Figure 3. Dots show MSE estimates (by HRCp criterion) and respective degrees of freedom (by v2 definition) of LOWESS fits  so that the M-plot curve shows only the effect of the f parameter on the bias–variance compromise. Large f values tend to yield simple fits (with fewer degrees of freedom), which have a small variance, but a large bias. On the other hand, minimal f values tend to yield complex fits (with many degrees of freedom), which have a small bias, but a large variance.M-plots illustrating the f value (0.2). However, any f value within the flattening region near to the minimum (the region with light-colored dots) is a good choice, in the sense that it yields a low-MSE fit . The biggest dot in each M-plot indicates the selected f value. The detection of the flattening region is made by searching points for which the derivative of the MSE curve is small. We check for each sequence of three points near the minimum whether the difference between the MSE values is small. If so, these points are considered as belonging to the flattening region.For the microarray slide in Figure 3, a selection method based only on the minimization of the MSE curve would choose the smallest evaluated -MSE fit . Dependif parameter selection method can be summarized in the following discrete and constrained optimization problem. Consider a sequence of l different values for f, {f1, f2, ... , lf}, and denoted by kf for the f parameter. Also, let:The v2 function provides the effective degrees of freedom of a given fit, the selected f value is the solution *f, if it exists, of the following problem:Since f value that yields the fit with lowest MSE estimate is selected. Specifically, the f parameter value is selected by solving the following problem:If the minimum of the M-plot curve is far away of the point corresponding to the second lowest MSE estimate, the previous problem has no solution. In that case, the Intestinal Metaplasia Database by using all discussed methods and compared the identified differentially expressed genes.To investigate the effects of the proposed methods, we preprocessed the data described in the section normexp method with offset value of for removing the background influence. Then, we compute the tM and tA values both by the conventional estimation methods, defined in Eq. (5) and (6), and by the proposed estimation methods, defined in Eq. (3) and (4). The LOWESS within-slide normalization was carried out as discussed in the section Optimal Selection of the LOWESS Parameters. For comparison purpose, the f smoothing parameter was selected both by the OLIN method  and by the proposed method, discussed in the section LOWESS Smoothing Parameter Selection. Since data from all microarray slides present overdispersion, we used the HRCp criterion as cost function of our selection method.First, we applied the Therefore, the following four preprocessing procedures were applied separately to the original data:tM and tA and LOWESS within-slide normalization using f parameter selected by OLIN;Conventional estimation for tM and tA and LOWESS within-slide normalization using f parameter selected by OLIN;Improved estimation of tM and tA and LOWESS within-slide normalization using f parameter selected by the proposed method;Conventional estimation of tM and tA and LOWESS within-slide normalization using parameter selected by the proposed method.Improved estimation of f parameter, according to the proposed selection method with HRCp criterion, for the entire database, separated by normal and intestinal metaplasia conditions . In the first plot, the LOWESS curve was fitted on the f values was close to 0.5.tM and tA values were estimated by the conventional and by the proposed estimator. Such behavior has been reported in the literature, implying that the optimal f values according OLIN are usually close to the default one  for most of the slides. Same results were obtained when the ault one .After preprocessing the data, a two-sample t-test assuming unequal variance was performed for each spotted gene to determine whether its expression is statistically different between gastric tissues in normal and intestinal metaplasia groups. However, since we are interested in directly assessing the impact of each proposed method on the t-statistics and p-values rather than making inference about differential expression, the comparative study was performed before applying a multiple testing correction.Results of a pairwise comparison among the p-values and t-statistics obtained by the four preprocessing methods are shown in The left-column plots show that most of the points are distributed around the 45-degree line. Thus, the p-values and, consequently, the total number of differentially expressed genes, even at a lower significance level, were similar among the four methods.tM and tA values with the proposed method, which takes into account the pixel-level uncertainties. The genes represented by blue plus signs were identified as differentially expressed only when using the proposed estimator for the tM and tA values.The first- and second-row plots show how p-values and t-statistics were affected by estimating the tM and tA values.The genes represented by green crosses were identified as differentially expressed only when using the conventional estimator for the f parameter is selected by OLIN (first-row plots), it is clear that the within-group variability decreases when using the proposed estimators for the tM and tA values. When the LOWESS parameter is selected by our method (second-row plots), there is still a reduction in the within-group variability. However, this impact is less clear because of the variability introduced when the LOWESS f parameter is selected by our method.When the LOWESS f parameter. The genes represented by blue plus signs were identified as differentially expressed only when f was selected by the proposed method. The genes represented by green crosses were identified as differentially expressed only when selecting f by OLIN. It is clear that, for most genes, both within-group variabilities increased, implying that the normalization procedure was more conservative, and thus, more potentially relevant information is retained. In addition, for many genes, the increase in the within-group variability was counterbalanced by an increase in the distance between the groups. Such effect is even most pronounced when the proposed estimator for the tM and tA values are used. Thus, their respective p-values reduced enough to consider them as differentially expressed genes.The third- and fourth-row plots compare p-values and t-statistics obtained by OLIN and the proposed approach for selecting the LOWESS The diagrams in f parameter selected by our method, fewer genes are identified as differentially expressed. However, regardless of the normalization method, more genes could be identified as differentially expressed when the tM and tA values were estimated by the proposed estimation method that incorporates pixel-level variability. Given that both proposed methods make the analysis more robust by incorporating and preserving information neglected by the conventional methods, we can argue that they are contributing to the reduction of both false-positive and false-negative rates.Note that the four methodologies are quite different in terms of which genes were identified as differentially expressed. As a consequence of the more conservative (milder) noise reduction performed in the LOWESS within-slide normalization procedure with To check the consistency of our analysis, we compared our results with those reported in the literature. Out of the genes which are associated with intestinal metaplasia according to the Gene Expression Omnibus platform  of the NGreater differences in inference were observed among the genes whose p-value is close to the significance level. These genes have a more subtle differential expression, which can be easily damaged by measurement errors and poor estimation and normalization methods. Thus, the more accurate and careful analysis provided by the proposed methods is especially important for making decisions on the differential expression of these more sensitive genes.tM and tA values and our selection method for the LOWESS f parameter. Thus, the estimation of the tM and tA values by the proposed estimators was crucial in determining the differential expression of the HSPB1 gene.Two replicates of the HSPB1 gene could not be identified as differentially expressed when using both the conventional estimators for the tM and tA values were also used. In contrast, the gene KRT14 was no longer identified as differentially expressed when the LOWESS f parameter was selected by our proposed method.The genes PTEN, CTNNB1, MLH1, CXCR4, and CXCR1 could only be identified as differentially expressed when the LOWESS parameter was selected by our proposed method. Particularly, the gene CXCR4 only was determined as differentially expressed when the improved estimators for the In the following, we briefly describe the association of those genes with intestinal metaplasia of the stomach according to the literature data:HSPB1 : It has a protective role against stress-induced cell damage, and its expression has been considered critical for mucosal protection in the stomach . Also, iPTEN (phosphatase and tensin homolog): It has been identified as overexpressed in intestinal metaplasia and is a known marker for tumorigenesis and progression of gastric carcinoma .CTNNB1 (beta-catenin 1): It is a canonical oncogene that has been identified as overexpressed in intestinal metaplasia and gastric adenocarcinomas .MLH1 (mutL homolog 1): Its expression has been reported as absent or downregulated in intestinal metaplasia, dysplasia, and gastric cancers .CXCR4 (chemokine receptor type 4): Its expression has been associated with the staging of gastric cancer, being reduced in the majority of gastrointestinal tumors and significantly higher in patients with advanced stages of gastric cancer .CXCR1 (C-X-C motif chemokine receptor 1): It has been reported to be strongly expressed in gastric carcinoma .KRT14 (keratin 14): It is a squamous cell marker that is down-regulated by CDX2 transfection . In addiBy performing a gene enrichment analysis, we identified, at a significance level of 5% (after FDR correction), 31 differentially expressed genes that are involved in cancer. Their respective p-values and fold changes are shown in Particularly, two replicates of the CCND1 gene and the LAMB2 gene were identified as differentially expressed only by the conventional approaches, suggesting that they may be false positives. Next, we briefly describe their association with cancer:CCND1 (cyclin D1): In contrast to its underexpression identified by the conventional analyses, it has been frequently reported as overexpressed in intestinal metaplasia, human neoplasias, and several tumors .LAMB2 (laminin subunit beta 2): Although its expression has been associated with some carcinomas, ts expression is tightly regulated in normal human tissues and in disease .Faced with the growing trend of multi-omics data integration in the midst of a replication crisis, improved microarray data analyses are crucial to identifying more reliable results .tM and tA values, which takes into account the pixel-level variability. Specifically, we applied the multivariate delta method to derive estimators for the expected values of tM and tA, considering their Taylor’s expansion up to the second-order terms. The conventional estimators, nonetheless, approximate the expected values considering only the zeroth-order term. Since the functions that define tM and tA are analytic (they are combinations of logarithmic function through addition or subtraction), the higher the number of terms of the Taylor expansion, the better the approximation of the function. Thus, we expect that the proposed estimators provide a better quantification of the hybridization signal. Also, by using these improved estimators, pixel-level dispersion can play an essential role in the analysis, increasing reliability.Given that several pixel-level summary statistics are readily available in microarray databases, but are usually discarded in conventional approaches, we propose an improved estimation method for the tM and tA values have to be properly normalized. Thus, we also propose a method for selecting the LOWESS smoothing parameter f that provides an optimal bias–variance compromise, considering some specific characteristics of microarray experiments, such as heteroskedasticity. This optimal normalization method leads to a more parsimonious correction of the systematic biases and, consequently, to greater preservation of the biological variation of interest.To minimize the propagation of errors, the By using the proposed methods, more variability information is considered and retained, improving inferences and preventing false conclusions. Thus, we expect to perform a more conservative analysis, where possibly fewer but more reliable differentially expressed genes are identified. In other words, we expect a reduction in both the false-positive and false-negative error rates.tM and tA values and significantly affected by the LOWESS within-slide normalization using a smoothing parameter selected by the method. Both proposed methods tend to increase the within-group variability (the denominator of the t-statistic). However, for many genes, such increase occurred along with an increase in the difference between the group means , significantly reducing their respective p-values. Thus, many genes were identified as differentially expressed only when the proposed methods were used and some of them have been validated by other studies.Besides the theoretical support, relevant empirical observations could be drawn by a comparative study between the methods using real intestinal metaplasia microarray data. The results shows that inferences on differential gene expression were moderately affected by the incorporation of the pixel-level variability in the estimation of the It is important to remark that most of the genes reported in the literature as differentially expressed in intestinal metaplasia were validated with a very strong association with the disease. Thus, these genes are probably more robust to difference approaches for estimating and normalizing the gene expression levels. On the other hand, genes sensitive to methods that address essential uncertainties in measurements are precisely those plagued with major reproducibility issues. Measurement error is one of the most damaging sources of error and has been neglected in many published analyses, thereby increasing uncertainty in parameter estimates and even inflating the estimates of effect sizes . Thus, ptM and tA values are defined by comparing pairs of arrays instead of pairs of channels and that the LOWESS normalization is applied to all distinct combination of two arrays. Although not so straightforward, it is also possible to adapt our methods to handle next-generation sequencing (NGS) data. Recently, Law et al.  microarray data. Particularly, the cyclic LOWESS normalization method  could bew et al.  showed tw et al. . Therefow et al. , a possihttps://github.com/adele/omicsMA, and the latest release is available at https://github.com/adele/omicsMA/releases/latest.The omicsMA R package contains the source code of the proposed methods and part of the metaplasia dataset analyzed in this study. It was implemented using R, version 3.5.1, and depends on the locfit , maigesPThis study was carried out in accordance with the recommendations of the international guidelines for investigations involving human beings with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Institutional Committee of the A.C. Camargo Cancer Center (process number 1023/07).AR and RH conceived of the presented ideas. AR derived the models, implemented the methods, and analyzed the data. AR wrote the manuscript with support from RH and JS. All authors discussed the results and contributed to the final manuscript. RH and JS supervised the project.This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001; National Council of Technological and Scientific Development (CNPq); and NAP eScience – PRP - USP. It was also supported by the Foundation for Research Support of the State of São Paulo (FAPESP) .The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.