Quantitative proteomics has matured into an established tool and longitudinal proteomics experiments have begun to emerge. However, no effective, simple-to-use differential expression method for longitudinal proteomics data has been released. Typically, such data is noisy, contains missing values, and has only few time points and biological replicates. To address this need, we provide a comprehensive evaluation of several existing differential expression methods for high-throughput longitudinal omics data and introduce a Robust longitudinal Differential Expression (RolDE) approach. The methods are evaluated using over 3000 semi-simulated spike-in proteomics datasets and three large experimental datasets. In the comparisons, RolDE performs overall best; it is most tolerant to missing values, displays good reproducibility and is the top method in ranking the results in a biologically meaningful way. Furthermore, RolDE is suitable for different types of data with typically unknown patterns in longitudinal expression and can be applied by non-experienced users. Longitudinal proteomics holds great promise for biomarker discovery, but the data interpretation has remained a challenge. Here, the authors evaluate several tools to detect longitudinal differential expression in proteomics data and introduce RolDE, a robust reproducibility optimization approach. Currently, MS-powered quantitative proteomics can be considered as an established method routinely used for proteome exploration in biomedical research3.In the course of the past few decades, mass spectrometry (MS)-based proteomics has developed significantly and emerged as a powerful tool for clinical biomarker discovery5. While requiring more measurements per individual, less individuals are required to achieve the same statistical power as in cross-sectional studies5. In addition to having more statistical power, longitudinal study designs deliver information concerning the changes in the studied individuals over time. In the context of high-throughput transcriptomics, longitudinal experiments for detecting time-resolved gene expression changes have been performed already for more than two decades8. For proteome profiling, most of the experiments thus far have utilized cross-sectional study designs, but longitudinal proteomics experiments with two3 or multiple10 time points have begun to emerge.Longitudinal study designs are generally regarded as having more statistical power to detect differences between the examined study groups than cross-sectional designs11. With only a few time points, most of the tested longitudinal methods produced a high number of false positives11. When the number of time points was increased, the performance of many of the longitudinal methods also improved11. However, as currently relatively few time points are typical in biomedical studies, the usability of such methods which cannot perform well on short time series is limited and new approaches are required.A recent study comparing longitudinal methods for RNA-sequencing (RNA-seq) gene expression experiments discovered that most of the specific longitudinal methods performed worse than timepoint-wise analysis of the data using traditional pairwise differential expression tools when the number of time points was small (<8)11 is that many of them are specifically designed for discrete negative binomially distributed count data and, as such, are not directly applicable. Proteomics data is typically close to normally distributed after logarithm transformation and/or normalization12, making methods originally proposed for the analysis of longitudinal gene expression microarray data better suited. Among those methods, BETR 13 and Timecourse14 rely on a Bayesian framework, and Microarray Significant Profiles (MaSigPro) builds on a two-step regression strategy15. In addition, the popular R package for the analysis of microarray and sequencing data, Limma16, also contains tools for analyzing longitudinal differential expression. However, since these methods do not take into account the characteristics of MS proteomics data, they might not be optimal for proteomics experiments. In particular, as has been extensively observed in previous studies18, missing values are prevalent in proteomics data and their handling is not trivial18. This is particularly the case with the data-dependent acquisition (DDA) label-free proteomics approach, which is popular due to its cost efficiency, speed, and ability to handle complex samples19. Furthermore, MS data is prone to noise in quantification21, rendering especially the lowly abundant proteins subject to false positive detections22.In the context of proteomics data, another limitation of the longitudinal methods developed for RNA-seq data25. For example, Liu et al.10 used mixed-effects regression modeling with quadratic random terms to detect trends and differential expression patterns in the longitudinal expression of proteins between children developing type 1 diabetes (T1D) and healthy controls. However, no comprehensive comparison on the performance and application of the different approaches exists and, therefore, specific standard practices for analyzing differential expression in longitudinal omics data with regression modeling have not been established. Furthermore, the statistical frameworks developed for the analysis of longitudinal clinical variables might not be best suited for the analysis of longitudinal omics data, in which the number of individuals and the number of measured time points are typically small, but the number of simultaneous variables  is very large. Given the complexity of such data and the variety of potential methods available, the selection of an appropriate method is not straightforward.In addition to the methods specially designed for longitudinal omics data, different statistical modeling approaches have been utilized in their analysis. Several types of linear and non-linear regression-based approaches, with or without random effects, have been applied in various contexts10 to further illustrate its applicability to non-aligned measurements in a real clinical dataset.To facilitate the selection of a suitable method for the discovery of differential expression from longitudinal proteomics data, we present an extensive evaluation of altogether 15 longitudinal approaches and a baseline cross-sectional method, including a method named RolDE  that we develop here. The methods are evaluated in their ability to correctly detect longitudinal differential expression using over 3000 semi-simulated proteomics spike-in datasets with and without missing values, a varying number of time points and including a large variety of linear and non-linear longitudinal trend differences between the examined conditions. In addition, the reproducibility and reliability of the methods, as well as their ability to provide biologically meaningful results are assessed in two large-scale experimental biological datasets. Finally, after demonstrating the robustness and overall good performance of RolDE with various datasets and longitudinal differential expression trends, we apply it to a previously published longitudinal type 1 diabetes proteomics dataset26, Bayesian Estimation of Temporal Regulation (BETR)13, Linear Models for Microarray Data 16, Timecourse14, Microarray Significant Profiles (MaSigPro)15, Extraction of Differential Gene Expression (EDGE)27, Linear Mixed Model Spline Framework for Analysing Time Course Omics Data (LMMS)28, Omics Longitudinal Differential Analysis (OmicsLonDA)29, linear mixed effects regression modeling (Lme), and polynomial mixed effects modeling (Pme). The performance evaluation of the methods was conducted using over 3000 semi-simulated datasets generated on the basis of three spike-in proteomics datasets   dataset 32) Fig. . Reprodu 33 Fig. . In addi 33 Fig. . Finally 10 Fig. .Fig. 1IlFirst, we investigated the performance of the methods in the filtered semi-simulated spike-in datasets, where a single trend category per condition was generated for each dataset . The baseline method ROTS that ignores longitudinal trends also performed well with an IQR mean pAUC of 0.941, while the difference to the best performing method RolDE was highly significant (p < 10−15). Among the regression-based models, the lower order regression models (denoted by the extension L) performed overall worse than models of higher polynomial degree (denoted by the extension H).In the UPS1-based semi-simulated datasets involving five time points, the overall highest partial areas under the ROC curves (pAUCs) were obtained by RolDE with an interquartile range (IQR) mean pAUC of 0.977 Fig. , performp < 10−11).In the SGSDS-based filtered datasets with eight time points and no missing values, all the tested methods performed relatively well Fig. . Again, A closer look at the performance of the methods in the different trend categories suggested that the proposed method RolDE, together with Timecourse and BaselineROTS, performed consistently well in every category in both the UPS1-based and SGSDS-based datasets . In the SGSDS-based full datasets, with missing values also in the true positive spike-in proteins, RolDE and LMMS with IQR mean pAUCs of 0.995 and 0.993, respectively, clearly outperformed the other methods, with RolDE performing significantly better than the second-best method LMMS (p < 10−9). BETR and EDGE do not tolerate missing values and were therefore excluded from the analysis of the full datasets.Second, we investigated the performance of the methods in the full semi-simulated spike-in datasets with a single trend category per condition but also proteins with missing values included in the analysis. Similarly as with the filtered datasets, which did not involve any missing values, RolDE performed overall best in the full datasets in the presence of missing values Fig. . In the Investigation of the performance of the methods in the different trend categories suggested that the performance of RolDE, LMMS and BaselineROTS in the full datasets with missing values remained on par to the filtered datasets without missing values . Similarly as with the single trend category datasets, the higher order regression models outperformed the lower order models.In the filtered UPS1 Mix datasets without missing values, BaselineROTS with an IQR mean pAUC of 0.953 performed best, followed by RolDE with an IQR mean pAUC of 0.948 , while the rest of the methods performed significantly worse, with IQR mean pAUCs below 0.9 in both the filtered and full datasets.In the full UPS1 Mix datasets including missing values, RolDE with an IQR mean pAUC of 0.945 performed best, followed by Timecourse with an IQR mean pAUC of 0.942 . Both BaselineROTS and RolDE performed significantly better than the next best method LMMS with an IQR mean pAUC of 0.740 (p < 10−5).The original UPS1 and SGSDS datasets contained moderate to low numbers of missing values  and only few missing values in the spike-in proteins . Therefore, to further push the methods, semi-simulated datasets with a larger proportion of missing values were generated using the CPTAC dataset, which had a larger overall proportion of missing values (19.5%) and a considerably larger proportion of missing values in the spike-in proteins (29.4%) than the UPS1 and SGSDS datasets. In the presence of such a large proportion of missing values, all the methods performed clearly worse than in the UPS1 and the SGSDS datasets Fig. . The croFurthermore, many of the evaluated methods struggled to provide valid scores for proteins in the presence of missing values in the data. To evaluate how consistently the different methods were able to provide scores for the proteins in the datasets, IQR mean proportions of valid scores for each method and dataset type were calculated Table . In addip < 0.02 in all scenarios, Supplementary Fig. p < 0.001 in all scenarios) and Limma (p < 10−15). None of the methods were largely affected by the missing values in the true negative proteins.Since it has remained common to have only few time points in real biomedical studies, we also evaluated the performance of the best-performing methods RolDE, Timecourse, Limma, and the baseline ROTS in additional 1200 semi-simulated datasets with only three or four time points. Altogether 300 filtered  and 300 full  UPS1-based semi-simulated datasets of both lengths were explored. Again, RolDE performed overall best, significantly outperforming the second-best method Timecourse (Francisella tularensis subspecies novicida (Fn)  Fig. , includiFn data was 10.8%, being highest in the 37 °C samples with more than 25% of all the values missing in all strains but L . In addition to the overall correlations of the result lists, we examined the median overlaps of the top findings between the replicate datasets, when the size of the overlap was varied from 1 to the total number of proteins in the dataset. In line with the correlation results, the different variants of Limma, LMMS, and RolDE showed a high overlap between the technical replicate result lists, especially among the top findings  and the null mutants of lpxD1 (D1), lpxD2 (D2) and lpxL (L) in the full data with technical replicates averaged for each biological replicate. The result lists from each method were ordered based on the strength of differential expression given by the method.Typically, the top findings in any experiment are the most interesting ones and will most likely be selected for further validation. Therefore, consistency of a method in delivering the same findings at the top of the list is a highly desirable quality. However, reproducibility alone does not guarantee biological relevance. Therefore, we also examined how the different methods detected the proteins related to the modified acyltransferases in the When investigating the acyltransferases themselves, most of the methods ranked the proteins related to the acyltransferases lpxL and lpxD2 to the top of their result lists, with the exception of BETR, EDGE_H, and EDGE_L, which failed to provide a score for lpxD2. More variation was observed in how the methods ranked the protein related to lpxD1, with BETR, Pme_H, Timecourse, Limma, LimmaSplines_H, EDGE_H, EDGE_L and OmicsLonDA failing to provide a score for the protein Fig. .Francisella tularensis subsp. novicida U112 (ftn00540) and the associated Lipopolysaccharide biosynthesis knockout pathway (ko00540) were detected in our Fn dataset . Accordingly, we examined how the relevant proteins in different comparisons in the KEGG pathway Lipopolysacchararide biosynthesis—set Fig. . Also MaFn data, an additional publicly available dataset of human iTreg in vitro induction33 was included in the comparison to comprehensively assess the methods in delivering biologically meaningful results. In this dataset of almost 10,000 proteins, naïve T cells were stimulated and iTreg cells were induced using two different protocols. Proteome profiles were acquired at multiple time points during iTreg induction or mock stimulation at 6 h, 24 h, 48 h, and 6 d. Longitudinal differential expression was determined between the iTreg and the mock-stimulated control cells, separately for both protocols, resulting in two comparisons.In addition to evaluating the biological relevance of the findings of the different methods in the 34 and not known in detail, several Treg related gene sets were used to assess the biological relevance of the findings. Altogether 28 Treg cell-related human gene sets were downloaded from the Molecular Signatures Database (MSigDB version 7.5.1)35 and combined into 14 Treg gene sets , and the interactome of the forkhead box P3 (FOXP3) protein, which is a lineage specification factor of Treg cells, were used to explore Treg-related functional enrichment. In both of the comparisons using the two different iTreg induction protocols, the highest enrichments were most consistently detected with RolDE, BaselineROTS, and BETR , RolDE had the largest number of shared and the lowest number of unique findings in both comparisons , this is not always the case in various real experimental settings. Therefore, we demonstrate the applicability of the overall best-performing RolDE method in a previously published longitudinal type 1 diabetes proteomics data with non-aligned time points10. The dataset contains blood plasma protein expression measurements of 11 children developing T1D and 10 matched controls10. Nine longitudinal samples were collected for each child with non-aligned time points. The aim was to detect early T1D related biomarkers from the blood even before seroconversion and T1D diagnosis, which would allow earlier disease prediction and intervention.While the semi-simulated spike-in datasets and the experimental 36 as well as increased keratin metabolism37 have been previously associated with diabetes and hyperglycemia with keratinocyte proliferation, differentiation, and function38. Downregulation of another detected protein  in T1D cases was also detected  was detected as highly significant  of 0.05 , had FDR of ~0.07 with RolDE , the ability of a differential expression method to deal with versatile missing values might be crucial in detecting relevant proteins to the experimental question. RolDE also displayed the most balanced performance of detecting all possible types of trend differences with good consistency  modularity, (2) application of ranks and rank products, (3) application of longitudinal regression modeling, and (4) reproducibility-optimization with ROTS. The benefit of modularity is that, even if some modules would not be able to provide a valid score for a protein, the other modules with diverse approaches can still provide enough evidence for detecting possible differential expression, whereas proteins with multiple poor ranks will end up towards the end of the result list. The motivation to use ranks and rank products in RolDE is to avoid relying on any distributional assumptions for the data. The rank product approach has been previously shown to be especially effective for noisy datasets and for small numbers of replicatesFn data. However, missing values in the spike-in proteins clearly hindered the performance of the method and Timecourse was able to provide a ranking only for ~72%–79% of proteins in the UPS1 and CPTAC semi-simulated datasets and the experimental Fn data when missing values were present  performed similar to the cubic splines approach (LimmaSplines) when the data did not contain missing values. Some differences in the performance could be observed when missing values were present in the data. In particular, the LimmaSplines_L approach with less degrees of freedom was best among the Limma-based approaches when missing values were present also in the spike-in proteins in the SGSDS and CPTAC semi-simulated datasets  can sometimes provide a meaningful ranking for the strength of longitudinal differential expression in the data. Our results are in line with the previous results of a study comparing longitudinal methods for RNA-sequencing, where most longitudinal methods performed worse than pairwise differential expression approaches applied timepoint-wiseBETR and variants of EDGE did not tolerate missing values and did not match the performance of RolDE, Timecourse and Limma in the semi-simulated datasets and in terms of reproducibility, but fared better on the experimental iTreg data, especially BETR. While performing decently in many of the tested semi-simulated datasets, OmicsLonDa struggled in many scenarios and in the iTreg data most of its top 1000 findings were unique. However, OmicsLonDa was initially developed for identification of time intervals with significant differences between the conditions in longitudinal data and is perhaps not suitable as a general tool to detect any longitudinal differential expression between the conditions. Similarly, by default MaSigPro explores the overall time-associated changes over both conditions, but these effects were not included in this study, as the interest was explicitly in the longitudinal differential expression between the conditions.59.There was considerable variation in the performance of the various regression modeling approaches tested. Overall, the higher order polynomial regression models outperformed the lower degree models. Only in the SGSDS full datasets, the overall performance of Pme_L was better than the performance of Pme_H. This reversal in performance was most likely related to missing values in the spike-in proteins in the SGSDS full datasets and the inability to reliably define the more complex models related to Pme_H for all the spike-in proteins. Furthermore, the performance of the full regression modeling approach was better over a broader spectrum of trend difference categories than the performance of the lower degree models .By default, the use of RolDE requires the user to provide only a design matrix linking the samples in the data to the time points and conditions, and information whether the time points are aligned or not. RolDE then detects any kind of longitudinal differential expression  from the data. RolDE is freely available as an R package in Bioconductor  proteins spiked into a yeast proteome digest with five different concentrations: 2, 4, 10, 25, and 50 fmol/μl31. Three technical replicates of each sample group were analyzed using a Q Exactive Orbitrap mass spectrometer both in the DDA and data-independent acquisition (DIA) modes. In this study, the DDA shotgun proteomics data was used. After preprocessing, all 12 spike-in proteins and 3487 proteins remained in the SGSDS data.In the SGSDS spike-in data, 12 nonhuman proteins were spiked into a stable human background  in eight different sample groups with known concentrations32. Three technical replicates of each sample group were analyzed using an LTQ Orbitrap mass spectrometer (test site 86). After preprocessing, 41 spike-in proteins and 1247 proteins remained in the CPTAC data.The CPTAC data from study six contains 48 UPS1 proteins spiked into a stable yeast proteome digest in five different concentrations: 0.25, 0.74, 2.2, 6.7 and 20 fmol/μl62 version 1.5.3.30. Peptide identifications were performed using the Andromeda search engine. The ‘match between the runs’ option was enabled with the default time window of 0.7 min and alignment time window size of 20 min. ‘Require MS/MS for comparisons’ was on, and decoy mode was ‘revert’. False discovery rate (FDR) of 0.01 was set as a threshold for peptide and protein identifications. A FASTA database of the yeast Saccharomyces cerevisiae protein sequences merged with the Sigma-Aldrich 48 UPS1 protein sequences was used to search protein identifications for the UPS1 and CPTAC data. For the SGSDS data, a FASTA database of the human HEK-293 cell proteins merged with the sequences of the nonhuman spike-in proteins was used. MaxLFQ with ‘advanced ratio estimation’, ‘stabilize large LFQ ratios’ and ‘advanced site intensities’ were on. Non-normalized protein intensities were extracted from MaxQuant and imported into the R statistical programming software environment. The datasets were normalized using the variance stabilization normalization (vsn)63 shown to perform well with proteomics spike-in data12.The default peak-picking settings were applied to process the raw MS files of all the datasets in MaxQuantThe normalized spike-in datasets were used to create semi-simulated datasets with varying pre-defined longitudinal trends in the spike-in proteins. Unlike the spike-in proteins, the expression of the background proteins was expected to remain stable in all sample groups. It should be noted, however, that due to experimental noise and fluctuations, there is always some variation also in the abundance of the background proteins, which reflects the nature of MS data in a real biological experimental setting.i in sample group j for each replicate was drawn from a normal distribution For the creation of the semi-simulated datasets, the means and standard deviations of the proteins in the sample groups of the original normalized spike-in data were used. More precisely, the simulated expression of protein 31. In total, this approach resulted in 21 trend difference combinations between two simulated conditions for the UPS1 and CPTAC datasets and 15 combinations for the SGSDS dataset. When each dataset contained only one type of trend for a condition, a total of 300 semi-simulated datasets with two conditions and different longitudinal trends and/or expression levels were generated for the UPS1 and CPTAC datasets and 210 for the SGSDS dataset , whereas the filtered datasets contained only proteins without any missing values.18, the CPTAC dataset with relatively large numbers of missing values in the spike-in proteins was used to generate semi-simulated datasets with high proportions of missing values to explore the tolerance of the different methods against such missing values. Here, only full datasets were considered.As missing values are a common occurrence in proteomics dataFrancisella tularensis subspecies novicida (Fn). Francisella tularensis subspecies tularensis (Ft) is a highly pathogenic Gram-negative bacterial agent responsible for the disease tularemia in humans. Research interest on Ft has peaked during the past decades due to the possible application of the bacterial agent in biological warfare and bioterrorism64. As reliable methods for the genetic manipulation of the highly virulent subspecies Ft have been lacking, the more responsive and avirulent subspecies Fn was used as a proxy in an effort to shed light into the mechanisms of pathogenicity of Ft64. We considered two temperature regulated N-acyltransferases, designated lpxD1 and lpxD2, which have been identified in Fn. LpxD is involved in the production of a key virulence factor, lipid A, in Fn50. The null mutant of lpxD1 has been shown to be more sensitive to environmental factors and attenuated in virulence when compared to the wild type51. In addition, we studied a late acyltransferase lpxL, which has been discovered in Ft and shown to be essential for cell viability at temperatures above 33 °C65. Thus, the dataset consisted of four strains: the wild type (WT) and null mutants of lpxD1 (D1), lpxD2 (D2), and lpxL (L). In order to activate the temperature sensitive enzymes responsible for the production of the lipopolysaccharide, the global protein expression of the Fn strains was examined at five temperatures: 18, 21, 25, 32, and 37 °C (surrogates for time points). Given that LPS is assembled in the membrane of Gram-negative bacteria, we chose to isolate and generate data only for a membrane-enriched fraction. Each mutant and wild type strain consisted of three biological replicates , and each biological replicate was measured in three technical replicates, resulting in a total of 36 unique samples at each temperature and 180 samples in total. For evaluation of the methods, differential expression between the null mutant and the wild type strains was investigated, while their growth at different temperatures offered a surrogate for the different time points.To evaluate the longitudinal differential expression methods in a real proteomics study setting, we generated an experimental membrane enriched longitudinal proteomics data of Fn strain U112 was originally obtained from Professor Francis Nano 66 and maintained at −80 °C. The lpxD1, lpxD2, and lpxL null mutants were generated previously67. The lpxL strain used was a transposon insertion mutant of FTN_0071 (lpxL) identified as tnfn1_pw060510p04q127 carrying a kanamycin cassette disrupting the coding sequence. The mutant was functionally characterized as having an lpxL knockout lipid A phenotype. All bacterial strains were grown in tryptic soy broth supplemented with 0.1% cysteine at the designated temperatures . Cultures were harvested at log phase, supernatant was aspirated, and cell pellets were flash frozen and stored at −80 °C until processing.Wild-type 68. Briefly, cell pellets were resuspended in 0.1 M NaPO4, 0.05 M MgSOD4, DNaseI digested, and sonicated. Unbroken cells were removed by centrifugation and the supernatant was again subjected to centrifugation at 39,000 × g for 45 min. The resulting pellet, which contained the membrane envelope, was resuspended in 50 mM ammonium bicarbonate and subjected to trypsin digestion. After digestion, samples were desalted using MACROspin C18 columns . The flow-through was collected, concentrated in a speedvac to near dryness, and resuspended in 5% acetonitrile/0.1% formic acid for MS analysis. Samples were stored at −80 °C until MS analysis.To isolate a fraction of proteins enriched in membrane proteins, cell pellets were first fractionated along the lines of a previous study69. Briefly, mass spectrometry data were collected over a 95 min LC gradient using data-dependent acquisition with a full MS scan for 350–2000 m/z range at 120 K resolution. Top 15 ions were selected with an isolation width of 2 m/z for fragmentation by CID, with a dynamic exclusion of 30 s. The default peak-picking settings were applied to process the raw MS data files in MaxQuant62 version 1.6.5.0. Peptide identifications were performed using the Andromeda search engine with a SwissProt/UniProt FASTA database of all reviewed and unreviewed protein sequences for Fn strain U112 (April 2019). Trypsin digestion with a maximum of two missed cleavages, carbamidomethylation of cysteine as a fixed modification, and methionine oxidation and N-terminal acetylation as variable modifications were used as search parameters. Precursor mass tolerance was set to 20 ppm and fragment mass tolerance to 0.5 Da. Minimum peptide length was set to 7. The ‘match between the runs’ option was enabled with the default time window of 0.7 min and the alignment time window size of 20 min. ‘Require MS/MS for comparisons’ was on, and decoy mode was ‘revert’. A false discovery rate (FDR) of 0.01 at the peptide and protein level was applied. The MaxQuant label-free quantification LFQ algorithm was used to calculate the relative protein intensity profiles across the samples. ‘Advanced ratio estimation’, ‘stabilize large LFQ ratios’ and ‘advanced site intensities’ were on. Non-normalized protein intensities were extracted from MaxQuant and imported into the R statistical programming software environment version 3.6.1. Reverse protein hits, known contaminants, and proteins with less than two peptides and at least one unique peptide were filtered out. The dataset was normalized using the variance stabilization normalization (vsn)63 shown to perform well with proteomics data12.The membrane samples were prepared for proteomics analysis by high performance liquid chromatography (HPLC)-tandem mass spectrometry on an LTQ Orbitrap Elite (Thermo-Fisher) according to previously published protocols33. Naive CD4+ T cells from three adult donors were stimulated and iTreg cells were induced using two different protocols with or without the vitamin A metabolite all-trans retinoic acid (ATRA) and rapamycin . Samples were collected at 6 h, 24 h, 48 h and 6 d of differentiation and subjected to proteomics analysis. The samples were analyzed using TMT-10plex-based LC-MS/MS with a Q Exactive Hybrid Quadrupole-Orbitrap mass spectrometer33 and the data was normalized using the TMT ratio sample median normalization. The preprocessed data available from the original study was downloaded and log2-transformed. Longitudinal differential expression was determined between the iTreg and the mock-stimulated control cells , separately for both protocols, resulting in two comparisons.To further evaluate the methods on real experimental biological data, we included a previously published proteomics dataset exploring the molecular landscapes associated with in vitro induction of human iTreg cells10. The data contained longitudinal blood plasma protein expression measurements from 11 children (aged 0.8–14.4 years) developing T1D and 10 matched controls. Nine longitudinal samples were collected for each child, with the T1D case samples covering seroconversion and clinical T1D diagnosis. The samples were analyzed using a TMT-10plex-based LC-MS/MS approach with a Q Exactive HF mass spectrometer with all time points of one individual and a common pooled reference sample included in one TMT-10plex MS run10. Altogether, 189 samples were analyzed. The data was normalized and filtered similarly to the original study: the reporter ion intensities were standardized to the reference sample, log2-transformed and median normalized, and only proteins with at least two individuals from either group with at least seven non-missing values were included in the analysis10.Finally, we evaluated the performance of RolDE on human type 1 diabetes proteomics data from a recent study by Liu et al.The proposed method, RolDE, is a composite method, consisting of three independent modules with different approaches to detect longitudinal differential expression  to preserve comparability between the coefficients of the different individual regression models.The RegROTS module combines the power of polynomial regression modeling and the reproducibility optimization of ROTSj are compared across the individual models by considering all possible pairs of individuals u and v between the two conditions C1 and C2: un, a total of n runs are considered. For instance, with three individuals in both conditions, we consider three runs: run 1 u1-v1, u2-v2, u3-v3; run 2 u1-v2, u2-v3, u3-v1; and run 3 u1-v3, u2-v1, u3-v2. Subsequently, within each run, multigroup ROTS is used to test the null hypothesis d :i and n is the total number of runs. Previous cross-sectional studies have suggested the rank product approach to be especially useful for noisy data53. The empirical distribution of Next, all coefficients of the same degree C1 are compared to all the individuals in condition C2 in multiple runs, but instead of comparing the coefficients of the fitted models, the expression measurements u and v at time point t are used directly: The DiffROTS module examines directly the expression differences between the conditions at each time point. Similar to the RegROTS module, all the individuals in condition T is the total number of time points considered. In each run, the proteins are ranked according to the significance values of the modified F-statistic from the multigroup ROTS. The protein with the smallest significance value gets the smallest rank. The final DiffROTS score for a protein is then calculated by combining these ranks from the different runs using the rank product :i and n is the number of runs. The empirical distribution of Similar to the RegROTS module, multigroup ROTS is then used within each run to detect differences in the expression levels between the conditions, with the null hypothesis that there are no differences in the expression values at the distinct time points between the conditions, i.e., j = 0,…,d are first fit to each protein: d is by default defined as u in each of their measured time point t, and differentially expressed proteins between the two conditions are determined by comparing all the residuals of the individuals in one condition to those in the other condition using two-group ROTS26, separately for the different polynomial degrees j. To detect any differential expression between the conditions, the final DiffROTS score for a protein is determined as the minimum over the significance values pj over the comparisons: If the time points in the data are not aligned between the individuals and the expression levels between the different conditions cannot be directly compared at different time points, the DiffROTS module is adjusted to such a design. This is done by examining the expression level differences between the conditions after accounting for time-associated trends of varying complexity in the data. More specifically, polynomial regression models of degrees t across the conditions c:d is the polynomial degree of the model, d is by default defined as t-test, which is the standard way of assessing the significance of regression coefficients. The corresponding significance values The PolyReg module applies polynomial regression modeling to each protein to detect longitudinal differential expression over time Finally, for a comprehensive inspection of differential expression between the conditions, the results from the different modules are combined using the rank product  of the module scores:To estimate the significance of the RolDE score, a simulation procedure is used. Given the null hypothesis is true, the significance value distribution within a RegROTS or DiffROTS run is (approximately) uniform. Based on this, a simulated internal rank product is calculated for the RegROTS and DiffROTS modules as follows. First, an equal number of simulated significance values as there are experimental significance values in the corresponding run are generated from the uniform distribution. Second, the simulated significance values within each run are ordered so that the ranks of the proteins according to the experimental significance values are retained to account for the dependencies between runs. Finally, each simulated significance value within each run is replaced with the rank of the closest experimental significance value and these are used to calculate the simulated internal rank products. Given the null hypothesis is valid, the experimental and simulated internal rank products are similar  simulated rank products and then calculating the fraction of simulated rank products smaller or equal to each experimental rank product. Given the null hypothesis is valid, the estimated significance value distribution for RolDE will be approximately uniform , taking correlations between time points into account; the magnitude of expression at time points closer to each other are assumed to be more correlated than those further apart13. Version ≥ 1.32.0 of the Bioconductor R package betr was used in this study.BETR utilizes the Bayesian framework and was first introduced for the analysis of times series DNA microarray data16. The first option is through group-mean parametrization, where the design matrix for the linear models is designed is such a way that a separate coefficient exists for each time point and condition combination and various contrasts between the coefficients can be defined to examine differential expression between the conditions. The contrasts were defined as differences in expression between the conditions at each time point and differential expression between the conditions was determined by examining whether all contrasts were zero simultaneously using the moderated F-statistic. The second option included using natural cubic spline curves to model the longitudinal variable over time and fitting separate curves for the conditions for each protein from which differences related to the conditions were explored, following the instructions in the limma R package user manual16. Version ≥ 3.40.2 of the Bioconductor limma R package was used in this study.Limma is a popular toolset used especially in the analysis of gene expression microarray and RNA-seq data. Longitudinal differential expression can be examined in two different ways according to the Limma user’s guide2 statistics through a multivariate empirical Bayes approach, taking replicate variances and correlations in expression levels between time points into account14. The method borrows information across features to better estimate the variance-covariance matrices to reduce the number of false positives and false negatives14. Version ≥ 1.56.0 of the Bioconductor timecourse R package was used in this study.Timecourse ranks features  according to probabilities for differential expression using the Maxwell–Boltzmann or the Hotelling’s T15. It follows a two-step regression approach to detect longitudinal differential expression between the conditions15. In the first step, a general regression model is defined for each feature . In the second step, only the significant models from the first step are then modeled using polynomial regression to find differences between the compared conditions15. Version ≥ 1.56.0 of the Bioconductor maSigPro R package was used in this study. For each protein, a minimum over the condition-related significance values was used as a representative significance value.MaSigPro is a method originally developed for the analysis of time series gene expression microarray data10. We considered two different types of models. In the first variant, a random effect only for the individual baseline (the intercept) was allowed. In the second variant, a random effect was allowed also for the slope. Finally, a likelihood ratio test was conducted to examine if adding a random effect for the slope yielded a significantly better fit at the significance level of 0.05. If this was not the case or the second model could not be defined due to insufficient number of measurements , the model with a random effect only for the intercept was used for the protein. For each protein, a minimum over the condition-related significance values was used as a representative significance value. The R package nlme version ≥ 3.1-142 was used for all the mixed effects modeling approaches in this study.Linear mixed effects regression is a popular approach in modeling longitudinal data, where individual variability can be incorporated into the model in the form of random effectsIn the polynomial mixed effects approach, we added polynomial fixed terms for the time-related effects in the models. Again, two different types of models were considered: models with a random effect only for the individual intercept and models with a random effect also for the slope. Similarly as with Lme, a likelihood ratio test was conducted to examine if adding a random effect for the slope yielded a significantly better fit at the significance level of 0.05. We did not consider random effects for the higher order polynomials as there was typically not enough data in the relatively short time series to define such models. For each protein, a minimum over the condition-related significance values was used as a representative significance value.27 is a well-established method developed for identifying differentially expressed genes in time course studies with DNA microarrays. It uses regression splines to fit two models for each feature: a null model  and the full model . A likelihood ratio test (LRT) or an optimal discovery procedure (ODP) can be used to determine whether the full model fits the data better than the null model, i.e., whether there is longitudinal differential expression. The LRT approach of the Bioconductor R package edge version ≥ 2.16.0 was used in this study.EDGE28 approach utilizes linear mixed effects model regression splines to detect longitudinal differential expression. Fitted models for longitudinal differential expression between the conditions are compared to the null model using likelihood ratio tests. The default cubic regression splines in the R package lmms version ≥ 1.3.3 were used. For each protein, a minimum over the condition-related significance values was used as a representative significance value.The LMMS29 offers a statistical framework for identification of time intervals with significant differences between the conditions. First, the measurements for each feature are adjusted according to the baseline for each individual. Regression splines are then fitted for the features and Monte Carlo permutations are used to generate empirical distributions for the test statistic under the null hypothesis. The Bioconductor R package OmicsLonDA version ≥ 1.6.0 was used. For each protein, all possible time intervals were explored and a maximum value for the test statistic over all the explored intervals was used as a representative value to detect any longitudinal differential expression.OmicsLonDAT-1, and a less complex one (denoted by the extension L) with the degree set to T is the number of time points.For each polynomial regression-based approach , we explored two different levels of complexity: a more complex model (denoted by the extension H in the results) with the degree of the polynomial set to R statistical programming software environment ≥ 3.6.1 was used for the computational analysis performed in this study.72. The pAUCs were visualized using the vioplot R package version ≥ 0.3.2.In the semi-simulated datasets, we evaluated the performance of the longitudinal methods in their ability to correctly detect true differential expression (the spike-in proteins) using receiver operating characteristic (ROC) analysis. In the ROC analysis, the sensitivity  was plotted against the specificity . The area under the ROC curve (AUC) was used to measure how well a given method was able to distinguish the true signal of interest when the detection threshold  was varied. As typically the interest of a differential expression analysis is focused on the top findings, we used partial AUC (pAUC) between specificity values 1 and 0.9 to score the methods on the essential part of the ROC curve. The calculated pAUCs were summarized using the interquartile range (IQR) mean across the datasets of each type. The R package pROC version ≥ 1.15.3 was used to conduct the ROC analysisDifferent methods had varying abilities to calculate a score (a test statistic or a ranking) for the examined proteins due to missing values or other reasons. To ensure comparability, a full result list was expected from each method, including a score for all the proteins in the examined dataset. If a method could not produce a score for a protein, a random score larger than the maximum observed score for the method in the dataset was generated, placing the protein at the end of the result list. Thus, all proteins without valid scores for a given method, were placed randomly at the end of the result list.Fn dataset, we assessed the reproducibility of the results of the methods using the three technical replicate datasets. After filtering out all proteins with missing values, three completely separate technical datasets were formed, which were assumed to be similar to each other. Longitudinal differential expression between all possible pairwise combinations of strains  were considered. To estimate the overall reproducibility of each method, the similarity of their outputs in the replicate datasets was assessed using the Spearman’s rank correlation coefficient. To evaluate the reproducibility of the top differential expression findings, the median proportional overlap between the top k findings was calculated over the replicate datasets, when the examined top list size k was varied from 1 to the number of proteins in the complete dataset. The proportional overlap at each value of k was calculated as the median overlap over the replicate datasets divided by k.In the experimental longitudinal Fn dataset, we examined how the proteins in the KEGG pathway Lipopolysaccharide biosynthesis—Francisella tularensis subsp. novicida U112 (ftn00540) complemented with the proteins in the associated Lipopolysaccharide biosynthesis knockout pathway (ko00540)  are associated with lipid A and LPS, the 18 proteins in the selected ftn00540 and ko00540 pathways are assumed to be affected by the modifications. To explore this, we used Gene Set Enrichment Analysis (GSEA)35 to investigate enrichment of the relevant pathway proteins in each wild type to mutant strain comparison in the normalized data with technical replicates averaged for each biological replicate. For a meaningful and fair comparison, the relevant and included pathway proteins in each comparison were determined as those pathway proteins with enough variation to be detected by the methods by having a coefficient of variation (CV) larger than the median CV for all the proteins 33, the associated Reactome pathway (RUNX1 and FOXP3 control the development of regulatory T lymphocytes) was included as a second additional gene set. Finally, interactome of FOX3P from the STRING protein-protein interaction database was downloaded, including 100 known and predicted interacting proteins for FOXP3. All the used gene sets are given in Supplementary Data Fn dataset.To further evaluate the biological relevance of the results by the different methods, we explored how the methods detected proteins in known Treg-related gene sets in the human iTreg data. Altogether 28 human Treg related gene sets were downloaded from the Molecular Signatures Database (MSigDB) version 7.5.1To evaluate the performance of the methods in terms of unique and shared findings in the iTreg data, the result lists for each method were ranked according to the strength of the differential expression. The overlap of the top 1000 findings was investigated. For each of the top 1000 findings for each method, the number of other methods sharing the same protein in their respective top 1000 findings was determined. For a fair evaluation of methods with multiple variants, only the best-performing variant was included, as the different variants share a considerable proportion of their result lists. The selected variants included the higher order variants for Pme, MaSigPro and EDGE, and the group-mean parametrization approach for Limma.Further information on research design is available in the Supplementary InformationPeer Review FileDescription of Additional Supplementary FilesSupplementary Dataset 1Supplementary Dataset 2Reporting Summary


Using these models, we identified a set of kinases, several of which are understudied, that are strongly influential in the cell viability prediction models. In addition, we also tested to see if a wider range of multiomics data sets could improve the model results and found that proteomic kinase inhibitor profiles were the single most informative data type. Finally, we validated a small subset of the model predictions in several triple-negative and HER2 positive breast cancer cell lines demonstrating that the model performs well with compounds and cell lines that were not included in the training data set. Overall, this result demonstrates that generic knowledge of the kinome is predictive of very specific cell phenotypes, and has the potential to be integrated into targeted therapy development pipelines.Protein kinases play a vital role in a wide range of cellular processes, and compounds that inhibit kinase activity emerging as a primary focus for targeted therapy development, especially in cancer. Consequently, efforts to characterize the behavior of kinases in response to inhibitor treatment, as well as downstream cellular responses, have been performed at increasingly large scales. Previous work with smaller datasets have used baseline profiling of cell lines and limited kinome profiling data to attempt to predict small molecule effects on cell viability, but these efforts did not use multi-dose kinase profiles and achieved low accuracy with very limited external validation. This work focuses on two large-scale primary data types, kinase inhibitor profiles and gene expression, to predict the results of cell viability screening. We describe the process by which we combined these data sets, examined their properties in relation to cell viability and finally developed a set of computational models that achieve a reasonably high prediction accuracy (R Being able to predict how a patient’s tumor will respond to a specific drug treatment is a core goal in the field of precision oncology. An emerging trend in targeted therapies is a focus on protein kinases, a family of over 500 proteins that form an integrated communication network that plays a central role in the development and progression of nearly all cancers. Despite the growing importance of these drugs in the oncologist’s therapeutic toolbox, our ability to predict the response of a tumor to a given treatment is poor. To see if we could improve our ability to predict a cancer’s response to kinase inhibitor treatment, we leveraged a large experimental dataset that quantifies the effect of these drugs on the kinases. Using these kinase inhibition state data within machine learning models, we found that we could predict the response of cancer cell lines representing over 27 cancer types with high accuracy. Including cell line-specific gene expression data that could be gathered in a clinical setting further improved the accuracy of predictions. Together, these results suggest that knowledge of the inhibition state of the kinome has significant potential to improve our ability to design and deliver more effective targeted cancer treatments. While chemotherapy remains a mainstay in cancer treatment, the use of targeted therapies clearly holds significant promise, with their use leading to improved outcomes in a variety of cancers ,2. ExampSince the development of imatinib, protein kinases have emerged as a primary focus for targeted therapy development –14. KinaRecent work characterizing kinome behavior in response to targeted kinase inhibitor therapies has established that the kinome is a highly dynamic system, with significant ramifications in our understanding of drug resistance, adaptive reprogramming and the broader design of effective therapies –23. UndeGiven the potential of targeted therapies and the potential to quantitatively assess their effect on the protein kinome, in this work we sought to establish a predictive framework that links the behavior of the kinome as defined by “kinase inhibition states” with a downstream phenotype—in this instance, cell viability. Enabling this effort is recent work by Klaeger et al., who conducted a comprehensive investigation using a proteomic kinobead approach, establishing a target landscape for 229 kinase inhibitors across a wide range of compound concentrations . This wo2 = 0.79). Integrating nearly half a million data points, we find that kinome inhibition profiles have by far the greatest predictive power of any single data set. While not highly predictive on its own, baseline transcriptomic data does significantly enhance prediction accuracy, “tuning” the model to individual cell lines. Remarkably, adding in other multi-omics data does not significantly increase the quality of predictions. As the model enables prediction of complete dose-response curves, we experimentally validate predictions for over two dozen compounds on two breast cancer cell lines and find strong agreement for most compounds tested. These results suggest that the link between kinotype and phenotype is significant, with sufficient power to enable the prediction of cell viability and likely other cellular phenotypes as well. Along with integration of transcriptional data, these predictive models can greatly enhance our understanding of adaptive kinome reprogramming and drug resistance while facilitating the development of future targeted therapy regimes.Here, we describe a framework that integrates kinome profiling data with general multi-omics, and build tree-based regression models to predict cell viability for 480 cancer cell lines across 230 kinase inhibitors with high accuracy Fig 1A). The primary challenge with combining these data sets is a lack of overlap between some of the concentrations used in the PRISM assay and those used by Klaeger et al. To overcome this problem, we used the viability curve fits provided by the PRISM database and imputed cell viability values for all of the concentrations used by Klaeger et al . These cAfter combining the PRISM and Klaeger et al. data sets, we have 168 compounds which have been assayed across 480 cell lines. We imputed the cell viabilities at each of the 8 concentrations used in the Klaeger et al. work, yielding about half a million treatment combinations across combinations of cell line, compound and concentration. With this data set, we also integrated the gene expression data available through the Cancer Cell Line Encyclopedia . These gWe investigated the relationship between kinase inhibition states (~520 proteins) and gene expression values with inhibitor-induced cell viability. To do this, we took each individual kinase inhibition state and gene expression value  and calculated the Pearson’s correlation coefficient with the imputed cell viabilities . The kinWhile single features with correlation coefficient values in the ~0.3 range  will not produce sufficiently predictive models, the integration of multiple features may provide greater power. As such, we next sought to use the correlation values for feature selection. The most obvious way to use the correlation values is to put all the potential features  in correlation rank order and then select the top-X number of features for model inclusion. This produces differing sets of feature class counts and ratios depending on the number of features selected  right. IWith our initial analysis of the predictive power of single features from the Klaeger and gene expression data sets completed, we next moved to the development of models that integrated more than one feature with the end goal of predicting cell viability. To do this, we tested four types of models: linear regression, random forest, TabNet and XGBoost. For our initial tests with these models, we used the default settings for all four model types and varied the number of features  provided to the model. Our cross validation strategy sought to balance our eventual goals of using the resulting models to make predictions about the cell viability effects in new cell lines and in untested compounds. As such, we choose a 10-fold cross validation strategy that randomized fold exclusion across the cell line-compound treatments  to improve the likelihood that our model testing results would be similar to downstream experiments. After producing the cross validation splits, we selected a specific number of features and built corresponding models .2 and RMSE) peaked at 500 features and out-performed our baseline dose-concentration-only model. To ensure that we had indeed found the peak in feature performance, we then tested 600, 700, 800 and 900 feature models and found that the 500 feature model was the peak . To better understand this model, we also looked more closely at the predicted versus actual imputed viability of the 500 feature random forest model  or understudied (Dark) kinase  35]. Se. Se35]. 2 of ~0.01 and RMSE of 0.33), which was expected due to the fact that the gene expression values are fixed and do not vary with the compound concentrations. These model performance differences were also reflected in direct comparisons between individual cell lines and compounds, where none of the expression-only models outperformed the inhibition state only models. When we built models using the inhibition states alone, we observed identical performance for feature counts 300 and below. This was also expected as the correlational feature selection methods always select inhibition features for the first ~350 features. With feature counts of 400 and 500, we observed that the additional information provided by the gene expression features began to improve the model (0.05 improvement in R2 and a 0.02 decrease in RMSE). Thus, while the expression features alone are not sufficient to predict cell viability, they do provide an appreciable improvement in the model performance in combination with inhibition features.After thoroughly examining the results of the inhibition state and gene expression combined model, we next wanted to investigate how the model would perform when we excluded certain parts of the full data set. Using the same 10-fold feature selection cross validation strategy and the same cross validation fold splits described above, we rebuilt the model using only inhibition state or only gene expression . The gen2 of 0.76 and a RMSE of 0.17 (compared to R2 of 0.79 and RMSE of 0.15 for the full set). These results indicate that the non-kinases are providing some additional information that the model is able to use, which is in agreement with the presence of non-kinases in the top 25 of the variable importance metrics Copy-number-variation: gene level copy number variation (CNV)Whole Genome Proteomics: mass spectroscopy-based measurement of relative protein abundance (proteomics)Given the broad and complementary nature of these data sets, we investigated whether we could integrate these data sets to improve upon the kinase inhibition and gene expression models we described above. The DepMap, CNV and proteomics data sets all overlap with a different number of cell lines present in the PRISM data set . All of 2 values of 0.794 (0.153 RMSE) and 0.793 (0.154 RMSE) for the all data and inhibition/expression models respectively. This indicates that gene expression values alone contain substantially similar information as the remaining set of multiomics data. Given our desire to build a model which uses the most easily reproducible data sets and only minor improvements were observed with the full data collection, we decided to move forward with the integrated model combining kinase inhibition states and gene expression values.Based on our previous experience with building the kinase inhibition and expression models, we decided to only test the best-performing random forest method. We also used the same 10-fold cross validation across the cell line/compound combinations. This resulted in higher instability in feature inclusion across the cross validation folds . As showWith the model production decisions finalized, we then applied this model to the untested cell line and compound combinations. The final model was produced using the 63189 cell line and compound combinations with interpolated viability values . Of the 2 to 0.492 and the RMSE to 0.299  cell line HCC1806 and a set of compounds that displayed a range of viability effects from the 134 Kleager kinase inhibitors that had been used in the PRISM assay with the HCC1806 cell line . Severalto 0.299 . To put of 0.296 . These r2 (0.518) and RMSE (0.239) values were comparable with replication effort, indicating that the model was performing well on new compounds (2 of 0.588 and RMSE of 0.246) were comparable to the other model validation results  and concluded that these additional data sets were not able to significantly improve model performance. Finally, we tested some of the model predictions in several triple negative and HER2 positive breast cancer lines and found acceptable agreement between the model predictions and experimental results.This work demonstrates how knowledge of the inhibition state of the kinome, derived from a proteomic assay based on a four cell lysate mixture, can predict a cellular process as fundamental as viability. Importantly, the models achieved these surprising results by using a "generic" or "general" kinase inhibition profile measured with proteomic kinobead profiling of a four cell line lysate exposed to an extensive library of kinase inhibitors at multiple doses . Thus, tWe acknowledge several limitations of this work. First, all of the results in this paper rest on the availability of kinome profiling data specific to a given kinase inhibitor, so the methods here are not applicable to prediction of cell viability effects in any other class of compound. We believe that a similar strategy could be used to build models in compound classes where the spectrum of targets were as comprehensively identified. The universe of small molecule kinase inhibitors is substantially larger than those that were surveyed by Klaeger et al., but since our modeling methodology depends on the comprehensive nature of their work, we’re limited in the number of compounds where we can make predictions. One of our next goals is to attempt to broaden the scope of compounds through integration of other high-content kinome profiling techniques such as KinomeScan and Nano-BRET. In addition, while the models described in this paper do make somewhat accurate predictions, these results point to a degree of missing predictability in cell viability for which new methods and data will need to be developed and collected. Also, since this work has targeted building a single comprehensive model, it is likely that subtle cancer type specific relationships are not captured such as the relationship between RXRG expression and melanoma . This caThis work also suggests several extensions that would broaden or improve the model. Given recent interest in finding new compound combinations computationally, we are beginning to examine how best to combine the information from multiple compound kinome inhibition states to predict the resulting cell viability effects. This would allow us to run computational drug combination screens. In addition, the methods outlined here will also likely work for any phenotype that can be measured after treatment with small molecule inhibitors and with sufficient throughput to gather a large enough data set. Finally, while we have made all of the code and data necessary to reuse our models available to the public on github, we also acknowledge that this is not the most user-friendly method for allowing non-computationally minded users to access the model. Thus, we also plan on developing a web-based system for allowing non-computationally minded users to submit a gene expression profile and receive a set of predictions concerning how their cellular system is expected to respond to the Klaeger set of kinase inhibitors.Overall, we hope that this paper makes a contribution to our understanding of how the overall state of kinome in response to small molecule inhibitors contributes to cell viability phenotypes. Our findings demonstrate that while individual kinase inhibition states and other single gene or protein readings are not very predictive of cell viability, machine learning approaches are able to combine sets of measurements related to the small molecule kinase inhibitors and gene expression data to make cell viability predictions. The results presented here show how a thorough understanding of kinase activity levels in conjunction with baseline omics data can be used to gain a better understanding of phenotypes such as cell viability.Our methods can be divided into two parts describing the computational aspects of this work and the experimental methods used to test the output of the computational components.https://gygi.hms.harvard.edu/publications/ccle.html), specifically Table S2  A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).Important additional instructions are given below your reviewer comments.Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.Sincerely,Inna LavrikAssociate EditorPLOS Computational BiologyFeilim Mac GabhannEditor-in-ChiefPLOS Computational Biology***********************Reviewer's Responses to QuestionsComments to the Authors:Please note here if the review is uploaded as an attachment.Reviewer #1: The recent development of cancer treatment emphasizes targeted therapies. Protein kinases represent one of the largest classes of anti-cancer targeted therapies. Therefore, uncovering the correlation between kinome activity and cancer cell viability via regression model could provide a list of potent drugs based on predicted cell viability value ahead of treatment and infer potential key kinases in regulating cell viability. Berginsiki et al set out to determine whether a combination of kinase inhibition profiles at multiple doses and gene expression could predict the effect of kinase inhibitors on cancer cell viability. They integrated kinase inhibition profile data and gene expression data of different cancer cell lines to build a random forest model to predict cell viability. The final model achieved R2 value as 0.79 in 10-fold cross-validation. This manuscript compared linear and non-linear models, utilized feature ranking via Pearson’s correlation, and introduced data from various aspects like copy number variation and proteomics as additional input for model training. Besides developing the model, this manuscript showed some insights from the model itself, pointing out that most top-ranking features are from kinase inhibition data and that genes selected as model features have a high possibility of interacting with kinase. This manuscipt also considered the experimental variation and evaluated the difference between the original PRISM value and lab assay result before validating the model results on new compound–cell line combinations.Major comments1. The conceptual bias of training data. The kinase inhibition profiles comes from Klaeger et al, whose data is derived from a mixture of four cell lines, while cell viability and gene expression data (PRISM) are from a panel of over 400 cancer cell lines. The combination of kinase inhibition state from a mixture of cell lines and cell line-specific gene expression data might not be a proper input to train a model predicting individual cell line viability. Have the authors tried to use kinase inhibition profiles from gold standard radioactive assays which could avoid in vivo bias due to cell line variation? For example, GSK kinase inhibitor profiling or HMS LINCS profiling done at multiple doses.2. Lack of innovation from a modeling perspective. To build the model, Berginsiki et al applied linear regression, random forest, and XGBoost with a limited parameter tuning process. Introducing a neural network with different transformers might improve the performance for large-scale multi-dimensional data. To optimize the random forest model, there should be parameters other than tree number to adjust, like subsample size, max leave, depth and child weight, etc. Show optimization of RF. Is it possible to make some more improvements to this model if including these features in parameter tuning, such as using grid search or randomized search for 1000 iterations or so?3. I applaud the author's effort to examine the difference between original PRISM data and lab assay results, and there exists a similar trend between model prediction curves and lab assay results, however, R2 score of 0.5 is merely reasonable. Moreover, this comparison is based on the assay results and the model prediction values, which could not infer how the model might perform on the original PRISM data, given the observed difference between PRISM value and assay result.Reviewer #2: Summary:In this study, the authors develop a set of predictive models for cell viability across varying concentrations of kinome inhibitors, using kinome "inhibition states" as predictive features. The work builds on a 2017 paper by Klaeger et al. which profiled 243 kinase inhibitors to identify potential off-target/polypharmacology effects. This study integrates the kinome profiles from Klaeger et al. with -omics data from CCLE and PRISM, showing that kinome profiles tend to be more informative in viability prediction than baseline gene expression values, and additionally that kinome profiles and gene expression can be combined to produce slightly better models than either data type alone. The authors validate their modeling strategy on held-out kinase inhibitors and cell lines, showing that for a subset of breast cancer cell lines their models tend to generalize well.The paper is clear and well-written, and the experiments provide compelling evidence that kinome inhibition profiles contain predictive signal for cell viability screening of kinase inhibitors. The validation on breast cancer cell lines and the predictions across all of PRISM will serve as a useful resource, and the GitHub repository containing the code is well-documented and easy to navigate. However, there are some aspects that could be explored further to ensure the robustness of the study's main conclusions, particularly with relevance to existing work in viability prediction.Major comments:https://doi.org/10.1101/2020.02.21.959627) which found that baseline expression was generally effective for predicting cell viability for various types of perturbations, moreso than genomic profiles. Importantly, the studies are looking at different datasets and drug classes: the Dempster et al. study looked at a broader set of compounds rather than just kinase inhibitors, and at GDSC and several other datasets in addition to PRISM. However, the application to cell viability and overall conclusions of both studies still seem quite related.1. The conclusions of the authors in this study seem slightly at odds with previous work, particularly Dempster et al. 2020 , or whether a difference in experimental design is causing the discrepancy . Glancing through the methods of both papers I noticed a few differences - perhaps most notably the Dempster et al. study uses only the PRISM dose with the most variance over cell lines for each compound, while the present study looks at multiple doses as described in the Methods section and in Klaeger et al.It would be informative if the authors tried running their gene expression pipeline on their set of kinase inhibitors with the labeling strategy (selecting a single representative dose) described in Dempster et al. - if the results are unchanged, it would inspire more confidence that the set of kinase inhibitors studied in this paper are indeed biologically different from the rest of the compounds tested in the Dempster et al. study, rather than reflecting an experimental design difference. If the results are different, this would also be a noteworthy outcome - it would support the idea that looking at multiple doses, rather than picking a single representative dose, is a necessary/important difference between the studies.2. Related to the first point, the example of BRAF inhibitors in Figure 6 of the Dempster et al. paper (dabrafenib and vemurafenib) could be a useful positive control, or example to use to examine the differences between the studies. Their data suggest that baseline RXRG expression, particularly in melanoma cell lines, is a strong univariate predictor of response to dabrafenib and vemurafenib - is that still the case for the authors' pipeline , or are the results different when the labels for multiple doses are introduced? Does the authors' multivariate gene expression-based model perform well for BRAF inhibitors?3. Many of the figures in the paper  show results/performance metrics summarized across all cell lines and compounds in the dataset. This makes sense for a summary figure, but it seems to me that this could potentially be obscuring variation between cell lines or between compounds. For instance, despite the fact that gene expression does not perform well for viability prediction in general , are there any compounds or any cell lines/tissues of origin where gene expression does outperform kinase inhibition data? If so, is there any biological interpretation for the variation? If this is not the case  this would also be interesting to note.4. Overall the paper is very clear and well-written. However, as a reader, I would appreciate a bit more clarification earlier on in the paper about exactly what a "kinome inhibition profile" is in the introduction/summary figure since this is central to understanding the paper, especially given that this is a general computational biology journal with readers from diverse fields. Reading through the Klaeger et al. paper, this quote seems like a good summary of why kinome inhibition profiles are useful:"Owing to the fact many compounds target the structurally and functionally conserved adenosine 5′-triphosphate (ATP)–binding site, polypharmacology  is commonly observed. Target promiscuity may have advantageous or detrimental therapeutic consequences."The idea seems to be that kinase inhibitors can be either targeted or more general, but almost all of them have some "off-target" or polypharmacology effects. Knowing the empirical profile of kinase inhibition for a drug, then, can give you more information than just knowing what it's designed to target. It would be good to put some form of this summary in one of the introductory paragraphs, and/or to explicitly mention exactly what the model features represent  since many readers might not be immediately familiar with the data format, or with the observation that kinase inhibitors do not necessarily bind specifically to their intended targets.Minor comments:5. The authors should be more specific about what form of linear regression is being used in the Methods section. Does the tidymodels default use any form of regularization , or is it an ordinary least squares fit? It may also be good to describe the default hyperparameter selection strategy for the random forest and XGBoost models - does tidymodels test a variety of parameters using some kind of internal cross-validation, or does it choose a single set of default parameters? Is there any particular reason the authors chose to tune the number of trees in the random forest model and not any other hyperparameters? Hyperparameter selection details can have a considerable effect on performance, at least in my experience, and it is useful to be clear about what was explored/settled on for the final models and why.6. Methods section, page 18 under "STRING": Ensemble -> Ensembl**********Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #1: YesReviewer #2: Yes**********what does this mean?). If published, this will include your full peer review and any attached files.PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool, Data Requirements:http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: Reproducibility:https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocolsTo enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at  22 Aug 2022AttachmentResponseToReviewers.pdfSubmitted filename: Click here for additional data file. 20 Jan 2023Dear Professor Gomez,We are pleased to inform you that your manuscript 'Kinome Inhibition States and Multiomics Data Enable Prediction of Cell Viability in Diverse Cancer Types' has been provisionally accepted for publication in PLOS Computational Biology.Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology. Best regards,Inna LavrikAcademic EditorPLOS Computational BiologyFeilim Mac GabhannEditor-in-ChiefPLOS Computational Biology***********************************************************Reviewer's Responses to QuestionsComments to the Authors:Please note here if the review is uploaded as an attachment.Reviewer #2: The comments from my previous review have been addressed - thank you for the great work. In particular, the experiments showing that kinase inhibitor profiles still provide the most predictive signal even when only a single representative (highest variability) dose is used emphasize the robustness of the results, and the added analyses exploring per-compound and per-cell line performance variation is another interesting aspect of the revised paper.I have a few small comments/suggestions on the revised version of the manuscript:1. One aspect of the response to reviewers that stood out to me was this: "We think of the kinase inhibitor profiles as providing a cell-line agnostic initial “state” that is expected to be induced by inhibitor treatment, with the RNA expression altering that state in a cell-line-specific manner (and thus the improvement in predictions with its inclusion)." I thought this was an interesting way to think about the findings in the paper - i.e. the kinase inhibition data provides information about each compound or perturbation, and the gene expression/omics data provides information about the cell line. Since the kinase inhibition profiles were so valuable in terms of predictive signal, I wonder if there are there any other data sources that could provide information about each compound in a cell-line-general manner similarly to, or complementarily to, the kinase inhibition profiles: maybe information on chemical structure, where applicable (e.g. Morgan fingerprints or some other kind of molecular embedding) could provide a performance improvement as well?If so (or if there are other examples I'm not aware of) this could be a useful addition to the discussion section. To be clear, the findings in the paper are valuable in their own right and adding more compound-specific information is probably beyond the scope of this study, but mentioning it as a future direction or an existing area of research could help to put the study in a broader context.2. In the first sentence of the revised abstract, should "compounds that inhibit kinase activity emerging as a primary focus" be "compounds that inhibit kinase activity *are* emerging as a primary focus"?**********Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #2: Yes**********what does this mean?). If published, this will include your full peer review and any attached files.PLOS authors have the option to publish the peer review history of their article  1223-442824 | ploscompbiol.org | @PLOSCompBiolPLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom