Route determinationof sulfur mustard was accomplished throughcomprehensive nontargeted screening of chemical attribution signatures.Sulfur mustard samples prepared via 11 different synthetic routeswere analyzed using gas chromatography/high-resolution mass spectrometry.A large number of compounds were detected, and multivariate data analysisof the mass spectrometric results enabled the discovery of route-specificsignature profiles. The performance of two supervised machine learningalgorithms for retrospective synthetic route attribution, orthogonalpartial least squares discriminant analysis (OPLS-DA) and random forest(RF), were compared using external test sets. Complete classificationaccuracy was achieved for test set samples (2/2 and 9/9) by usingclassification models to resolve the one-step routes starting fromethylene and the thiodiglycol chlorination methods used in the two-steproutes. Retrospective determination of initial thiodiglycol synthesismethods in sulfur mustard samples, following chlorination, was moredifficult. Nevertheless, the large number of markers detected usingthe nontargeted methodology enabled correct assignment of 5/9 testset samples using OPLS-DA and 8/9 using RF. RF was also used to constructan 11-class model with a total classification accuracy of 10/11. Thedeveloped methods were further evaluated by classifying sulfur mustardspiked into soil and textile matrix samples. Due to matrix effectsand the low spiking level (0.05% w/w), route determination was morechallenging in these cases. Nevertheless, acceptable classificationperformance was achieved during external test set validation: chlorinationmethods were correctly classified for 12/18 and 11/15 in spiked soiland textile samples, respectively. Analternative approach is to examine intrinsic properties of the CWAs,such as elemental isotope ratios in the compound (s) under investigation.18Chemicalforensics is the scienceof attributing chemical samples to sources by analyzing their contentof specific compounds or establishing links between samples basedon similarities in their chemical profiles.19 and its recentuse in the Arabic Republic of Syria has been reported.20 HD can be synthesized via several differentroutes, either directly from ethylene or in a two-step process viathe intermediate thiodiglycol (TDG). We have previously used a targetedmethod based on GC–MS analysis to identify CAS for some ofthese routes, enabling partial resolution of 11 production routes.21 By this method, all detected chemicals, notpresent in the blank samples, were manually included in a target librarybased on their mass spectra and retention indices. Both compoundsidentified by spectra library search and unidentified compounds wereincluded. This method can source unknown samples correctly to single-stepsynthesis routes and identify the chlorination method used in two-steproutes involving TDG. Unfortunately, GC–MS analyses of HD samplesdid not provide sufficiently detailed CAS profiles to enable discriminationbetween the three synthetic routes to TDG included in the study. Thisindicates that an analytical tool with higher sensitivity and resolutionis needed to detect markers specific to particular routes of TDG synthesis.The extraction of significant CAS from large high-resolution datasetsalso requires an alternative strategy, possibly based on nontargetedscreening.Sulfur mustard (HD) has historically been the most widelyusedCWA in armed conflicts,22 The analytical data is processedwithout prior assumptions about possible target compounds. Nontargetedscreening has successfully been used in environmental chemistry todiscover potentially toxic chemicals and investigate changes in levelsof pollutants over time. For example, it was used to screen groundwatersites23 and sewage sludge22 for pollutants or to study environmental contaminants.24 Identification of analytes is often preferredin nontargeted screening; the level of identification can range fromsimply determining an exact mass to structural confirmation usingreference standards.25The aim of nontargeted screening is to include asmany compoundsas possible by combining a broad sample extraction method with a generalanalytical method that can detect chemicals with diverse chemicalproperties.26 CWA and/or their degradation products can beextracted from contaminated samples of water, soil, or solid materialsfrom the site of a chemical attack.27 Therehave been a few prior studies on route attribution of CWAs in relevantmatrices, focusing on compounds related to production methods forRussian VX in food samples,29 acephate and CWA-relatedcompounds extracted from dust,30 nerveagents extracted from indoor furnitures,31 and the sarin surrogate dimethyl methylphosphonate extracted frompainted wallboards32 and office media.5Efforts toverify alleged uses of chemical weapons can be expectedto require chemical analysis of CWA in environmental samples.33 and it has alsobeen suggested as a tool for assessment of propositions using multivariatechem-bio forensic data.34Chemometric data analysis and machine learningalgorithms are powerfultools for the extraction of information from the high-dimensionaland complex data normally generated by chemical profiling methods.Based on learning datasets, statistical models can be built in orderto classify unknown samples. To be useful in court, the statisticalmodels have to be validated and the classification results translatedinto a quantitative measure of evidence for or against a proposition.The use of likelihood ratio methods for evidence evaluation is establishedin forensic statistics,The aimof the study presented here was to develop a sensitiveanalytical methodology based on high-resolution mass spectrometryto allow extraction of CAS suitable for discriminating between HDsynthesis routes. A general nontargeted data processing method wasused to enable efficient processing of large data matrices and therebyimprove CAS detection.2) were also used in the experiments.Ethylacetate (99.8% purity), dichloromethane(99.8% purity), and dibenzothiophene (98% purity) were purchased fromMerck, Darmstadt, Germany. Soil  from Sigma–Aldrich, MO, U.S., and a textile  wereprepared by mixing 250 μL of each replicate sample for the routein question. The use of pooled crude HD samples made it possible toadd exactly the same solution to all six soil and textile matrix samples.Spiked matrix samples were prepared by adding 50 μL of the appropriatepooled crude HD sample (1 mg/mL) to 0.1 ± 0.01 g of soil or textile,theoretically resulting in the addition of HD to a level of 0.05%w/w. Spiked matrix training samples were prepared in triplicate ontwo occasions, giving six replicates for each material. The matrixtest set was prepared by adding 50 μL of the crude HD test setbatches  to 0.1 g ± 0.01 g of soil andtextile samples in duplicates. All spiked matrix samples were storedin closed glass vials at room temperature overnight. Spiked soil sampleswere extracted by adding 1 mL of ethyl acetate and shaking  at 200 rpm for 30 min, followed by centrifugation at 2500 rpm for 3 min.Ethyl acetate was then decanted, and the extraction process was repeatedonce, after which the two ethyl acetate extracts were combined. Spikedtextile samples were extracted by ultrasonication  in 3 mL of ethyl acetate for 10 min. Theethyl acetate phase was then removed, and the material was extractedonce more, after which the two extracts were combined. The combinedextracts were filtered  and concentrated to a volume of 50 μL under a streamof nitrogen at 40 °C. At least one blank sample of each matrixwas prepared on each sample preparation occasion. The blank matrixsamples were spiked with 50 μL DCM and thereafter treated identicallyto the spiked matrix samples.26 Crude HD and spiked matrix samples were analyzed in random order,and every sixth analysis run was done using a solvent blank sample.When analyzing spiked matrix samples, a solvent blank sample and asample preparation blank were analyzed after every sixth nonblanksample.Samples were analyzed with a Trace1310 gas chromatograph coupled to an Exactive GC Orbitrap mass spectrometer. A DB-5MS column  was used to separate the compounds. The sample(1 μL) was injected splitless at 200 °C with helium asthe carrier gas at a constant flow of 1.2 mL/min. The GC program startedat 40 °C for 1 min, followed by a 10 °C/min increase to300 °C and a hold at 300 °C for 5 min, giving a runtimeof 32 min. Mass spectrometric scans were performed in the range of30–750 m/z with a resolution of 30,000. The temperatures ofthe transfer line and ion source were set at 250 and 230 °C,respectively. Daily tuning and calibration ensured the quality ofthe mass spectrometry, and the instrument’s performance wasmonitored by daily analyses of QC samples.The chromatographic datawas processed by peak detection, retention time alignment, and peakintegration followed by isotope ratio filtering. This resulted ina processed dataset with peak areas from extracted ions at specificretention times corresponding to different compounds. Data processingwas done in Tracefinder using the analysis mode for unknown screening, which enables nontargetedscreening of data. Peak picking was done with the deconvolution plugin. The retention time alignmentwindow was set to 10 s, the accurate mass tolerance to 10 ppm, thesignal-to-noise (s/n) threshold to 5, the total ion-chromatogram intensitythreshold to 500,000, the ion overlap window to 90–99%, andthe response threshold to 10,000.m/z deviation threshold was increased to0.01 to permit merging.The extracted peaks were automaticallytime-aligned and integrated in the unknown screening view using theAvalon detection algorithm and the nearest RT detection method withseven smoothing points. Data representing compounds present in blanksamples were manually removed from each crude HD dataset. The datasetsfor spiked soil and textile samples were processed and manually mergedafter removing peaks found in blank samples. The variation in thespiked matrix data was higher than in the crude HD data, so the 35Data analysiswas done both withand without isotope ratio filtration using sulfur and/or chlorineisotope filters, 1.9958 ± 10 ppm and 1.99705 ± 10 ppm, respectively.The internal standard was used as a control compound to ensure thatpeak detection was performed correctly and the sulfur isotope ratiofilter settings were appropriate. Filtration was done to extract allpeaks corresponding to analytes containing chlorine, sulfur, or both.It thus removed all other peaks, including matrix-associated peaksirrelevant to HD route classification. Mass defect filters have previouslybeen used to select for metabolites based on their specific isotoperatios.36 and random forest (RF),37 which were performed in Simca  and the R software (R38 version 4.0.3 with a random forest 4.6–1439 package), respectively.Data from the crude HD samples wereinitially analyzed by principal component analysis (PCA) to get anoverview of the variation in the data and detect potential outliers.Subsequent analyses were performed in parallel using two machine learningalgorithms: orthogonal partial least squares discriminant analysis(OPLS-DA)28 The hierarchic decision tree method (2X), the variation in the response matrix explained bythe model (R2Y), and the variation in the response matrixpredicted by the model (Q2). CAS important for class separationwere identified by their predictive variable importance in projection(VIPpredictive) values.40Priorto analysis, the data was normalizedby total area normalization, log-transformed, and scaled to unit variance.The data was too complex to allow classification of 11 routes witha single OPLS-DA model; full classification required a hierarchicdecision tree guided by multiple OPLS-DA models. A similar approachwas previously used to classify production routes of the nerve agentRussian VX, based on data for spiked food samples.e method 2 first u41 The Gini impurity metric measuresthe purity of the classification tree nodes; variables with lowerGini values are more important in RF models.The data wasnormalized by total area normalization.RF can handle complex datasets, so a classification model with 11classes was constructed. The number of trees was set to 10,000 inall RF models, and the number of randomly sampled variables was optimizedby comparing the out of bag (OOB)-estimated error rates for preliminaryRF models with varying numbers of randomly sampled variables. TheOOB-estimated error rate is the mean of the errors for each trainingset sample, calculated from decision trees generated while excludingthe sample in question from the bootstrap sample. The number of randomlysampled variables in the final classification models is ranged from3 to 75. To allow direct performance comparison with OPLS-DA, RF modelswere also used together with the hierarchical decision tree. In RF,CAS important for class separation were identified by consideringtheir Gini impurity.All RF and OPLS-DA models werevalidated using corresponding HD test sets.Attribution of chemical samples to the production method is basedon CAS profiling, i.e., identification and analysis of trace componentsthat are diagnostic for specific production conditions or startingmaterials used. We evaluated a nontargeted GC–HRMS method forthe analysis of crude HD samples and the scope for using the datagenerated to build classification models  and was thus applied throughout the study.The models’ performance was first testedby classifyingtest set samples originating from different synthetic batches of crudeHD 3, right.crude and M2crude, 21 A significant advantage ofnontargeted data processing in this context is that there is no needto create a target library of the detected compounds for each route.Theisotope ratio filtered data contained 714 potential CAS whosenormalized peak areas were modeled in parallel using two machine learningalgorithms, OPLS-DA and RF. The nontargeted method gave classificationmodels of high quality .Chemical identificationof CAS is important for understanding their formation during HD productionand for verifying that relevant CAS were extracted during the analysis.CAS can be identified in several ways, for example, by searching forspecific compounds expected to be present based on prior knowledge,by visual comparison of detected compounds in samples, or by lookingat variables found to be important in the classification models. Thefirst two targeted approaches have limitations that restrict the amountof CAS information that can be extracted from the sample data. Inthis nontargeted work, we identified key CAS that were important variablesin the route classification models. While some CAS were clearly associatedwith the chemistry of HD production, others had no obvious origin.However, evaluations of the effects of different variables on classificationmodel performance clearly showed that successful attribution dependedon complete CAS profiles rather than individual compounds . Chlorination with SOCl2  produced HD containing chlorinated derivativesof amylene, a stabilizer in the dichloromethane used as the reactionsolvent. Chlorination with HCl  yielded few CAS of lowintensity. CAS associated with routes 10 and 11 included polysulfidesand vinyl chlorides. Dithiane  is a CAS related to TDG synthesis.Most of the compounds relevant to TDG synthesis method attributionusing models M3acrude, M3bcrude, and M3ccrude could not be identified due to their low abundance (Table S3).The CASfound to be related to the chlorination method in two-step routeswere largely identical to those previously found by GC–MS.Figure S2, the most abundantpeaks in the chromatograms, aside from the HD peak, originated fromthe matrix. The use of isotope ratio filters was essential when processingdata for these samples because it extracted information on compoundscontaining chlorine and/or sulfur from the complex HRMS data. Despitethe low spiking level, CAS profiles related to specific routes weredetected in spiked matrix samples. However, there were large differencesbetween the CAS profiles of crude HD and spiked matrix samples  of spiked matrixmodels 1 and 2 (M1matrix and M2matrix) was comparableto the corresponding crude HD models , with a few relating to SOCl2 chlorination , as shown in Table S4. Of the 12 most important variables, only one wasrelated to HCl chlorination. The chlorination method was correctlypredicted in 70% of the spiked matrix test set samples independentof classification algorithm . However, since RF is a tree-based algorithm, it does not requirethe use of such a decision tree; the tree was only used with RF modelsto permit detailed performance comparison with OPLS-DA. RF could alsobe used to resolve all of the classes in a single model; accordingly,11-class models were constructed for both crude samples (M4crude) and matrix samples (M4matrix). The M4crude model outperformed the OPLS-DA decision tree models for crude samples,with 10/11 correct classifications  performed less well than the hierarchical decisiontree models generated using both RF and OPLS-DA. In the test set validationof model M4matrix . Such data would be difficult to acquire because the compositionof the HD reference samples is not necessarily stable over time. ACAS HRMS library would enable easy processing of new samples and couldalso be shared between laboratories. CAS for inclusion in such a databasecould be selected based on their relevance for route classification.Some (48/714) of the CAS from crude HD and spiked matrix samples werefound in all three datasets, but many variables were matrix-specific4. This d33 used inorder to support court decisions. It could be applied to the classificationmodels presented above, in which estimated assignment probabilitiesare used, together with prior information, to calculate likelihoodratios of competing propositions. The outcome  can then be communicated to court hearings in a transparentand scientific sound way. The methods for such calculations basedon multivariate data are currently under development.34Bayesian statistics is a widely acceptedforensic statistic frameworkWe successfully developed a nontargetedmethod for attributionof crude HD and spiked matrix samples to specific synthesis routesusing HRMS data. The nontargeted approach enabled efficient processingof large numbers of CAS, and the high recovery of CAS facilitatedthe generation of significant HD route classification models usingtwo independent classification methods, OPLS-DA and RF. The two classificationmethods achieved similar classification accuracy. Classification performancewas very good for crude HD samples but somewhat lower for spiked matrixsamples due to matrix effects. Route determination of the spiked matricessamples was made difficult due to the low spiking level. Real-worldsamples involving HD, may have higher concentrations, making the classificationof routes easier. Overall, our results show that nontargeted methodscan be valuable tools for CAS screening in chemical forensics, andthat a CAS library could be a powerful tool in future investigationsinto alleged uses of CWA.


The increasing use of mass cytometry for analyzing clinical samples offers the possibility to perform comparative analyses across public datasets. However, challenges in batch normalization and data integration limit the comparison of datasets not intended to be analyzed together. Here, we present a data integration strategy, CytofIn, using generalized anchors to integrate mass cytometry datasets from the public domain. We show that low-variance controls, such as healthy samples and stable channels, are inherently homogeneous, robust against stimulation, and can serve as generalized anchors for batch correction. Single-cell quantification comparing mass cytometry data from 989 leukemia files pre- and post normalization with CytofIn demonstrates effective batch correction while recapitulating the gold-standard bead normalization. CytofIn integration of public cancer datasets enabled the comparison of immune features across histologies and treatments. We demonstrate the ability to integrate public datasets without necessitating identical control samples or bead standards for fast and robust analysis using CytofIn. Challenges in batch normalization and data integration limit the comparison of existing mass cytometry datasets. Here, the authors report CytofIn that can integrate mass cytometry datasets from the public domain and reveal cellular features associated with immune oncology by analyzing five public cancer datasets. Mass cytometry offers the ability to analyze millions of cells quickly and inexpensively compared to single-cell genomic platforms yet can be combined with approaches like single-cell RNA sequencing for the complimentary analysis of cell populations2. The utility of mass cytometry has been demonstrated in numerous studies relating to immune response, cancer, and healthy tissue development8. Furthermore, mass cytometry is being widely integrated into correlative studies in clinical trials such as in the Cancer Immune Monitoring and Analysis Centers (CIMAC) and the Partnership for Accelerating Cancer Therapies (PACT) initiatives, where it has been used to determine immune correlatives associated with clinical outcomes in cancer immunotherapies9. The rapid growth in the applicability of mass cytometry for clinical measurement also led to an increased availability of mass cytometry data in the public domain. Platforms such as FlowRepository and Cytobank support storage, annotation, and sharing of flow and mass cytometry datasets11. Consequently, integrating datasets from different studies for comparative analysis emerges as a desired approach that could lead to unexpected discoveries not afforded by individual studies.Mass cytometry (cytometry time of flight or CyTOF) is an increasingly widespread technique for the discovery and monitoring of cell populations using single-cell, high-parameter protein measurements12  of median signal intensity when comparing healthy human peripheral blood mononuclear cells (PBMCs) analyzed at six different centers17. In this study, all PBMC samples were prepared at a single center a priori, therefore removing one common source of variability. Since bead information is not prevalently shared between experiments conducted at different centers, public datasets lacking bead data often cannot be normalized with new datasets. In addition, bead data only corrects for unimodal instrument sensitivity differences across channels.Within individual studies, batch effects are commonly addressed using bead normalization20. However, the lack of identical replicates across existing datasets remains a major limitation to applying these methods for the analysis of public datasets or datasets collected over time19. Non-anchor-based normalization methods like quantile normalization estimate a reference distribution based on the average of each quantile across all samples under the assumption that the statistical distributions of all samples are identical21. Although identical controls are not required when using quantile normalization, the reference distribution needs to be re-estimated with the addition of new samples and the assumption that all samples are identically distributed can remove features of biological importance as well as introduce artificial bias22. Thus, a method to enable cross-dataset comparison without identical technical replicates is needed  in each batch to correct data distributions of protein signals based on a goal distributionhttps://github.com/ParkerICI/premessa) enable manual editing of panel labels, preprocessing panels from large cohorts is prohibitively time-consuming and therefore necessitates automation.File heterogeneity between datasets is common when different naming conventions are adopted for antibodies or antibodies are labeled on different metals between panels. Homogenizing such heterogeneous files is often the first step to enable comparative analyses yet is currently unwieldy for large datasets. Panel homogenization refers to the process of aligning the antigen panels across multiple CyTOF experiments by removing channels not shared across cohort samples and standardizing the antigen name used in each channel. When comparing mass cytometry data across independent sources, multiple panels will need to be homogenized simultaneously. Although programs like Fluidigm and Premessa (Here we present CytofIn (CyTOF Integration), a computational pipeline for integrating mass cytometry data from the public domain using generalized anchors, which include healthy control samples and stable channels that exhibit low signal variability across datasets and can be used as an approximation of identical anchors Fig. . Using dHealthy controls are potential generalized anchors for batch normalization due to their low individual variability as well as their wide availability from clinical experiments. We first evaluated the within- and between-cohort variability using seven cohorts of acute lymphoblastic leukemia (ALL) samples collected over 6 years in our laboratory. Within each cohort, a healthy bone marrow or peripheral blood control was included in each barcode plate for a total of 989 leukemia samples distributed across 50 plates. The ALL samples were analyzed using three antibody panels on two generations of mass cytometers where each antibody panel had slight variations in naming conventions, the number of proteins measured, and the protein-metal label Fig. . Here, t23.To facilitate cross-dataset analysis, we developed a computational pipeline for automated panel homogenization of leukemia samples across the datasets. To enable batch preprocessing, a metadata table consisting of subfields including cohort names, plate numbers, patient identifiers, and perturbation conditions was used to annotate the datasets (Supplementary Data 4), we did not observe significant differences in the overlap in distributions of the mean protein expression in the stimulated or basal condition across the 36 consensus markers  than the patient subcluster (node degreeavg = 10.9), as revealed by denser between-node edges, indicating smaller within-sample variations  were aggregated by concatenating the mean expression of 36 consensus channels from all cells to define a reference distribution, referred to as universal reference. To select healthy anchors from each plate, we prioritized healthy samples in basal over stimulation conditions or stimulation conditions that have minimal effect on the expression. The data distribution of each healthy anchor was adjusted to the universal reference distribution using normalization functions (see Methods). The same adjustment was then used to normalize patient samples in groups per barcode plate  normalization function that scales original mean expression based on the slope of a best-fit regression curve between healthy and universal reference expression across all channels, similar to that performed by the bead standardization procedure  by projecting onto the same embedded space of the original samples  and compared performance using two benchmarking metrics kBET and LISI, which measure local batch and cell mixing effects by comparing the local and global label distribution on the UMAP space  and CytofRUV (CV), and Seurat (ST), a batch correction method for scRNA-seq  obtained on a CyTOF 1 cytometer and Batch B (cohort 5–7) obtained on a Helios cytometer, where these two cytometers exhibited differences in sensitivity for signal readout (Supplementary Data –4 obtain2 > 0.99) to bead normalized signals  values   to rank-order the variability of each marker across the three patient samples and identify stable channels  Fig. .2 > 0.99)   Fig. . Importa99) Fig. . We yiel99) Fig. . Togethe35. To test the utility of the CytofIn pipeline for public databases, we first examined the number of datasets that may have sufficient overlapping panels to use with CytofIn. We queried the Flow Repository database for any dataset tagged with the term PBMC. This retrieved a total of 44 mass cytometry datasets  . The datasets shared 11 consensus markers and CytofIn used the top three stable channels for normalization. We found that before normalization, the TIL’s from the breast cancer samples did not cluster together but following CytofIn normalization these samples clustered together and away from glioma, a known cold tumor with fewer infiltrating immune cells  across four different cancer histologies  monotherapy, but the effects of these immunotherapies on CD4+PD1+ T cells were not well-characterized40. To investigate how each immunotherapy affects T cells and other immune populations in melanoma patients, we integrated these datasets for combined analyses: dataset 1 containing the pretreatment and Pembrolizumab-treated PBMC’s from Greenplate et al. and dataset 2 containing Pembrolizumab, Nivolumab, Ipilimumab, or a combination of Ipilimumab and Nivolumab treated PBMC samples by Wei et al.   and from the Pediatric Clinic of University of Milano-Bicocca  and from St. Jude Children’s Research Hospital . The use of these samples was approved by the Institutional Review Boards at each institution. Cryopreserved primary bone marrow and peripheral blood samples, from leukemia or lymphoma patients or healthy controls, thawed rapidly in thawing media 16. Cohorts 1–5 were acquired on a CyTOF1 generation machine, while Cohorts 6 and 7 were acquired on the Helios (3rd generation CyTOF).After ex vivo perturbation, cells were fixed with paraformaldehyde, washed in cell staining media (CSM) twice, followed by one wash in PBS and one wash in PBS + 0.02% saponin. Cells were then 20-plex barcoded, using new in-house batch preparations. Aliquots of the same healthy BM controls were used, one per barcode plate. Samples were washed in CSM after barcoding and combined into a single tube. Blocking was performed with Human TruStain FcX receptor blocking solution . Cells underwent surface staining with surface markers outlined in Supplementary Table 16. Following bead normalization, the FCS files were de-barcoded into individual samples48.Data acquired for internal cohorts on the CyTOF1 was downloaded and re-processed along with external cohorts acquired from the Helios. Due to the limitations of the first-generation instrument, some FCS files were re-processed to include the IMD header, re-extracted from IMD and concatenated through the Fluidigm software. Once all barcoded files for internal and external cohorts were concatenated into single individual files, the data was bead normalizedhttps://github.com/bennyyclo/cytofin).Before data normalization, the antigen panels in FCS files were homogenized. Briefly, a standard antigen panel containing metal name, antigen name, regular expression pattern, and the standardized name was generated  as generalized anchors to batch correct each plate relative to the universal reference. The universal reference is obtained by concatenating and averaging the data from each plate’s generalized anchor. The second of CytofIn’s batch normalization strategies uses the combined datasets’ most stable  channels as generalized anchors in order to batch correct all other antigen channels in the consensus CyTOF panel. The implementation of these strategies is discussed in-depth below:Batch normalization using healthy control samples: CytofIn batch normalization using healthy control samples as generalized anchors is performed in 3 steps. First, one control sample per batch is identified as that batch’s generalized anchor. Second, universal reference statistics are computed using the combined single-cell data from all generalized anchors. Third, the differences in marker expression means and/or variances between each generalized anchor and the universal reference are used to define a normalization function (from a choice of 5 functions) that adjusts each generalized anchor to match the universal reference. Finally, the same normalization function is then applied to all other samples of the same batch until all samples are batch normalized. The mathematical details of this procedure are outlined in the Supplementary Methods.ccontrol is the total number of cells across all control samples and m is the number of markers in the consensus antigen panel. The mean signal intensity (MSI) of the universal reference, j, 1:m][ represents an m-dimensional vector corresponding to the jth row of X. Likewise, the vector of marker variances Determination of the universal reference. The universal reference for batch normalization is computed by concatenating the single-cell data from all control samples to form an expression matrix  and Var are used in the next step of the algorithm.Both MSIFitting healthy control samples as generalized anchors to the universal reference. We have proposed five normalization functions for fitting the mean expression of generalized anchors to the universal reference based on a combination of the anchors’ and reference’s mean and variance values.j, which we denote as i across all cells in the generalized anchor  from batch j. Similarly, we define the marker variance vector across all cells in the control sample from batch j, which we denote as i across all cells in the generalized anchor  from batch j.First, we define the mean signal intensity (MSI) of the control sample from batch j, we also define the single-cell expression matrix obtained by concatenating (row-wise) cells from all samples within batch j. We denote this expression matrix ascj is the total number of cells in batch j and m is the number of markers in the consensus panel.For each batch Using these definitions, we can then apply any of the following batch normalization functions to the data from batch j to estimate and correct batch effects:j based on the differences between the entries of j.Meanshift normalization. This function performs per-channel additive adjustment to each entry in TMeanshift bulk normalization. This function performs additive adjustment based on the difference in the mean (across markers) of j, Variance normalization. This function first performs additive correction  followed by scaling based on the ratio of the standard deviation (SD) value of the generalized anchor from batch j is generated as follows:Z-score normalization. This function performs Z-score standardization. The batch-corrected expression matrix . The slope was approximated by regression analysis. The batch-corrected expression matrix j is generated as followsBead-like normalization. This function performs a multiplicative correction based on the slope between 34. Second, the n most stable channels in the combined dataset are used to establish a universal reference that can be used to batch correct the expression values of all samples in the combined dataset. Finally, the meanshift bulk normalization function is used to perform the batch correction for all samples in the combined dataset. Each of these steps is discussed in more detail below:Batch normalization using stable channels. CytofIn batch normalization using stable channels as generalized anchors is performed in 3 steps. First, the most stable channels across all datasets being integrated are identified using a principal components-based non-redundancy score (NRS)c(s) is the number of cells in sample s and m is the number of markers in the consensus antigen panel. For each sample, the top three principal components (PCs) of the expression matrix T(s) are then computed to yield the top three PC loadings for each marker. Doing so results in the matrix k is the number of principal components . In addition, the standard deviation (SD) vector j can then be rank-ordered using a PCA-based non-redundancy score (NRS) as previously described34:i-th PC in sample s, j-th row and i-th column of P(s)  and R is the total number of input samples to be integrated). Thus, each NRSj represents the average  of the following quantity for a given marker j, Identification of stable channels to be used as generalized anchors. To identify the most stable channels across multiple samples from different batches, we first express each sample s as a single-cell expression matrix, j calculations from the previous step, the universal reference can then be computed. To do so, we use the average expression of the n markers with the lowest NRSj in the matrix T, which we obtain by concatenating all R samples’ expression matrices T(s) row-wise such that c is the total number of cells in the combined dataset and m is the number of markers in the consensus antigen panel. Thus, if we let q index the n most stable markers (from 1 to n) as defined above, we can calculate the q-th entry in the universal mean signal intensity vector Determination of the universal reference. Using the NRSq-th entry in each sample’s mean signal intensity vector Likewise, we can define the  and MSI(sample) to apply the meanshift bulk normalization function defined above and yield the batch-corrected expression matrix for sample s, which we denote as s, Ts, we can calculate Finally, we can use these values of MSI39. As a proxy to manual gating for the developmental classification of ALL samples, we applied rectangular gates and a threshold of 10 counts for defining positive and negative cells. To enable controlled comparison, the same gating parameters were then applied to all samples in batch to retrieve the corresponding subpopulations. The computational gating was performed using the FlowCore and openCyto R software package.Prior to computational gating, the raw mass cytometry data were transformed using the arsinh function with a scaling factor of 5. The gating parameters were optimized on the healthy samples: bone marrow mononuclear cells for B-cell leukemia or the pretreatment PBMC sample for melanoma. Each subpopulation was gated based on the biaxial plot as previously reported49. 2000 cells were sampled from three healthy samples of each 7 cohorts and combined to form cohort-specific healthy samples. The original and bead normalized healthy samples were also included as negative and positive controls. Furthermore, comparisons between normalized data using generalized anchors based on healthy control, stable channels, or the combination were also performed. To evaluate local batch and cell mixing effects, 10% of the samples (200 cells) were randomly sampled and mapped onto the UMAP space defined by all 36 channels in the original healthy sample space. To assessed cell mixing, the healthy cohort samples were computationally clustered into 100 cell types using flowSOM. We compared local label distribution to global distribution using the k-nearest neighbor batch effect test (kBET) and the local inverse Simpson’s index (LISI). kBET computed local batch label distribution based on the k-nearest neighbors around each data point after SVD dimension reduction. We used the expected rejection rates (0–1) as the kBET index to quantify if the null hypothesis that all batches being equally mixed is rejected. Similar to kBET, LISI assessed local label distribution of a fixed number of k-nearest neighbors and perplexity. We used the Simpson’s index outputted from the LISI program as LISI index to quantify the diversity of label distribution in a local neighborhood.The performance of CytofIn normalization on a combined sample consisting of three representative healthy control samples from 7 cohorts was compared to three existing normalization methods, CytoNorm, CytofRUV and Seuratwww.r-project.org). Statistical parameters for protein marker distribution, change in expression and population abundance were quantified based on their mean and standard deviation (mean ± s.d.). Comparison between mean signal intensity or population abundance of mass cytometry files was performed using the Wilcoxon test. Comparison between bead and batch normalized signal was quantified using correlation analysis quantified by the R-square values or root-mean-square deviation (RMSD). Clustering quality assessments were quantified using the adjusted rand index to compare the similarity of clustering pre- and post-normalization to the pre-assigned cancer type label based on the mean expression values.Statistical analysis was performed using the R statistical software (https://github.com/bennyyclo/Cytofin/).The mass cytometry data were extracted and processed using the Fludigm CyTOF software (version 7.0). The mass cytometry data were analyzed using the following packages from the R bioconductor software (version 2.2): flowcore, ggcyto, openCyto and flowWorkspace. Network and statistical analysis were performed using the Cytoscape software (version 3.8.0) and the R statistical software (version 3.6.3). Benchmark studies were conducted using the CytoNorm, CytoRUV and Seurat (version 4.0.2) software packages and kBET, LISI and FlowSOM R packages. Data integration and normalization were performed using the CytofIn R package (Further information on research design is available in the Supplementary InformationDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Reporting Summary


Bulk RNA-seq analysis can only detect average change in expression levels and is unable to identify this dynamics. Single cell RNA-seq presents an unprecedented opportunity that helps in placing the cells on a hypothetical time trajectory that reflects gradual transition of their transcriptomes. This continuum trajectory or ‘pseudotime’, may reveal the developmental pathway and provide us with information on dynamic transcriptomic changes and other biological processes. Existing approaches to build pseudotime heavily depend on reducing huge dimension to extremely low dimensional subspaces and may lead to loss of information. We propose PseudoGA, a genetic algorithm based approach to order cells assuming that gene expressions vary according to a smooth curve along the pseudotime trajectory. We observe superior accuracy of our method in simulated as well as benchmarking real datasets. Generality of the assumption behind PseudoGA and no dependence on dimensionality reduction technique make it a robust choice for pseudotime estimation from single cell transcriptome data. PseudoGA is also time efficient when applied to a large single cell RNA-seq data and adaptable to parallel computing. R code for PseudoGA is freely available at  Cellular level gene expression profile can reveal the heterogeneity within a tissue and provides valuable information about ongoing biological processes inside a cell . In bulkGenes responsible for circadian rhythm, metabolism, cell death process etc. are regulated in a synchronized manner in different cells. Function of a cell may be affected by stages in development process, cell state transition, spatial effect, interaction with environment, cell–cell interaction and other internal ongoing processes. Effects of these simultaneous processes add to the heterogeneity in expression levels of thousands of genes at cellular level ,10. ThusAvailable methods in the literature mainly focus on dimensionality reduction followed by mapping of cells to a trajectory. The dimensionality reduction is performed by principal component analysis (PCA) , indepenExisting pseudotime construction algorithms are mainly based on construction of minimal spanning tree, kNN graph or principal curve fitted on first two reduced dimensions. The accuracy of a method depends on the dimensionality reduction method being used in the first step and the amount of information that is lost during converting original data to lower dimensions. To check whether different types of dimensionality reduction algorithm can indeed construct the true pseudotime properly and retain most of the information that is in the original data, we simulate three dimensional data, under three scenarios Figure . In eachWe apply different algorithms for each scenario Figure . In scenOur simulation shows that dimensionality reduction techniques may not always capture the full information about the pseudotime trajectory especially when few genes behave typically, like piece-wise linear etc. This simulation makes it clear that any method that is directly based on the actual gene expression values would have a higher chance to use more information and might provide more efficient and robust pseudotime ordering. We propose a novel method for pseudotime ordering of cells that is directly based on actual gene expression levels. Our method ‘PseudoGA’ uses genetic algorithm to come up with a best possible trajectory of cells that explains expression patterns for individual genes. Another advantage of this method is that it can identify any lineage structure or branching while constructing pseudotime trajectory.For pseudotime estimation we apply genetic algorithm, which is appropriate for the current problem, to develop ordering of cells in the entire cell population. If the lineage structure or the branching between cell populations is of interest in addition to pseudotemporal ordering, cells are clustered into homogeneous subpopulations before applying the algorithm. The subpopulation structure can also be provided as input. Next, we apply the same algorithm to construct ordering of cells within same cell types. Finally, another subroutine concatenates the ordered paths from different clusters to form a tree like structure. Another highlight of our method is that our pipeline produces an undirected tree, connecting the paths from each cluster when no information on root cell is available. However, if the root cell or the cluster is identified or specified, our algorithm would provide an ordered tree. No transformation or dimensionality reduction is used in the pseudotime estimation step. We utilize full information from gene expression values within cells. However, if the lineage or branching structure is not of interest, the entire cell population is considered as one single subpopulation.S = ((sij)) where sij is the gene expression corresponding to ith cell and jth gene  which is indeed a permutation of 1, 2, …, n. Since the chromosomal representation is only for computational purpose and has no biological significance, recombination, mutation and selection operators when applied on a permutation give birth to a new one that needs to be checked for a better solution.Ordering of lgorithm  is a comlgorithm . It useslgorithm  of a canRij is the rank of cell i in the expression levels of gene j, ti is the pseudotime for cell i and εij is the random error term. fj is an unknown function according to which gene expression changes over pseudotime. In our set up, ti ∈ {1, 2, …, n}, for all i and {t1, t2, …, tn} is a permutation of {1, 2, …, n}. If we approximate fj by a cubic polynomial, the regression equation becomes:jk, k = 0, 1, 2, 3, for a pseudotime ordering . Then, the cost associated with the ordering of j-th gene by cubic polynomial is given by Bayes Information Criterion: j1 and BICj2 as BIC values associated respectively with fitting linear and quadratic polynomial on the rank of expression values with pseudotime as explanatory variable. Now the cost associated with j-th gene for the given pseudotime is Cj = min. Overall cost associated with the pseudotime {t1, t2, …, tn} for the whole transcriptome expression profile is nG is the total number of genes. It is important to note that we treat the zero expressions in the data as numeric zeros and use them in ranking. If there are ties in expression values, we assign average rank to all observations with ties. The introduction of the cost function fj adds more flexibility to our model. Any prior knowledge leading to more specific form of fj can easily be incorporated in the model and the entire downstream protocol will follow accordingly. Naturally, this would result in more efficient estimation of pseudotime.Expression values of a gene over the pseudotime path may be a linear or nonlinear function of pseudotime. To make our proposed algorithm more general, we assume that the rank of the expression values over cells is a polynomial of pseudotime of degree at most 3 Figure , E. By un}. The cost function C is a function C contains penalty incurred due to non-optimality of an ordering. Hence, the optimal pseudotime ordering is obtained by minimising C(x) with respect to x that moves in the space of all possible permutations i.e. xopt is the optimal ordering, we have xopt. The algorithm uses the entire information from the dataset without any dimensionality reduction. Although it may not always find global optimality, it provides a reasonably good solution, at least, because no information is lost due to dimensionality reduction. Note that some other discrete optimisation algorithms may be used to address this problem. But we restrict to genetic algorithm and its modification tuned to this problem, mainly due to its wide applicability and better performance.Let y from a given permutation x by randomly choosing two positions i and j with i ≤ j such that yk = xk(1 − Ii ≤ k ≤ j}{) + xi + j − kIi ≤ k ≤ j}{ where I{.} denotes the indicator function. If xi and xj are the values of x at positions i and j respectively, after mutation the new values would be yi = xj and yj = xi. This mutation operator is essentially an inversion  fragments of equal length are formed from the parent string. Alternate fragments from one of the parents, say string I, are retained in one of the two newly formed offspring strings. To fill up the missing positions, the entries not retained in the newly formed string are recorded. A bipartite graph is constructed with the positional indices of those left behind entries of the two parent strings as vertices on two sides. All possible edges between vertices on the two sides are considered and the absolute differences between the ordinal position values are taken as edge weights. Based on the bipartite graph thus created, minimal bipartite matching is constructed. Entries in the positions considered in the other string, say string II, are put into the corresponding positions in string I based on the minimal bipartite matching graph. Using the same approach, another child is created by interchanging the role of string I and string II. Thus for an existing parent population of size N, same number of offsprings are added to it. It can be easily pointed out that the offsprings generated from this operation are not unique because the solution of minimal bipartite matching is not unique.Several recombination or crossover operators on permutations have been suggested, e.g. partially mapped crossover , order cN individuals from a pool of N. In the selection step  is the cost function as defined before and the ‘floor’ function ⌊x⌋ denotes the greatest integer less than or equal to x.Till now we consider applying our method to the entire dataset. However, if we want to see any existence of branching, first we have to cluster the data. PseudoGA will be applied on each cluster considering it as the full data. Once the pseudotime orderings within the clusters are formed, we can construct lineages assuming a continuum between clusters Figure . This isA common approach to construct lineage from disjoint clusters of homogeneous populations of cells is by constructing minimal spanning tree (MST) on cluster centres ,21,22. HGenetic Algorithm for finding optimal permutation scales poorly with number of cells. Some modification of our algorithm is required to construct pseudotime with large number of cells. First, we subsample a smaller number of cells and apply our proposed method.N total cells, pseudotime is estimated based on a subset of n (n < N) cells. Suppose  is the vector of estimated pseudotime for n cells. We define a score for every cell j, Nr(j) is the set of r nearest neighbors of cell j. The vector S =  or the ordering of S can be considered as the pseudotime for the original set of N cells.Out of B (say 30) subsamples each of size M (say 100) from the same dataset, we construct pesudotime trajectories separately. We find score  of the jth cell corresponding to each b-th trajectory and construct a principal curve based on B dimensional scores of individual cells. The principal curve has been used for pseudotime reconstruction in different manners  were expanded under high mitogen conditions and then differentiated by switching to low mitogen media (GSE52529) ,50. RNA-If we are interested to see any lineage or branch structure with respect to pseudotime, we have to cluster the original data and apply our algorithm on each cluster. Clustering with t-SNE creates three clusters with one cluster consisting of observations from 0 hours only (Cluster II), two other clusters (Cluster I and III) with mixture of observations from 24, 48 and 72 h . Cells fPseudoGA has the highest correlation among all methods under consideration when applied on the entire dataset Figure  althoughOur next dataset contains single cell RNA-seq data from 1861 mouse dendritic cells stimulated with three pathogenic components. This dataset is used to examine the variation between individual cells exposed to the same stimulus and study the complex dynamic responses to the stimulus exhibited by multicellular populations (GSE48968) . Cells wWe apply pseudotime estimation algorithms on all these three types of data with different stimuli. Application of PseudoGA on the entire data for each stimulus shows that estimated pseudotimes are in overall congruence with the actual pseudotime Figure . First tThe third dataset consists of microfluidic single cell RNA-seq on 185 individual mouse lung epithelial cells at four different stages  of development (GSE52583) . The traMonocle shows the highest correlation with actual pseudotime whereas PseudoGA turns out to be second best Figure . Top 6 gIn the fourth dataset, we study the gene expression patterns at single cell level across three different cell cycle stages each containing 96 mouse embryonic stem cells (E-MTAB-2805) . Single Maximum correlation among all possible permutations of G1, S and G2M stages were considered in this case as the order of cell cycle can be rearranged. Here again, PseudoGA shows the highest correlation among all methods Figure . Top 6 gWe consider another dataset where gene expression profiles of human ventral midbrain cells were studied at different developmental stages after gestation ranging between 0 and 11 weeks (GSE76381) . This daWe simulate datasets to evaluate different methods and compare their performance to PseudoGA. Simulations were performed with two different frameworks: simulation with our own simulation model and three other simulation schemes available in Bioconductor package ‘Splatter’ . Our simNote that all genes are not expressed in all cells. So, conditional on the fact that a gene is expressed in a cell, the read count corresponding to this gene is generated from a Poisson distribution whose mean follows a Gamma distribution. The shape parameter of the Gamma distribution for each expressed gene lies on a pseudotime curve. If a gene is not expressed in a cell, we consider its read count to be identically equal to zero. We consider different gene sets where a particular trend is being followed for a set. Thus expression values within a given set may follow increasing or decreasing linear trend, quadratic or sinusoidal trend. To add more generality, we also consider few genes whose expressions are independent of pseudotime.We know that the abundance of technical zeros or dropouts ,55 is a Note that if a gene regulation has a prominent relation with pseudotime, it would show a trend, at least approximately. In order to capture this trend, it is expected that enough information should be available for that gene. Hence it is natural to believe that the amount of technical zeros, which is a common characteristic of single cell data, should be relatively small.So the first two scenarios are more rational when the dataset is of good quality or genes with lower dropout rates are filtered successfully before pseudotime estimation. Scenario III is relevant when the data contain too many technical zeros and in addition to that, either gene filtering cannot separate out genes with lower dropout rates or no filtering is applied.We apply PseudoGA and other commonly available methods to these simulated datasets. Entire study is based on 100 replicates under each simulation scheme. We assess the accuracy of each method using two criteria, (i) absolute rank correlation coefficient between estimated pseudotime with the actual one and (ii) number of genes that show functional relationship with the estimated pseudotime.We present boxplots of the two criteria for all methods under consideration. Results of our simulation study indicate that our PseudoGA shows superior performance compared to other methods for the first two scenarios while for the third its accuracy is at least as good as other methods Figure . Thus PsOur simulation method is very general and has very little (or no) bearing with PseudoGA. However, to see its performance in wider scenarios, we also simulate expression data using Bioconductor package ‘splatter’ under three different methods: PROSSTT , Splat  and PhenTo verify whether our subsampling based approach performs comparably with other methods, we simulate 3000 cells and100 genes using PhenoPath under the third scenario. Details of the simulation procedure is described in Accuracies of different methods were compared using the same two criteria as before. Absolute rank correlation coefficient is a measure of concordance between estimated pseudotime and the actual pseudotime, whereas number of functionally related genes is an evidence based measure of concordance. The performance of PseudoGA is consistent across all three situations Figure . MonocleWe also check the performance of PseudoGA in case of a large dataset and effectiveness of the subsampling based approach. We simulate a trajectory with 10 000 cells with expression values following Gamma-Poisson distribution and construct trajectory using PseudoGA with only Runtime of any genetic algorithm depends on population size in each generation and the number of generations. In general, increasing the values of these parameters will improve the accuracy of an algorithm and at the same time will increase the runtime. In this article, the cost function has been evaluated on 400 permutations in each generation and a minimum of 30 generations have been considered in all simulation and real data analyses. The value of ε was taken to be a pre-assigned small positive number. Different algorithms scale differently with number of cells and number of features . PseudoGTo assess scalability of PseudoGA, we benchmark PseudoGA runtime against runtime of other methods. We consider two types of count data generated by Splatter: one with 300 cells and 10 000 features and the other with 3000 cells and 1000 features. In the second scenario, we run PsedoGA with three subsamples each of size 100 coupled with nearest neighbor matching and principal curve fitting. The boxplots of the runtimes based on 100 replicates are shown in Figure Several algorithms have been designed to order cells along pseudotime trajectory based on single cell RNA-seq data. These are mainly based on the philosophy of constructing trajectory on reduced dimensional data. Some genes might show distinct types of functional pattern of expression levels along different transcriptomic stages. Hence, it may be possible that information on gene expression level might be lost to some extent, sometimes substantially, during the dimensionality reduction step. This loss of information may lead to erroneous outcome at the next step while inferring the ordering of cells. Entire pseudotime construction depends on the amount of information captured in reduced dimensionality of data. Moreover, few genes may remain approximately constant over the entire time trajectory, whereas few may be outliers. Presence of such genes might influence the pseudotime construction. We devise an algorithm ‘PseudoGA’ that searches for the best possible ordering of cells in the set of all permutations and infers the ordering based on actual gene expression levels.Our proposed method PseudoGA assumes that the dependency structure of gene expression on pseudotime is based on ranks of its values. This allows our method to encompass a large class of functions that gene expression values can assume along a trajectory. To tackle with zeros, we accommodate average ranking to all cells with zero expressions for a given gene. This nonparametric assumption makes our method robust in different types of single cell expression datasets.We first cluster the cells in homogeneous subgroups and apply genetic algorithm on each of these homogeneous groups to increase the efficiency, followed by a novel method to concatenate paths from different clusters. This will help us in identifying any lineage or branching structure that may exist in the data with respect to pseudotime. Otherwise, we can always apply our algorithm directly to the entire dataset.Compared to other existing methods, PseudoGA seems more robust when applied on various real datasets as well as on simulated datasets. PseudoGA has been shown to be consistent in various simulation schemes. In presence of outliers or highly variable genes, methods based on dimensionality reduction could fail but PseudoGA maintains its accuracy and robustness.Our proposed method can be applied to a variety of datasets with even large number of cells. Our study reveals that even in such a situation, the performance of PseudoGA is extremely well both in terms of accuracy and time. We speculate that further improvements on scalability of the method are possible by implementing a more efficient genetic algorithm and parallelization of the code in case of large datasets. One can use more operators in addition to the three operators used in PseudoGA and apply them in different manners. Improvement in the genetic algorithm would certainly improve the efficiency of PseudoGA. To the best of our knowledge, this is probably the first application of genetic algorithm in pseudotime estimation with some novel ideas and methods inbuilt in the main algorithm. PseudoGA is a freely available software implemented in R and can conveniently be applied on any single cell expression data.gkab457_Supplemental_FileClick here for additional data file.


Intersectionality recognizes that in the context of sociohistorically shaped structural power relations, an individual's multiple social positions or identities  can interact to affect health-related outcomes. Despite limited methodological guidance, intersectionality frameworks have increasingly been incorporated into epidemiological studies, both to describe health disparities and to examine their causes. This study aimed to advance methods in intersectional estimation of binary outcomes in descriptive health disparities research through evaluation of 7 potentially intersectional data analysis methods: cross-classification, regression with interactions, multilevel analysis of individual heterogeneity (MAIHDA), and decision trees . Accuracy of estimated intersection-specific outcome prevalence was evaluated across 192 intersections using simulated data scenarios. For comparison we included a non-intersectional main effects regression. We additionally assessed variable selection performance amongst decision trees. Example analyses using National Health and Nutrition Examination Study data illustrated differences in results between methods. At larger sample sizes, all methods except for CART performed better than non-intersectional main effects regression. In smaller samples, MAIHDA was the most accurate method but showed no advantage over main effects regression, while random forest, cross-classification, and saturated regression were the least accurate, and CTree and CHAID performed moderately well. CART performed poorly for estimation and variable selection. Sensitivity analyses examining the bias-variance tradeoff suggest MAIHDA as the preferred unbiased method for accurate estimation of high-dimensional intersections at smaller sample sizes. Larger sample sizes are more imperative for other methods. Results support the adoption of an intersectional approach to descriptive epidemiology. •Intersectionality may inform methods for describing health inequity.•Seven statistical and machine learning methods were evaluated for accuracy.•For binary health outcomes, method performance was quite sensitive to sample size.•MAIHDA was preferred for accurate intersection-specific estimation in small samples.•In larger samples most methods out-performed non-intersectional main effects. Since tIntercategorical intersectional analyses describe intersectional groups, and differences between them . CalculaMany health outcomes are binary, with a wide range of prevalences, creating challenges when studying high-dimensional intersections. Low-prevalence outcomes are prone to influence by outliers and require large samples to produce suitable numbers of events, problems exacerbated by increasing numbers of intersectional subgroups. For conventional regression analyses with binary outcomes, inclusion of higher-level interaction terms reduces the probability of model convergence and increases the potential for both high variance in estimates leading to wide confidence intervals and for bias away from the null . These iThis motivates the evaluation of currently used alternative methods for intersectional analyses of binary outcomes . ClassifWe previously evaluated intersection-specific estimation accuracy of these conventional and alternative methods (except CHAID) for continuous outcomes, using simulated data . We founThe objective of this study was thus to evaluate seven methods  for accuracy in intersection-specific prevalence estimation, alongside a non-intersectional main effects regression and a perfectly specified (but impracticable) regression model, using a variety of simulated but realistic data scenarios. We then sought to demonstrate analyses using National Health and Nutrition Examination Survey (NHANES) 2015 to 2018 data on high blood pressure. Finally, the decision tree methods were assessed for variable selection performance. While epidemiological studies often focus on significance testing and estimation of effect sizes or interactions, accurate outcome estimation for subpopulation groups is an important objective for population health. Therefore, we focus on improving estimation. This study is an opportunity to reexamine how we approach descriptive epidemiology for binary outcomes from both a theoretical and statistical standpoint. While it originated in concerns regarding intersectionality methods, results are relevant to all epidemiological applications that face similar statistical challenges in exploring heterogeneity in binary outcomes.22.1A rare outcome was simulated with an average prevalence of 3% (range: 1.35%–5.60%). A common outcome was simulated with an average prevalence of 15% (range: 7.15%–28.58%). Both outcome types were created with a set of categorical inputs and a set of mixed inputs, resulting in four possible models. Each model was iterated 1000 times for four different sample sizes . This simulation was structured with identical sample size parameters and input variable combinations as our paper on continuous outcomes , to alloOutcome generation formulas were:2.1.1P(Y=1) = exp (intercept + β1.1 (if X1=1) + β1.2 (if X1 = 2) + β1.3 (if X1 = 3) + β2X2 + β3X3 + β4X4 + β5X5 + β6(if X1=2 & X2=1) + β7(if X1=3 & X2=1) + β8X3*X4*X5).2.1.2P(Y=1) = exp (intercept + β1X1 + β2X2 + β3X3 + β4X4 + β5X5 + β6 X1*X2 (if X1>1 & X2=1) + β7X3*X4*X5).Coefficients were sampled so that outcome probability was less than or equal to 1.https://github.com/m-mahendran/methods_for_intersectionality_simulation_binary_outcomes). Analyses were conducted in R version 3.6.1  in the categorical inputs model; other variables were binary. Input variable structure is described in 2.22.2.1cross-classification, the prevalence of the outcome within each intersection was calculated with no further statistical adjustment. Correctly-specified regression included only the lower-level coefficients (the intercept and variables X1 to X6) and the interaction terms modeled into the simulated data . Saturated regression included all possible lower-level and interaction terms, and represents a more real-world application wherein the underlying data structure is unknown. Note that interaction terms were for improving overall estimation of intersection-specific outcome prevalence, rather than inference regarding the size or statistical significance of interaction effects. Finally, main effects regression was included as a non-intersectional method for comparison. Poisson regression, modified to use robust variance estimation, was used for all single-level regressions, because it produces risk ratio estimates for both rare and common binary outcomes . The MAD for each method was calculated as,n was 192 representing 192 possible intersections, i, Pi was the true prevalence of the outcome Y=1 for intersection i, and p was the prevalence of the outcome in the entire sample. The true prevalence of the outcome was known by using the outcome generation formula, but not running it through the binary sampling function. Because estimations were for each intersection rather than each individual, accuracy is measured at the subpopulation (intersection) level, rather than the individual level. This emphasizes equal performance across intersections, rather than favouring better performance for more-populated intersections. A MAD of 0 is only achieved by perfect estimation of the prevalence within each intersectional group.such that 2.3.2In population health analyses, even those rooted in theory, there may be more social identity/position variables than can be incorporated into an intersectional analysis. While decision trees do not produce traditional outputs such as variance estimates, they can assist in variable selection to identify relevant variables to include in subsequent analyses that do . Variabl2.4We conducted two post-hoc sensitivity analyses assessing the performance of MAIHDA, main effects regression, correctly-specified regression, and cross-classification, at sample sizes 2000 and 5000. First, we simulated four scenarios with a higher outcome prevalence of approximately 50%: categorical inputs, mixed inputs, categorical inputs with larger effect sizes only for interaction effects, mixed inputs with larger effect sizes only for the interaction effects. Second, we assessed the bias and variance of intersection-specific prevalence estimates, using one simulation scenario for the rare, common, and 50% prevalence outcomes with categorical inputs, with one set of effect sizes for each scenario, iterated 1000 times. Bias  and variance were estimated for each of the 192 intersections, and results present the median, minimum and maximum values for the 192 intersections. Further description of simulation procedures for sensitivity analyses are provided in Web 2.5We used NHANES data to demonstrate and compare differences in results between the methods. Using a multistage probability sample, NHANES is designed to represent the U.S. non-institutionalized population . Sixty i33.1We considered the feasibility of the different analysis methods for binary outcomes. While other analyses ran smoothly, the saturated regression did not always converge at smaller sample sizes . Results3.23.3P < 0.05 is effective at maintaining a low detection of false positives in the mixed inputs scenario, where X6 is continuous, but not in the categorical inputs scenario, where X6 is a three-category variable.3.4Sensitivity analyses present more detailed information on small samples and the performance of main effects analysis. Results from the first set of simulation analyses evaluating method estimation accuracy at 50% outcome prevalence show correctly-specified regression as equivalent or slightly more accurate than main effects regression, and MAIHDA in some scenarios performing better than main effects, correctly-specified regression, and cross-classification A-D. The 4Challenges in assessing binary outcomes are amplified when assessing outcomes for high-dimensional intersections. At smaller sample sizes our results differed substantially from those in our earlier evaluation of continuous outcomes  with a kWe modeled 192 simulated intersections, a number typical of published studies using decision tree or MAIHDA methods; researchers studying fewer intersections may want to adjust our “larger” and “smaller” samples accordingly. For example, N=50,000 corresponded to a mean intersection size of N=260, while N=5000 corresponded to 26. Our small-sample analyses intentionally pushed limits for demonstration purposes, and a saturated regression model assessing 192 intersections at N=2000 is not a reasonable expectation.In larger samples, all intersectional methods but CART were accurate estimators and outperformed non-intersectional main effects. However, at small sample sizes the main effects analysis performed better as an estimator than most methods, except for MAIHDA. Additionally, in smaller samples the traditional methods of cross-classification and saturated regression were poor estimators, and saturated regression was often not feasible given convergence issues.Our sensitivity analysis assessing the bias and variance of the intersection estimates aimed to explain why at small sample sizes almost all the intersectional methods performed worse than the main effects approach, which given our data scenarios is mis-specified and incompatible with intersectionality. The overall estimation accuracy of a method is attributable to both the bias and variance of the estimates . A main MAIHDA was the only applicable intersectional method  that performed equivalent to or better than a main effects analysis, at smaller and larger sample sizes. Additionally, the two methods were similar in bias-to-variance tradeoff. We suggest that this is because MAIHDA models are comprised of a main effects regression , with additional residual estimates to account for effects within each intersection . The resMAIHDA accuracy surpasses main effects at larger and more plausible sample sizes, with decreased number of intersections, or with a higher outcome prevalence, due to reductions in estimation bias compared to main effects regression. Our NHANES analysis was a real-world example with a high prevalence outcome, assessing a realistic number of intersections given the sample size, and demonstrated the non-equivalence of MAIHDA and main effects regression results. Intersection-specific prevalence estimates differed by up to 10% between MAIHDA and main effects. We recommend MAIHDA for estimating high-dimensional intersectional outcomes, especially when the sample size is small relative to the number of intersections such as in the NHANES example. It theoretically accounts for intersectionality (resulting in reduced bias with increasing sample size) and estimates fewer parameters .Previous studies assessing classification by random forest have concluded that it performs similarly or more accurately than logistic regression, CART, and CHAID . HoweverIn both the simulation and NHANES example, CART under-identified relevant variables representing heterogeneities in the data. These results are important given CART's use in intersectionality research as the decision tree method of choice . For CTrTo note, variable relevance as referenced in this study is purely quantitative. Our prior study detailed how sequential variable selection by decision tree methods may be a useful tool for variable selection, to narrow down a list of potential variables to be used in other analyses such as MAIHDA or regression with interactions . HoweverThis study compared multiple methods for describing high-dimensional intersections with binary outcomes, and found MAIHDA most accurate for intersection estimation, especially at smaller sample sizes. We acknowledge that our results may be limited by the simulation data generation process. Additionally, alternative applications of these methods, such as penalized regression with interactions, may improve their performance for estimation or variable selection . Future Canadian Institutes of Health Research Sex and Gender Science Chair to GB [GSB-171372].This work was supported by an Ontario Graduate Student Scholarship to MM and by a Research ethics approval was not required for this study, which uses only simulated and publicly downloadable data.Mayuri Mahendran: Conceptualization, Methodology, Software, Formal analysis, Data curation, Visualization, Writing – original draft, Writing – review & editing. Daniel Lizotte: Conceptualization, Methodology, Writing – review & editing. Greta R. Bauer: Conceptualization, Methodology, Writing – review & editing, Funding acquisition.The authors declare no conflicts of interest.


The high-throughput gene expression data generated from recent single-cell RNA sequencing (scRNA-seq) and parallel single-cell reverse transcription quantitative real-time PCR (scRT-qPCR) technologies enable biologists to study the function of transcriptome at the level of individual cells. Compared with bulk RNA-seq and RT-qPCR gene expression data, single-cell data show notable distinct features, including excessive zero expression values, high variability, and clustered design. We propose to model single-cell high-throughput gene expression data using a two-part mixed model, which not only adequately accounts for the aforementioned features of single-cell expression data but also provides the flexibility of adjusting for covariates. An efficient computational algorithm, automatic differentiation, is used for estimating the model parameters. Compared with existing methods, our approach shows improved power for detecting differential expressed genes in single-cell high-throughput gene expression data. Recently, single-cell high-throughput gene expression profiling technologies, including single-cell RNA sequencing (scRNA-seq) and parallel single-cell single-cell reverse transcription quantitative real-time PCR (scRT-qPCR), have enabled researchers to examine mRNA expression at the resolution of individual cell level, which provide further biological insights of the transcriptomes and functional genomics ,2,3,4. CExcessive zero expression values. The proportions of genes with observed zero expression values in single-cell gene expression data are much larger than bulk RNA-seq or RT-qPCR data . A similar yet different two-part model without random effects is proposed to model the scRNA-seq data in a recent paper, which is named MAST  distribution , which are shown in p-values. As a general rule, if the sample size is large, then the p-values from the p-values for those top genes.If the type I error rate is correctly controlled, the ial part  or the jial part . On the p-values   . This isp-value evaluated by the empirical Satterthwaite method, an NBMM with the library size as an offset term implemented in [A recent study compared various methods for differential expression analysis in scRNA-seq using a number of scRNA-seq datasets with matched bulk RNA-seq in the same purified cell types as reference . This stented in  and a psFollowing , we firsNext, we apply the TMM model to another scRNA-seq dataset and compare it with MAST and TM. This dataset is published in , which cThe goal of our analysis is to identify differentially expressed genes between the adult and fetal brains. We fit TMM with the following two covariates as fixed effects:p-values for saving the computational time. We also fit the MAST and TM models, where X1 and X2 are included as covariates in these two models. Multiple comparison adjustment is performed using the Benjamini–Hochberg FDR procedure [The 12 subjects are treated as clusters, which are included as random effects in the model. The likelihood ratio test is used to test the individual Gaussian part and binomial part and also to jointly test the two parts, and the rocedure .p-values and FDR for the top 20 differentially expressed genes . We can see the results from the three models show considerable overlaps  for differential expression analysis with single-cell gene expression data. This model not only adequately accounts for the distinct features of single-cell expression data, including extra zero expression values, high variability, and clustered design, but also provides the flexibility of adjusting for covariates. Since scRNA-seq is still a developing and growing technology, it brings more challenges in data analysis than bulk RNA-seq. These challenges can be technical , and alp-value estimation of TMM should be further optimized, which is also common for many mixed-effect models [More recently, several new models and approaches have been proposed for the DE analysis on scRNA-seq data. As studied in  and Sectt models . On a set models  and Sect


The increasing number of scRNA-seq data emphasizes the need for integrative analysis to interpret similarities and differences between single-cell samples. Although different batch effect removal methods have been developed, none are suitable for heterogeneous single-cell samples coming from multiple biological conditions. We propose a method, scINSIGHT, to learn coordinated gene expression patterns that are common among, or specific to, different biological conditions, and identify cellular identities and processes across single-cell samples. We compare scINSIGHT with state-of-the-art methods using simulated and real data, which demonstrate its improved performance. Our results show the applicability of scINSIGHT in diverse biomedical and clinical problems.The online version contains supplementary material available at (10.1186/s13059-022-02649-3). Single-cell RNA sequencing (scRNA-seq) technologies enable gene expression measurement at a single-cell resolution and have opened a new frontier to understand animal development, physiology, and disease-associated molecular mechanisms –5. RapidA fundamental goal in integrative scRNA-seq data analysis is to jointly define cell clusters, obtain their functional interpretation and annotation, and identify differentially activated biological pathways in distinct cell types and biological conditions. However, a key challenge for achieving this goal is the heterogeneity present in single-cell gene expression data. As expression data from different sources are associated with various types of technical effects , expressTo help remove the batch effects emerging from scRNA-seq data generated by different sequencing platforms or library-preparation protocols, several batch correction methods, including mnnCorrect , BBKNN , and BEEINterpreting single cell gene expresSIon from bioloGically Heterogeneous daTa) to jointly model and interpret gene expression patterns in single-cell samples from biologically heterogeneous sources. scINSIGHT uses a new model based on non-negative matrix factorization (NMF) K×n is the membership matrix of K common gene modules, and it is shared by all L samples ; Eℓ is the residual matrix for sample ℓ. In addition, we require all the matrices to be non-negative. The above model allows scINSIGHT to detect common gene modules by borrowing information across samples and condition-specific gene modules by borrowing information among samples of the same condition.where In order to solve the scINSIGHT model and identify the common and condition-specific modules, we formulate a minimization problem with an objective function in the following form A denotes the kth row of matrix A, ∥·∥F represents the Frobenius norm, ∥·∥1 represents the sum of all elements of a matrix, and ∥·∥2 represents the L2 norm of a vector. The first term in the objection function aims to minimize the differences between observed and reconstructed gene expression matrices; the second term serves as a regularization term to control the scale of condition-specific components; the third term controls the similarity between condition-specific gene modules of different conditions. In the first and second terms, the Frobenius norms are scaled by cell numbers in each sample to prevent the results from being dominated by large samples.where To obtain the optimal solutions for problem , we haveℓ , To highlight the difference between scINSIGHT and the iNMF model , we alsoℓ; [Hℓ]K×n is the membership matrix for dataset-specific gene modules in sample ℓ; [V]K×n is the membership matrix for shared gene modules. To solve the model, the objective function is formulated as: where Unlike the scINSIGHT model, the above formulation has the following limitations: (1) it cannot account for the assignment of biological conditions, (2) it assumes that there exists the same number of shared gene modules and dataset-specific gene modules, and (3) it cannot decompose the original gene expression data into the expression of shared and dataset-specific gene modules. These limitations are resolved by scINSIGHT to better jointly analyze single-cell samples from different biological conditions.L read or UMI count matrices, we use the following three steps to obtain the L processed gene expression matrices. First, the count matrices are normalized by cell library sizes. We scale the counts such that each cell has a total of 105 reads or UMIs. Second, we perform log-transformation on the scaled values. Third, we remove genes that exhibit low variation among individual cells to increase the signal-to-noise ratio in gene expression data. The highly variable genes are selected using the same approach as described in Seurat [Given the n Seurat . If usern Seurat , the scIL2 norms all equal 1. With a slight abuse of notation, we still use The row vectors in ℓ and ℓ′ , from sample ℓ′ we find the nc nearest neighbors for each cell in sample ℓ. When searching for nearest neighbors, the cell-cell distance is calculated using the Euclidean distance between normalized module expression in nc=20.Given every pair of samples L samples as the nodes. Cell i in sample ℓ and cell i′ in sample ℓ′ are connected in the graph if and only if they are mutual nearest neighbors.A mutual nearest neighbor graph is constructed using cells in all The Louvain method  is used To produce visualization that better reflects the clustering result, we align the module expression within each cell cluster using quantile normalization. We denote the quantile normalized module expression as As the NMF framework has an inherent clustering property, we can use the expression of common gene modules to cluster single cells from different samples, and use the membership of common gene modules to interpret cellular identities. However, since minimizing the objective function in problem  does not methods , 27, andK, the number of common gene modules, and then select λ1 and λ2, the two regularization parameters in the scINSIGHT model  in our analysis to facilitate interpretation of factorization results. To select K among a set of candidate values, we define a stability score inspired by consensus clustering [K, we run our algorithm B (defaults to 5) times with different initializations and λ1=λ2=0.01. We denote the clustering result from each run as a binary matrix Cij(b)=1 indicating that cells i and j belong to the same cluster. We then calculate a consensus matrix IM×M−C and the cophenetic distance matrix  [K value. To identify the optimal value of K for accurate and robust results, we first find the three candidate values that lead to the largest stability scores, and then select the middle value among the three candidates.We propose a heuristic method to first select ustering . For eveng on C) . A largeλ1 and λ2, fixing K to the selected value. In our simulation study, we found that λ1 has a much greater effect on scINSIGHT’s results than λ2, so we set λ1=λ2 to simplify the computation. We consider five candidate values, {0.001,0.01,0.1,1,10}, and select the optimal value based on a specificity score. The specificity score indicates how well the condition-specific gene modules capture specific gene expression patterns in each condition. For the kth gene module in Next, we select the regularization parameters ℓi to denote the sample index of cell Ni represents the k-nearest neighbor set of cell i. For observed (unintegrated) data, Ni is defined based on cell-cell distance calculated using the first 20 PCs. For scINSIGHT, Ni is based on cellular distances calculated using Ni is also determined based on the gene expression data after integration. For each set of Ni’s, the corresponding integration score is defined as The integration score aims to quantify how well the (unintegrated or integrated) data removes sample-specific technical effects. We consider cells from all samples and use The Silhouette score was calculated using the R package cluster and the adjusted Rand index was calculated using the R package mclust.To better benchmark the performance of scINSIGHT, we used the scDesign method  to simulAs scDesign learns gene expression parameters from real data to simulate new synthetic cells, we used the following approach to incorporate biological heterogeneity into the simulated data. First, the expression parameters corresponding to the cell type effect were learned from six immune cell types in a dataset of peripheral blood mononuclear cells . Second,Kj=2 for all datasets. The selected K was 11 for variant 2 and 13 for the original simulation and variants 1 and 3 . The regularization values were selected as λ1=λ2=0.001 for variant 1 and λ1=λ2=0.01 for the other datasets. LIGER was applied with nrep = 5 and k = 11; the other parameters were set as the default values. Seurat, Harmony, and Scanorama were applied with default parameters. scMerge was applied with kmeansK=rep in variant 1 and kmeansK=rep in all other cases. We let scMerge identify its own highly variable genes, and the other parameters were set as the default values. Follow its tutorial, scGen was applied with max_epochs=100 and batch_size=32, early_stopping=True, and early_stopping_patience=25, and the other parameters were set as the default values. For clustering, scINSIGHT, Seurat, and LIGER have their own clustering methods, Harmony used Seurat’s clustering in its tutorial, and Scanorama used the Leiden algorithm [For analysis based on the observed (unintegrated) data, we calculated the first 20 principal components (PCs) after library size normalization and log-transformation, and used the cells’ scores on the PCs to calculate tSNE coordinates and perform Louvain clustering . scINSIGlgorithm  in its tlgorithm  to perfoThe scINSIGHT, LIGER, Seurat, Harmony, scMerge, scGen, and Scanorama methods were applied using R package scINSIGHT (v0.1.3), R package rliger (v0.5.0), R package Seurat (v3.2.3), R package harmony (v0.1.0), R package scMerge (v1.6.0), Python package scgen (v2.0.0), and Python package Scanorama (v1.7.1), respectively.et al [Kj=2 and the best K was 9 . The regularization values were selected as λ1=λ2=1. LIGER was applied with nrep=5 and k=5 as larger values would lead to errors, and the other parameters were set as the default values. Seurat, Harmony, and Scanorama were applied with default parameters. scGen was applied with the same parameters as described for the simulated data. For analysis with highly variable genes, 2000 genes were selected using Seurat. The methods were applied in the same way as described above. scMerge encountered errors in both analyses.For the melanoma dataset , we procet al . The norKj=2 and the best K was 13 . The regularization values were selected as λ1=λ2=1. LIGER was applied with nrep=5 and k=13, and the other parameters were set as the default values. Seurat, Harmony, and Scanorama were applied with default parameters. scGen was applied with the same parameters as described for the simulated data. scMerge was applied with kmeansK=rep and internally identified highly variable genes, as it encountered an error when selected gene features were provided. The other parameters were set as the default values.For the COVID-19 dataset , we procKj=2 and the best K was 11 . The regularization values were selected as λ1=λ2=0.1. LIGER was applied with nrep = 5 and k = 11. The other parameters were set as the default values. Seurat, Harmony, and Scanorama were applied with default parameters. scGen was applied with the same parameters as described for the simulated data. scMerge was applied with kmeansK=c and internally identified highly variable genes. The other parameters were set as the default values.For the wound dermis dataset , we procKj=2 and the best K was 26 . The regularization values were selected as λ1=λ2=1. LIGER was applied with nrep = 3 and k = 20. The other parameters were set as the default values. Seurat, Harmony, and Scanorama were applied with default parameters. scGen was applied with the same parameters as described for the simulated data, except for batch_size=128. scMerge was applied with kmeansK=c, svd_k=20, and internally identified highly variable genes. The other parameters were set as the default values.For the mouse retina dataset , we procAdditional file 1 Supplementary figures S1–S21 and table S1.Additional file 2 Supplementary methods.Additional file 3 Review history.