X-ray fluorescence spectroscopy is a non-destructive technique employed for elemental analysis of a wide range of materials. Its advantages are especially valued in archaeometry, where portable instruments are available. Considering ancient glass, such instruments allow for the detection of some major, minor, and trace elements linked to the deliberate addition of specific components or to impurities in the raw materials of the glass batch. Besides some undoubted advantages, portable XRF (p-XRF) has some limitations that are addressed in this study. The performance assessment of four different p-XRF units and the reconciling of their output were conducted. The results show the limitations in cross-referencing the data obtained from each unit and suggest procedures to overcome the issues. The p-XRF units were tested on the set of Corning reference glasses and on a small set of archaeological glasses with known composition. The compatibility of the output was assessed using multivariate statistical tools. Such a workflow allows us to consider data from multiple sources in the same frame of reference. XRF analyses of archaeological materials conducted with portable instruments (p-XRF) are increasingly popular within the archaeometry community and have a history of development that spans decades ,2,3,4,5.Unfortunately, p-XRF has several drawbacks for the investigation of ancient glass. The most evident is a significant limitation in providing accurate information for “light” elements (Z < 14), which may constitute a large part of the total mass fraction in the glass matrix. This may inhibit a characterization of the fluxing agents used in glass making, which is one of the first research questions to be faced when studying archaeological glass. To overcome the issue, the p-XRF units can be equipped with helium-flow setups or with small chambers for vacuum analyses to reduce the absorption of X-rays by Ar from the atmosphere and enhance the possibility of detecting the weak XRF signals from Na and Mg ,12,13.The above-mentioned issues combine with those inherent to the general XRF analytical approach. Characteristic fluorescence lines of different elements may overlap, with typical examples relevant for glass studies being the interference of Fe Kβ and Co Kα, or that of Pb Lα and As Kα. These phenomena limit the accuracy of the quantification and may prevent elements from being detected if they are present in minor quantities . MoreoveAdjustment of the acquisition parameters can improve the quality of the data and might expand the number of applications for p-XRF as well as mark their limits ,16,17.Under the p-XRF label, we can find many types of spectrometers, with a variety of features . MoreoveSome users explore the spectra in a purely qualitative way: as long as energies are associated with elements and the position and height of the peaks correspond to the respective element’s presence/concentration, the comparison of the relative heights of the peaks provides information on the qualitative composition of the examined samples , though The so-called “semi-quantitative” approach is based on different workflows. This approach does not allow direct comparison of the results with external data. Therefore, results obtained with different instruments cannot be directly compared. Nevertheless, it enables the possibility of distinguishing groups of artifacts within the obtained dataset. The basic assumption in this approach is that the collected data are similar but may be biased from an accepted nominal value. Using the definitions given in ISO 5725-1 , we shal2 values to compensate for the difference in total composition to 100% [Some operators use face values of factory calibration provided by the p-XRF unit . They ca to 100% . Regardi to 100%  or estim to 100% . Again, Accurate concentrations can be obtained when standard or reference materials are used to calibrate the spectra. This approach yields the highest quality results. The output of the process should in principle be comparable with results obtained from other p-XRF units or other types of analytical equipment. Some producers of p-XRF units prepare built-in calibrations that can be used when analysing archaeological glass or prepare them by request, using the Fundamental Parameters (FP) method or the Compton normalization ,33. The In general, the above-mentioned examples demonstrate that p-XRF is widely used in the analysis of archaeological glass, although most of the matrix remains undetected. Despite extensive applications to ancient glass, many researchers rely on other methods for quantitative compositional information, reducing p-XRF to a tool for preliminary analysis and using the p-XRF values in a strictly qualitative or semi-quantitative way.Recently, the need for implementing integrated approaches between chemistry and archaeology has significantly grown within the entities that are responsible for studying and protecting cultural heritage. As a consequence, many relevant archaeological museums and research institutions own analytical equipment, with p-XRF units being among the most popular, as there are many research questions or purposes that can be answered or fulfilled by this technique. Consequently, an extended archaeometric investigation may face the issue of having different p-XRF units available for analysing multitudes of archaeological objects in different institutions. As the collected data needs to be compared to produce a new knowledge on the investigated collections, we must exclude any bias introduced by the use of different equipment.There have been several papers reporting a comparison of the results obtained with different p-XRF units. They are mostly related to environmental samples and metals, whereas, to the best of the authors’ knowledge, none deals specifically with archaeological glass. Some of these papers highlight specific issues that cannot be overlooked while assessing the reproducibility of data from different instruments. In particular, it has been demonstrated how the data treatment can influence the final output  and thatThe present work aims at highlighting the main limitations in reproducibility of p-XRF analyses performed with units of different configurations , thus shedding light on the actual possibility to face the main archaeological questions for glass with a p-XRF approach if different instruments are employed. The study is preliminary to an analytical campaign that will be conducted in situ on large collections of Iron Age glasses held in several archaeological museums in Italy. The focus is set on quantification using FP combined with calibration using the reference glass approach, which would allow results to be exploited throughout a specific research project and beyond.Certified Reference Materials (CRM) for glass that are offered on the market provide a wide selection of elements with certified or informational concentration values. Yet, the resemblance of the compositional range of ancient glasses with these (modern) reference materials remains poor. Therefore, for this study, we selected a set of three reference glasses provided by the Corning Museum of Glass (CMOG). The CMOG reference glasses were developed to represent different compositional groups of historic glass, and they feature the elements that play a role in glass (de)coloring and opacification, as well as some common impurities that enter the glass batch with the silica source or with other glass components. Each glass piece is named with letters for different glass types. CMOG A and CMOG B are soda-lime-silica glasses that represent typical soda plant ash (A) and natural evaporites (B) varieties of Mediterranean production glasses from the Bronze Age to the Medieval times . CMOG D The compositions of CMOG glasses have been established through an inter-laboratory study, and since the 1970s, the glasses have been tested and their concentration revised according to the new results ,47. The In addition to the CMOG reference glasses, five pieces of archaeological glass  were also included in the investigated glass set and their composition is reported in 2 area and 2–3 mm thick). In addition, they were polished using diamond pastes of 6 and 1 µm grain size.In this work, VA08 and VA27 were analysed as chunks of glass, while VA70 and the CB samples were analysed as cross-sections. This allows us to evaluate the role of the shape of the surface on the quantification. Successful quantification of irregularly shaped samples will substantiate the quantification on future analyses to be performed non-invasively on archaeological glasses. The CMOG reference glasses underwent polishing to ensure the correct geometry of the acquisitions. Grinding paper of 500, 1200, and 2400 grit was used on flat pieces (about 1 cmXGLab ELIO (hereafter ELIO). The commercial unit is produced by XGLab S.R.L. ;Unisantis XMF-104 (hereafter Unisantis). The commercial unit is equipped with polycapillary optics and produced by Unistantis . Regarding the size and configuration of the instrument, this is the only one with limited sample size analysis due to the dimensions of the sample chamber and can therefore be included in the category of transportable X-ray spectrometers ;Frankie (hereafter Frankie). An ad hoc unit with policapillary optics, developed by the Italian National Institute of Nuclear Physics-National Laboratories of Frascati ((INFN-LNF) Italy);NITON XL3T-900 GOLDD (hereafter NITON). The commercial unit is by Thermo Fisher Scientific . This instrument acquires a series of spectra in different energy ranges to enhance the signal in specific parts of the spectrum.Four p-XRF units were included in this study:The most relevant technical features of the equipment and the acquisition parameters used in this study are reported in On each CMOG reference glass, 10 spectra (ELIO and NITON units) or 3 spectra (Unisantis and Frankie units) on different spots were acquired. For the archaeological references, 3 measures on different spots were acquired for each p-XRF unit, except for Frankie, for which only one acquisition was performed on each archaeological reference.The obtained spectra were converted to .spc, .spe, or .mca before fitting with the PyMCA software . The conThe Limit of Quantification (LOQ)—the lowest amount of analyte that can be quantified with a given uncertainty—for each element’s concentration was estimated by using the value of the gross area B under the relative peak . The LOQ was set to be equal to B + 10σB. This corresponds to an uncertainty of ±30% in the measured value at the 99% confidence level . This vaThe relative standard deviation (RSD) of tri- and decaplicates was used to calculate measurement precision. To check the accuracy, the values were converted to elemental weight % and compared to the nominal concentration . In ordeIn order to ensure that all the data points are a real representation of the concentration in each of the references, the Limits of Quantification (LOQs) were calculated. The automated fitting of the spectra assigned values to the selected elements even when there were no pronounced peaks in the spectra. Therefore, LOQs are needed to filter out the data that were not significant for the quantification process. According to this procedure, the data that were equal or below the LOQ were removed before any precision and accuracy checks.There are several ways of establishing LOQs ,56,57. ALOQs in XRF analyses usually depend on the matrix composition and the element’s atomic number. It is lower for elements with Z ranging between 25 and 40, and higher for lighter ones. The features of the source of radiation are also important to determine LOQ, as source peaks due to Compton and Rayleigh scattering might interfere with the characteristic energy peaks of the analysed elements, raising the LOQs. The same effect is observed when the characteristic energies of the elements in the analysed sample overlap in the detected spectrum.Unfortunately, for many minor elements, the number of data is equal to or less than five. Therefore, calculated uncertainty may not reflect the true value for these elements. Larger datasets are available for K, Ca, Mn, Fe, Cu, Sr, and Pb. Some elements  may have only one data point considered by a single instrument (there might be more data for another instrument). Data on fewer represented elements will reflect poorer precision, and this will be considered in the further discussion of the data. Furthermore, because of their heteroscedastic nature, quantitative data near the LOQ should be handled with caution .The variability among the repeated measures was estimated as the Relative Standard Deviation (RSD). Mean RSD values per unit of equipment, together with their respective maximum values, are shown in It can be noticed that, generally, the relative dispersion of the data is less than 20% for most of the elements across all the XRF units. The polished cross-sections of CMOG glasses do not differ significantly from the sections and the chunks of archaeological references. The only exceptions are some of the Unisantis acquisitions . One of the reasons for high RSD values might be the proximity of the measured values to the respective LOQ (as in the case of ELIO CB65 Sn (68%RSD) and CMOG B Co (36%RSD) values) since the integration of smaller peaks yields a greater error on the fitted peak area estimation. The Frankie unit data shows the tendency to increased RSDs for Sn Sb, Ba and Pb. This might be due to the lower sensitivity of the spectrum after the peaks of the W anode . The precision found in our set of data is normally higher than the instrumental precision obtained with other spectroscopic techniques . NeverthThe FP method is a well-established quantification algorithm that can provide accurate quantitative results, especially when used in conjunction with matrix-matching standards ,36. ValuThe 2σ error values for all the XRF units  are reported for each element. These values were calculated as two standard deviations of the absolute values of relative error for each data point. Many of the data points are within ±20% of the nominal values, but a significant number of them fall within ±50% of error. This quite broad range renders this data unacceptable for many applications. The accuracy values of each XRF unit differ for each element. It is remarkable that the Frankie unit values for the heaviest elements exceed the 100% limit. It should be kept in mind that Frankie unit values had lower precision for Sn, Sb, Ba, and Pb . The UniIt was decided to further look into the data in order to find the best way of improving the FP quantification of the datasets. By observing the data, it was noticed that some of the elements’ values were systematically under or overestimated within a single instrument’s dataset. This points out the limits of the FP model implemented through PyMCA: it accounts for a large number of variables, making the task of their optimization ever more laborious when approaching perfection. Instead, it was attempted to offset these variations by using the average ratios of nominal to acquired values of CMOG reference glasses to obtain the coefficients for normalization of all the values in the dataset. The formula for this type of correction is the following:The division of CMOG and archaeological reference groups was done to check the usefulness of such a correction. Since the coefficients were calculated on the CMOG set of three glasses, it was expected that the average recovery for the corrected values would be close to 100%. In such a case, one should look at the improvement of the data of the archaeological reference group. Indeed, corrected CMOG glasses’ recoveries for all instruments tend to align very near the 100% mark (red circles). As for the corrected archaeological reference glasses group, the resulting values mostly improve when compared to the values of initial quantification, but it is not always the case. For the Elio instrument, for example, an improvement in Cu recovery is “compensated” by a larger bias for Zn. However, most of the corrected values lie in the 20% interval from the nominal values (black circles) that was arbitrarily decided to be an acceptable deviation from the true values. Some corrected values are less than 10% off the nominal ones. Some elements do not have representation in the archaeological reference group of some XRF units . For such elements, it is impossible to assess the effect of correction. Hence, the data on these elements should not be considered as quantitative, yet the rest of the elements that are represented in the archaeological reference group do obtain better values after this type of correction, so it can be considered a useful transformation on the way to improving the overall accuracy.Another type of empirical coefficient correction that was tested in this study is the application of linear regression coefficients. This approach was described elsewhere . Here weIn order to build the final dataset, we adopted a mixed approach: with less than four data points for one XRF unit, data were corrected by the application of ratio coefficients; when four (or more) data points were available, we used linear regression correction. Concentration values for elements that were obtained by the application of ratio coefficients lack adequate uncertainty values. Those elements are presented in concentrations approaching the LOQ, therefore high uncertainty is expected. It is worth noting that no systematic bias from the nominal values emerged for VA08 and VA07, which were analysed as chunks. Therefore, no significant bias determined by surface morphology or glass alteration emerged from the data.In order to check if the obtained reproducibility among the different types of equipment supports the purpose of investigating a large set of archaeological glass with different p-XRF units, we employed multivariate analyses. Principal Component Analyses (PCA) and Hierarchical Cluster Analyses (HCA) were used to check if the compositions of the same set of samples obtained from different p-XRF units will result in a proper representation (PCA) or classification (HCA) of the samples. In archaeometry, the compositions of glass are normally used to highlight similarities or differences among archaeological samples, therefore it is fundamental to assess if the use of different p-XRF units will prevent the possibility to draw any conclusions of archaeological relevance based on the dataset.PCA and HCA are widely used tools in data exploration. PCA allows for reduction of dimensionality with a subsequent representation of all the data in a single space with two or three coordinates, whereas HCA organizes samples into groups based on how closely associated their compositions are and represents them in one single graph that highlights the relationships within the dataset ,60.In this particular instance, we employed both PCA and HCA to check if the data were from the same material, but different XRF units will be clustered together. Such group arrangements will support evidence of reproducibility for these datasets. On the other hand, distinguishing the reference samples will demonstrate the ability of quantitative p-XRF data to serve as a basis for archaeometric discussion on glass technology and, perhaps, even provenance. In order to perform the algorithm, the data corrected by linear regression or ratio coefficients . The scores and loadings plots of PC1 against PC2 and PC3 are reported in Clusters were calculated using Euclidean distances on SNV pre-treated data and were based on at least three valid measures. It was decided to present eight clusters . Both The investigation highlights some limitations and pitfalls of the quantitative p-XRF method for archaeological glass analyses, which remains a challenging task not only in terms of quantification strategy for adaptation to a specific matrix in p-XRF analyses but also due to the intrinsic complexity of the interpretation of compositional data for ancient glass that results from the great variety of components in the glass batch, and therefore in a large quantitative variability.On the other hand, by focusing on the analytical part, we demonstrated that, despite their hardware differences, four p-XRF units produced quite similar results provided that strictly controlled data handling is performed. Unfortunately, the set of certified reference materials available for elemental analyses of archaeological glass is very limited, and therefore it has not been possible to gather a large number of data points across all the elements of interest. Nevertheless, the results we presented in this work demonstrate that it is possible to achieve good-for-purpose reproducibility between different p-XRF units by following fairly practical procedures: ratio coefficients correction can be used on FP data when a low number of reference standards are available (less than four), whereas the data can be corrected by linear regression correction when four (or more) reference data points are available. This procedure, besides increasing the potential pool of quantitative compositional data on ancient glass, allows us to state that the compositions of glass determined through different p-XRF equipment can be actually used to highlight similarities or differences among archaeological samples , and that the data obtained through different p-XRF units will allow us to draw conclusions of archaeological relevance based on the glass composition determined in situ.


Single-cell omics technology is rapidly developing to measure the epigenome, genome, and transcriptome across a range of cell types. However, it is still challenging to integrate omics data from different modalities. Here, we propose a variation of the Siamese neural network framework called MinNet, which is trained to integrate multi-omics data on the single-cell resolution by using graph-based contrastive loss.By training the model and testing it on several benchmark datasets, we showed its accuracy and generalizability in integrating scRNA-seq with scATAC-seq, and scRNA-seq with epitope data. Further evaluation demonstrated our model's unique ability to remove the batch effect, a common problem in actual practice. To show how the integration impacts downstream analysis, we established model-based smoothing and cis-regulatory element-inferring method and validated it with external pcHi-C evidence. Finally, we applied the framework to a COVID-19 dataset to bolster the original work with integration-based analysis, showing its necessity in single-cell multi-omics research.MinNet is a novel deep-learning framework for single-cell multi-omics sequencing data integration. It ranked top among other methods in benchmarking and is especially suitable for integrating datasets with batch and biological variances. With the single-cell resolution integration results, analysis of the interplay between genome and transcriptome can be done to help researchers understand their data and question.The online version contains supplementary material available at 10.1186/s12859-022-05126-7. Diseases like cancer, heart disease, and Alzheimer's are highly complex . Unlike Two main strategies have been proposed to integrate different omics data modalities: the experimental approach –11, whicThe unpaired cell effect refers to the problem created when different omics data are sequenced from different batches of cells so there is no correspondence available to link modalities. To solve this problem, researchers typically project all the cells into a shared latent space, from which unpaired cells can be aligned to share all omics data. Seurat  applies The second challenge arises from both modality and batch effects during integration. Most algorithms remove modality effect when projecting and aligning cells but to our knowledge, batch effect is not especially considered in these integration models. The Siamese neural network  has beenTherefore, we introduce here a new Siamese neural network design with a graph-based loss to integrate multi-omics datasets at single-cell resolution. Trained to integrate cells from different modalities while removing the potential batch effect, our model outperforms other algorithms in multiple benchmarking datasets. Furthermore, to show the integration’s impact on downstream analysis, we developed a model-based smoothing and cis-regulatory element-inferring approach and demonstrated its efficacy by validating in 10X Multiome datasets. Finally, we applied the framework and analysis to a published COVID-19 dataset, improving the original work by adding integrated, multi-modal analysis.As with other state-of-the-art integration methods, the MinNet framework follows the statistical concept of integrating omics data: Cells from different modalities are projected into the same latent space that captures the shared variance in all omics data. To generate this co-embedding space, the Siamese neural network simultaneously receives as inputs one cell from modality 1  and another from modality 2  and projects them into the same n-dimensional vectors using the encoder. To ensure this n-dimensional vector space is a good representation of the shared main biological variance, two losses are applied following the encoder.The first and most important is the contrastive loss . Here, wm is a constant for all negative pairs during Siamese neural network training. However, cells from different cell types are more diverse than those from the same cell type. Thus, to maintain these differences in the co-embedding space, we designed mas a flexible value depending on how much the cell pairs differ. . Intuitively, cells that differ more pose higher variance, so a larger margin is assigned to separate them in the space. In contrast, similar cells pose little variance, so a small margin value is assigned  data and Cite-seq BMMC data, we evaluated our method and compared it with existing ones, including GLUE , bindSC After applying all algorithms to the benchmarking datasets, we evaluated all final integration results  score  was measFor broad usability, a supervised model must be generalizable. Our model’s success in integrating untouched donor datasets and untouched sequencing site datasets already demonstrated its generalizability, but we also wanted to test it with other tissues. First, we undertook the same evaluation using the 10X Multiome peripheral blood mononuclear cell (PBMC) dataset . Based oHowever, this generalizability was limited to similar tissues, such as BMMC and PBMC. We also applied the trained model to the 10X Multiome human brain dataset , an entiTo distinguish between batch variance and biological variance, we trained our model with multiple batches from different donors and sequencing sites. While the training input was normalized data without batch correction, the contrastive loss was based on the graph after batch correction by ComBat implemented in Scanpy . With thTo test the performance of batch effect removal, we generated three more testing scenarios with the available benchmark datasets that represent real case practice problems. The first scenario tested all algorithms’ performance when both scRNA-seq and scATAC-seq experiments are performed independently on identical batches. The second and third scenarios tested the integration performance of scRNA-seq and scATAC-seq datasets profiled from different batches. In all three cases, we compared our model with those mentioned above.The silhouette score and label transfer accuracy were chosen for evaluation since cells were different in the second and third cases and FOSCTTM score is not feasible. Figure This evaluation is especially important as it mimics real-world practice in which researchers use independent profiling from different batches or even from independent, publicly available datasets. We also conducted testing on Cite-seq data in a similar case setting  evidence of the interaction between genome regions . We applThe extremely high correlation peak-gene pairs yielded by this approach are worth further investigation because they indicate potential regulatory relationships. For example, LEF1 encodes the protein that can bind to a functionally important site in the T-cell receptor-alpha enhancer  and therTo demonstrate how our model can help study diseases, we applied it to a publicly available COVID-19 dataset  where hePreprocessed and normalized data were provided to either the PBMC or BMMC trained model to create the final co-embedding space Fig. A. The fiUsing the model’s co-embedding space, we generated the mini-bulk data per cell type and then inferred the potential regulatory relationships between genes and peaks within a 150 k bp distance from the transcription starting point. Two cell types, Natural Killer (NK) cells and monocytes, were chosen for the integration analysis because they were the most dysfunctional cell types identified in the original study. Only DEGs by severity group were included in this analysis as compensation for their primary scRNA-seq DEG analysis. For example, in NK cells, the DEG HLA-DPB1 is correlated with chr6-32940846–32941346 . Intuitively, cells far away from each other in the graph have larger margin values; highly similar cells that are close in the graph have smaller margin values . Thus, the total loss is:The is the weight between classification loss and contrastive loss.During training data processing, one of the omics data is processed with batch correction, principal component decomposition (PCA), and KNN graph construction. The modality chosen is scRNA-seq for 10X Multiome and Cite-seq training data, because it presents the variation in data better in most cases. With the graph, the shortest distance The two preprocessed feature matrices are scaled by genes to unit variance and zero mean, followed by clipping values larger than 10. We use the Adam optimizer to train the model with user-provided hyper-parameter values including Two assigning strategies were tried and performed equally well. The first is the between-modality strategy, in which negative pairs are different cells in different modalities. The second is the within-modality strategy, in which negative pairs are different cells in the same modality. Because the positive pairs are the same for the two strategies, both within and between-modality co-embedding space correction works well. In the final model, we chose the between-modality strategy for the scRNA-seq and scATAC-seq integration tasks, and the within-modality strategy for the scRNA-seq and cell surface protein integration tasks.The preprocessed and well-annotated bone marrow mononuclear cells data from the NeurIPS 2021 competition can be downloaded in GSE194122. The AnnData object was loaded in Python 3.6.13 with AnnData 0.7.6. All Scanpy-based processing mentioned below is done with Scanpy 1.7.2.pp.highly_variable_genes function with Cell Ranger flavor. Only genes marked as highly variable genes in more than one batch were kept. We also kept cell surface protein genes as the features for training. We then performed stricter cell filtering based on mitochondria gene expression proportion (< 4), number of genes expressed (100–4000), number of peaks , and the total number of fragments . This is the final feature set and cell set for training.Samples from sequencing sites 1, 2 and 3 were taken as the training dataset, including s1d1, s1s3, s2d1, s2d4, s2d5, s3d3, s3d6, and s3d10. S stands for sequencing site and d stands for donor number. First, we performed feature and cell selection. Highly variable genes in scRNA-seq data of all batches were determined by Scanpy gene_activity. It is the count sum of peaks 2 kb upstream of the selected genes' TSS region, calculated by Seurat v3. Together with the feature-selected scRNA-seq data, log-transformed per cell normalization was performed to correct sequencing depth difference. These were the final input of two matrices for model training.We used the already processed gene activity matrix saved in the Anndata obsm connectivity. Distances between neighbor cells were then estimated by 1.01—connectivity value. To calculate the shortest distance between all pairs from the large sparse matrix efficiently, Scipy 1.5.4 dijkstra function was used to generate the To determine the margin value between cell pairs, we constructed a KNN graph using the scRNA-seq data. ComBat implemented in Scanpy was used to perform batch correction, followed by PCA and K nearest neighbor graph construction saved as a large sparse matrix in the AnnData object named pp.highly_variable_genes function with Cell Ranger flavor. Only genes marked as highly variable genes in more than two batches were kept. We also kept cell surface protein genes as the features for training. We then performed stricter cell filtering based on mitochondria gene expression proportion (< 15), the number of genes expressed (75–1200), and the number of peaks (75–1500). All features in ADT data were kept and cell orders were consistent between modalities. The final feature and cell-selected matrices were under log-transformed normalization before training.Samples from sequencing sites 1, 2 and 3 were taken as the training dataset, including s1d1, s1d3, s2d1, s2d4, s2d5, s3d1, and s3d6. We performed similar feature and cell selection as 10X Multiome data in GEX data. Highly variable genes in scRNA-seq data of all batches were determined by Scanpy The same strategy was applied as the 10X Multiome dataset to determine the shortest distances between all cell pairs.gene_activity. Before applying the neural network, we selected the features in training and compensated for missing features with all 0 values. Then the two matrices were scaled by genes to unit variance and zero mean, followed by clipping values larger than 10. Finally, the test mode of the trained model was run to generate the 32-d co-embedding space coordinate for every cell.Samples from s1d2 and s3d7 were taken as the first testing set. Samples from s4d1, s4d8, and s4d9 were taken as the second testing set. Log-transformed transcriptome matrix was used as one of the inputs for the trained model. The gene activity matrix was from the already processed samples saved in the Anndata obsm Samples from s1d2 and s3d7 were taken as the first testing set. Samples from s4d1, s4d8, and s4d9 were taken as the second testing set. The same process was done as the 10X Multiome dataset, but we used the ADT data instead of the gene activity matrix.https://support.10xgenomics.com/single-cell-multiome-atac-gex/datasets/1.0.0/pbmc_granulocyte_sorted_10k. We followed all the same processing of Seurat integration tutorial document at https://satijalab.org/seurat/articles/atacseq_integration_vignette.html. The gene activity matrix was calculated using Signac 1.1.1 summing up counts 2 kb upstream of the gene TSS region. Gene activity matrix and genes count matrix were saved as HDF5 files together with the metadata. Then the files were loaded in Python and underwent log-transformed normalization using Scanpy. The subsequent processes were the same as those applied to the NeurIPS 10X Multiome data.The dataset can be downloaded on 10X Genomics website at https://www.10xgenomics.com/resources/datasets/frozen-human-healthy-brain-tissue-3-k-1-standard-2-0-0. Preprocessing was done by filtering cells in RNA-seq that had less than 1000 counts, larger than 25,000 counts or high mitochondria proportion (> 10%), and filtering cells in ATAC-seq with fragment counts less than 5000 or larger than QUOTE. Only cells remaining in both modalities were kept. Dimension reduction and clustering were done following the Surat default pipeline. The gene activity matrix was calculated using Seurat v3 summing up counts 2 kb upstream of the gene TSS region. The gene activity matrix and genes count matrix were saved as HDF5 files together with the metadata. Then the files were loaded in Python and underwent log-transformed normalization using Scanpy. The subsequent processes were the same as those applied to the NeurIPS 10X Multiome data.The dataset can be downloaded from the 10X Genomics website at https://www.covid19cellatlas.org/index.patient.html. Metadata is downloaded at their GitHub page at https://github.com/ajwilk/COVID_scMultiome. We performed log-transform normalization with the processed data. To stay consistent with scATAC-seq data, we kept only shared donor batches and re-annotated cell types.The fully processed scRNA-seq AnnData H5AD file can be downloaded at CreateGeneActivityMatrix function to generate the gene activity matrix instead of using the ArchR-provided gene activity matrix. The saved HDF5 file was loaded in Python and compiled into AnnData object together with the metadata from their GitHub page and went through log-transformed normalization. Again, to stay consistent with scRNA-seq data, the cell type was re-annotated and only shared batches were kept. Finally, the two log-transformed matrices provided to the model followed the same pipeline as other test datasets. Summary statistics of all datasets mentioned above are available in Additional file The raw data can be downloaded at GSE174072. The fragment files were processed using ArchR 1.0.1 following the same quality control as mentioned in the paper. The batch-specific TSS enrichment score and the minimum number of fragments cutoff can be found on their GitHub page mentioned above. It is worth mentioning that although these researchers claim the sequencing reads were aligned with the hg19 reference genome, we found that using only hg38 can yield the correct TSS enrichment score. Thus, hg38 was used in all following related processes. We followed the same data processing pipeline in the paper, removing doublets, clustering, batch correction with Harmony, and calling peaks with MACS2. To follow the same practice as the training dataset, we used the Seurat 3.1.1 GLUE 0.1.1, bindSC 1.0.0, Seurat 3.1.1, UnionCom 0.2.3, Liger 1.0.0, its Online-iNMF and UINMF version were all included to obtain a systematic benchmarking. Due to the memory outflow problem, UnionCom failed to get the results with GLUE-provided codes on their GitHub page. All others were implemented successfully according to the authors' tutorial. All codes are available at GitHub.https://scglue.readthedocs.io/en/latest/tutorials.html with all default settings. We started from raw test data to run the data preprocessing and model training steps. The final cell co-embedding space was saved for all benchmarking. GLUE was run eight times with different random seeds.We followed GLUE's tutorial at https://htmlpreview.github.io/?https://github.com/KChen-lab/bindSC/blob/master/vignettes/CITE-seq/CITE_seq.html for Cite-seq data integration and https://htmlpreview.github.io/?https://github.com/KChen-lab/bindSC/blob/master/vignettes/method_eval/method_eval.A549.html for 10X Multiome data integration task. The final embedding used was the bi-CCA generated results. BindSC was run eight times with different random seeds.We followed bindSC's tutorial at https://satijalab.org/seurat/articles/atacseq_integration_vignette.html for both Cite-seq and 10X Multiome data integration. The final embedding space is the UMAP dimensional reduction space following Seurat integration pipeline.We followed Seurat's tutorial at https://htmlpreview.github.io/?https://github.com/welch-lab/liger/blob/master/vignettes/walkthrough_rna_atac.html and http://htmlpreview.github.io/?https://github.com/welch-lab/liger/blob/master/vignettes/online_iNMF_tutorial.html. Cite-seq data was integrated using Liger and its UINMF version at http://htmlpreview.github.io/?https://github.com/welch-lab/liger/blob/master/vignettes/UINMF_vignette.html. Each model was run eight times with different random seeds.Liger and its online iNMF version were implemented for 10X Multiome data integration, following the tutorials at All the UMAP visualizations were done either using the software’s available functions or Scanpy default settings.Silhouette score was used to evaluate how well cell types were clustered and modalities were mixed. It is a measure of how similar an object is to its own cluster (cohesion) compared to other clusters (separation). The silhouette ranges from − 1 to + 1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. The silhouette score was calculated using Scikit-learn 0.24.2. To measure how well cell types were clustered, we used the raw silhouette score value. For modality mixing and batch mixing, adjusted_rand_score function comparing unsupervised clustering and cell type annotations. Unsupervised clustering was done using Scanpy tl.leiden function with different resolutions so that all algorithms received the evaluation on cluster numbers from eight to the number of cell types in each dataset.Rand index is a measure of similarity between two sets of data clustering. Adjusted rand index was calculated using Sklearn 1.0.1 n cells, when cells are projected into the co-embedding space, the FOSCTTM we calculated was defined as:FOSCTTM (Fraction of samples closer than the true match) score was used to evaluate the co-embedding space at single-cell resolution. Assuming two single-cell omics data profiled the same set of Label transfer accuracy is used to measure the performance of all co-embeddings on this common task. The transfer is measured from scRNA-seq cell type annotations to either scATAC-seq or cell surface protein data. While Seurat used its own label transfer method, all other algorithms' label transfer is done by weighted K nearest neighbors. That is, the label of cell from the second modality is predicted as the max weighted vote of its K nearest cells in scRNA-seq. K is chosen for each algorithm when it reached the best performance. With the predicted cell type label and the true label, the label transfer accuracy is defined as:Correlation-based regulatory element inferring is always weakened because of the high dropout rate. To solve this, we first undertook data smoothing and generated transcriptome and chromatin accessibility mini-bulk data with the single-cell resolution co-embedding space.pp.neighbors function with default parameters and different number of neighbors, including 5, 10, 15 and 20. The raw count matrix was multiplied by the binarized connectivity matrix to complement missing values by neighbors. The connectivity matrix was binarized by two steps: (1) cells are the nearest neighbors of the target cell; (2) the distance between them should be smaller than the 95% distance percentile value. Cells passing the criteria were used to complement the target missing values by keeping the value as 1 in the binary connectivity matrix.A nearest neighbor graph was constructed based on the model generated co-embedding space using Scanpy pdist function to perform hierarchical clustering of all cells in the two modalities. Then with the hierarchical order and cell types, cells were cut into N = 100 mini-bulks, and each mini-bulk was ensured to contain only one cell type. The mini-bulk matrix was generated with scglue 0.1.1 aggregate_obs function.We used Scipy 1.5.3 Next, genes of interest were selected and their correlation with their 150 kb upstream peaks were calculated from the mini-bulk data. The abnormally high correlations between remote peaks and genes might indicate cis-regulatory relationships. Correlation results were visualized with box and violin plots using ggplot2 in R.https://ars.els-cdn.com/content/image/1-s2.0-S0092867416313228-mmc4.zip and https://osf.io/e594p/. Our codes were based on GLUE's processing and scglue functions but simplified to only extract the pcHi-C evident pairs. Only evidence of overlapped cell types was chosen to be validated. Then scglue was used to map these peak-gene pairs to the 10X Multiome PBMC dataset peak-gene pairs. The evidence was saved as.graphml file for reading and writing efficiency.pcHi-C data is available at findMotifsGenome were applied to enrich the peaks of interest in known motifs. In some cases, we had only a few peaks, so the statistical test was not reliable. In such cases, we only trusted the information regarding which motifs were matched with most of the peaks.Peaks of interest were listed in BED format as the input for Homer v4.11.1. Function plotBrowserTrack and Seurat v3 CoveragePlot function.All visualization figures were done using ggplot2. The example genome tracks were plotted with ArchR Additional file 1: Table S1. The NeurIPS 2021 competition dataset summary. Figure S1. 10X Multiome BMMC test set 1 UMAP visualization of benchmarking algorithms. Figure S2. 10X Multiome BMMC test set 2 UMAP visualization of benchmarking algorithms. Figure S3. Cite-Seq BMMC test set 1 UMAP visualization of benchmarking algorithms. Figure S4. Cite-Seq BMMC test set 2 UMAP visualization of benchmarking algorithms. Figure S5. Model Generalizability Evaluation in external datasets. Figure S6. 10X Multiome PBMC dataset UMAP visualization of the co- embedding space labeled by cell type, modality, and batch. Figure S7. UMAP visualization of the co-embedding space labeled by cell type and batch in batch effect removal scenario 2. Figure S8. UMAP visualization of the co-embedding space labeled by cell type and batch in batch effect removal scenario 3. Figure S9. Cis-regulatory element inferring supplementary figures. Figure S10. Genome tracks of examples mentioned in COVID-19 data analysis.Additional file 2: Table S2. Summary of datasets used in the study.


Large compendia of gene expression data have proven valuable for the discovery of novel biological relationships. Historically, most available RNA assays were run on microarray, while RNA-seq is now the platform of choice for many new experiments. The data structure and distributions between the platforms differ, making it challenging to combine them directly. Here we perform supervised and unsupervised machine learning evaluations to assess which existing normalization methods are best suited for combining microarray and RNA-seq data. We find that quantile and Training Distribution Matching normalization allow for supervised and unsupervised model training on microarray and RNA-seq data simultaneously. Nonparanormal normalization and z-scores are also appropriate for some applications, including pathway analysis with Pathway-Level Information Extractor (PLIER). We demonstrate that it is possible to perform effective cross-platform normalization using existing methods to combine microarray and RNA-seq data for machine learning applications. An evaluation of normalization methods shows that it is possible to combine microarray and RNA-seq data for machine learning applications. Integrative analyses of multiple microarray cohorts have uncovered important signatures in human infection4. Sequencing-based RNA assays have certain advantages over array-based methods, namely quantitative expression levels, a lack of dependence on current annotations, and a higher dynamic range5. As a result, researchers have increasingly adopted this technology for their gene expression experiments.The union of large and diverse compendia of gene expression data with machine learning approaches has enabled the extraction of cell type-specific networks9. Integrative analyses of gene expression requires the combination of these data types—a task of utmost importance for rare diseases or understudied biological processes and organisms where all available assays will be required to discover robust signatures or biomarkers. Thus, effective strategies for combining data from the two platforms—perhaps in a manner that leverages both the advantages of RNA-seq and the abundance of microarray data—are paramount to transcriptomic and functional genomic experiments going forward.RNA-sequencing (RNA-seq) assays represent a growing share of new gene expression experiments. In February 2022, the ratio of summarized human microarray to RNA-seq samples from GEO and ArrayExpress was close to one to one (1.13:1), meaning that half of all summarized human samples come from either platform, although RNA-seq overtook microarray data as the leading source of new submissions to ArrayExpress in 201816. Quantile normalization (QN) is a widely used normalization technique originally utilized for microarray data17, and it has also been adopted for RNA-seq data normalization18 and in some cases, cross-platform normalization19. Probe Region Expression estimation Based on Sequencing (PREBS) was developed to make RNA-seq and microarray data more comparable, but this method requires raw reads and probe specific information and may not be feasible for large-scale public data efforts20. Training Distribution Matching (TDM) was developed by our group to make RNA-seq data more comparable to microarray data from transcript abundances specifically for machine learning applications21. In that work, it was demonstrated that QN, TDM and a method from the analysis of graphs, nonparanormal normalization (NPN), had good performance in the supervised learning evaluation. However, combining array and RNA-seq platforms was not evaluated. Recent efforts to integrate single-cell RNA-seq data across experiments and modalities has generated many new approaches23.Much work has been performed to develop methods for effectively combining multiple cohorts or batches of gene expression data, but no method has been widely adopted for the problem of combining mixtures of array and RNA-seq platform dataHere, we present a series of experiments to test what normalization approaches can be used to combine microarray and RNA-seq data for supervised machine learning and unsupervised machine learning, including pathway analysis. We chose subtype and mutation status prediction for our supervised machine learning tasks because they are commonly used in cancer genomics studies and the labels are well-defined in our data. Our unsupervised machine learning results focus on pathway analysis since this is an important downstream application for understanding biologically relevant gene expression patterns. Given the broader dynamic range of RNA-seq data, this work focuses on the problem of normalizing RNA-seq data to a target distribution of array data. We specifically add varying numbers of RNA-seq data to our training sets to assess at what point performance begins to suffer. We find that QN, TDM, NPN, and standardized scores are all suitable for some use cases, with the widely adopted QN performing well for machine learning applications in particular.We performed a series of supervised and unsupervised machine learning evaluations to assess which normalization methods are best suited for combining data from microarray and RNA-seq platforms. We evaluated seven normalization approaches for all methods: LOG, NPN, QN, QN (CN), QN-Z, TDM, and standardizing scores , plus untransformed data for comparison (UN) added as an additional negative control.We trained models to predict subtype from our BRCA and GBM training sets with varying numbers of samples from the RNA-seq platform. We used these models to predict subtype on holdout data sets composed entirely of microarray data or RNA-seq data Fig. . We trai21. We also saw that z-scoring data resulted in the most variable performance. This is not unexpected because the calculation of the standard deviation and mean will be highly dependent on which samples are selected from each platform and the random selection of RNA-seq samples to be included in the training set does not consider subtype distribution, which may not be known in practice. We found that NPN, QN, QN-Z, and TDM all performed well when moderate  amounts of RNA-seq data were incorporated into the training set. These results are consistent with the high performance of these three methods in our case study of training entirely on microarray data and using solely RNA-seq as a test set21. These three methods performed well on both the microarray and RNA-seq holdout sets. QN followed by z-scoring (QN-Z) showed similar results to QN without z-scoring. Untransformed data (UN) showed poor performance at all titration levels.Importantly, for BRCA, there were appreciable differences between normalization methods Fig. . GBM resQuantile normalization did not perform well at the extremes (0 and 100% RNA-seq data). We attribute this loss of performance to the lack of reference distributions in these cases—for all other amounts of RNA-seq (10–90%), the set of microarray data is used as a reference distribution for both the RNA-seq data included in the training set as well as for the holdout set (see Methods and Data). This result reiterates the importance of drawing training and holdout sets from the same distribution, as is well-documented in the machine learning literature, and highlights the necessity of proper cross-platform normalization.TP53 and PIK3CA mutation status in BRCA and GBM. We report results for TP53 prediction in GBM in Fig. PIK3CA mutation prediction in GBM showed negligible improvement over the null model, possibly due to the low proportion of samples with mutations (n = 10 out of 98 samples in training)  or mixed-platform (combination of microarray and RNA-seq). Here, we wanted to examine the benefit of creating a larger data set through the combination of platforms for downstream analysis. For BRCA, the half-size single platform data comprised 174 samples , while the full-size single platform and cross-platform data each had 348 samples (the full set of training samples). For GBM, the sample sizes were 51 for half-size and 102 for full-size data sets. As a negative control, we permuted gene-pathway relationships used as input to PLIER to establish the baseline rate of false positive pathway associations. We found that out of 110 PLIER runs, five runs reported one false positive significant pathway association, while 105 runs reported no significant pathways.We ran PLIER25. We found that doubling the sample size led to important cancer pathways being identified more stably and regularly in both the single-platform and cross-platform settings, including pathways related to ERB2, NFKB immune infiltration, and RAF in BRCA, and KRAS, MYC, and PRC2-related pathways in GBM. Pathway results are included for BRCA  and evaluated normalization method performance. The molecular subtypes in BRCA and GBM are strong, linear signals that we should be able to predict in the holdout sets given the performance of the classifiers visualized in Fig. We observed differences in classifiers and normalization methods as measured by the Kappa statistics , perform well for both types of machine learning approaches. NPN is a strong performer in the supervised ML applications and the unsupervised PLIER analysis but may not be appropriate in the unsupervised PCA reconstruction applications given the decrease in Kappa in predictions on reconstructed expression data. Other applications of cross-platform normalization, such as combining array-based gene expression data from different array manufacturers  could follow the same principles as what we have described here.This study has some important limitations. Because of our experimental design, we required large data sets of samples run on both platforms. This focused our work on two high quality data sets: TCGA BRCA and GBM data. BRCA and GBM subtypes and common gene mutations are well-defined signatures that have evident linear expression patterns. As a result, our guidance may not generalize to nonlinear classifiers, data sets of poor quality, or small sample sizes. In the context of mutation status prediction, we found the mutation class balance to differ between subtypes. Therefore, the overall expected accuracy calculated for the Kappa statistic may not adequately reflect the expected accuracy if, for instance, the classifier predicts the majority mutation class within a subtype. This suggests that the Kappa statistic may be less well-suited for this evaluation compared to predicting molecular subtype.26 and should be considered for future work, including recent approaches like meGPS that apply this idea across platforms and data types27.In addition, we stress that the biological question at hand when performing cross-platform normalization must be considered, as some assumptions underlying a normalization method may be violated. By harmonizing RNA-seq to microarray data, we gain the benefit of larger sample sizes required for more data-intensive machine learning applications. But, while gene expression distributions can be adjusted, the properties of count-based data, especially with respect to ties and true zero values, may hinder our ability to measure performance across multiple tasks. Alternative approaches may not require reshaping RNA-seq data to match array data. For example, methods utilizing gene-pair ratios may rely on relative expression levels of genes within samples to identify useful featuresWe designed these experiments to evaluate the combination of microarray and RNA-seq data in practice in the absence of samples measured on both platforms. This work indicates that it is possible to perform model training on microarray and RNA-seq. Combining both platforms could allow models to take advantage of the additional information captured in some RNA-seq experiments while benefiting from the historical abundance of microarray data. For example, with low false positive rates, we showed that increasing sample size with cross-platform normalization yielded more significant pathways in our PLIER analysis. Training sets comprising samples run on both platforms will be a new reality as RNA-seq becomes the platform of choice and the ability to perform such analyses will be of particular importance for understudied biological problems.We aimed to assess the extent to which it was possible to effectively normalize and combine microarray and RNA-seq data with existing methods for use as a training set for machine learning applications. We assessed performance on holdout sets composed entirely of microarray data and entirely of RNA-seq data. To design such an experiment, we required data sets that had matched samples—sets of samples run on both microarray and RNA-seq—and that were of sufficient size. Our matched samples design allowed us to directly evaluate how the addition of RNA-seq data into the training set affects performance without the added complication of a change in the underlying samples . Using a different set of samples for training or evaluation could have resulted in a change in performance that was unrelated to the platform difference, which could confound the results. For our machine learning experiments, we used both microarray and RNA-seq holdout sets composed of the same samples to assess how the addition of RNA-seq data into the training set affected test accuracy on both widely used genome-wide expression platforms.Although our experimental design required matched samples, we designed an analysis to evaluate normalization methods for the purpose of combining RNA-seq and microarray data when matched samples are not available. We compare holdout performance with both platforms separately to assess the extent to which models trained on a mix of RNA-seq and microarray data performed differently on test sets from the two platforms . We also took care to design the normalization process such that it is applicable for data sets without matched samples—that is, when reference distributions are required, we do not use all microarray samples to normalize the RNA-seq, but rather the subset of microarray data included in the training set .31. In addition, both BRCA and GBM have well-defined molecular subtypes that are suitable for use as labels/classes for supervised machine learning approaches we describe below. BRCA and GBM also show recurrent TP53 and PIK3CA gene mutations, allowing for the prediction of mutation status based on gene expression profiles34. For BRCA (520 pairs of matched samples), we used log2-transformed, lowess normalized Agilent 244 K microarray data29 and RSEM (RNA-seq by Expectation Maximization) gene-level count RNA-seq data35. For GBM (150 pairs of matched samples) we obtained Affymetrix HT Human Genome U133A Array data from refine.bio (GSE83130) normalized by the SCAN method37. GBM RNA-seq upper quartile FPKM gene expression data came from the National Cancer Institute Genomic Data Commons TCGA Pan-Cancer Atlas39. Likewise, SNP and indel mutation calls came from the GDC Pan-Cancer Atlas MC3 public MAF (v0.2.8)40. We consider these data to be the products of standard processing pipelines for their respective DNA and RNA expression platforms. Our methods focus on cross-platform normalization and assume that any within-platform batch effects have already been corrected upstream.The Cancer Genome Atlas (TCGA) breast cancer (BRCA) and glioblastoma (GBM) data sets include samples that have been measured with both microarray and RNA-seq platformsFor the purpose of these analyses, we restricted the data set to the tumor samples (termed ‘matched samples’) that were measured on both platforms . Mutation calls were missing for 12/520 BRCA samples and 4/150 GBM samples. A total of 16146 genes were measured on both expression platforms for BRCA and 11414 genes for GBM.41. See Supplementary Fig. An overview of our experimental design for machine learning evaluations is illustrated in Fig. Machine learning methods are applied to each training set , which have each been normalized in various ways according to the details below . We then applied prediction models based on each training set to our holdout data sets. Two holdout sets were used: a set of RNA-seq data and a set of microarray data. We refer to these as the RNA-seq holdout set and microarray holdout set, respectively. These holdout sets (or test sets) of normalized microarray and RNA-seq data are kept separate and not titrated together in order to evaluate how the composition of the training set impacts prediction performance in each data type. The pipeline for partitioning data into training and testing, titration, and normalization was repeated 10 times using different random seeds, as were the downstream analyses .In each randomized repeat of our experiment, the same set of patients was used across all training sets, with only the platform of the sample (either microarray or RNA-seq) varying across RNA-seq titration levels. Likewise, the microarray and RNA-seq holdout sets consisted of matched samples from the same set of patients. In accordance with how these methods would be used in practice, normalization was performed separately for training and holdout sets. For the normalization methods that require a reference distribution , the RNA-seq training and holdout sets were normalized using the microarray portion of the training data as a reference, separately. This is analogous to the normalization process in the absence of matched samples; the training microarray data , the normalize.quantiles.use.target method will normalize the columns (samples) of the RNA-seq data such that the data sets are drawn from the same distribution. For sets entirely composed of samples on a single platform without an applicable reference, normalize.quantiles was used.QN was performed using the preProcessCore R package43 combines data from different distributions by stacking columns from paired samples or, in the more general case, combinatorially stacking columns from each distribution before performing quantile normalization. Before combining data sets, the array, and RNA-seq sample expression values were rescaled zero-to-one. The QN (CN) process differs from QN or QN-Z because RNA-seq samples are not normalized to fit the array distribution; rather, the samples from array and RNA-seq are cross-normalized toward the same shared distribution together. Since CrossNorm requires two data sets, there is no QN (CN) data for training at 0 and 100% RNA-seq. Likewise, QN (CN) array and RNA-seq test data is simply rescaled and quantile normalized without reference to array training data.CrossNormThis normalization method sequentially combines quantile normalization (QN) and z-score transformation (Z).21. It identifies the relationship that the microarray training data has between the spread of the middle half of the data and the extremes and then transforms the RNA-seq test set such that it has the same relationship between the spread of the middle half of the data and the extremes. Specifically, given a reference distribution (array data), TDM calculates the reference interquartile range (IQR), and then finds the number of times the reference IQR fits into the reference distribution’s upper (max—Q3) and lower (Q1—min) quartiles. Then, TDM sets the new upper and lower bound of the target distribution (RNA-seq) by stretching or shrinking the target upper and lower quartiles by the IQR factors calculated from the reference distribution. Target distribution values are winsorized to match the new upper and lower bounds. The target distribution is then rescaled between 0 and 1, stretched to match the range of the reference distribution, and shifted to match the minimum value of the reference distribution.TDM was developed specifically to make RNA-seq test data compatible with models trained on microarray data2-transformed microarray data were used for prediction/reconstruction on the TDM normalized RNA-seq holdout set. TDM was not used when the training set consisted of 100% RNA-seq data, as there is no relevant microarray data to use as the reference distribution in this case.We used the TDM R package to perform TDM normalization. For the training set composed of 0% RNA-seq data, models trained on log44. NPN was performed using the huge R package prior to concatenating samples from both platforms and separately on single-platform holdout sets45.Our implementation of NPN is a rank-based inverse normal transformation that forces data to conform to a normal distribution by rank-transformation followed by quantile normalization, placing each observation where it would fall on a standard normal distributionz-scoring was performed on a per gene basis using the scale function in R prior to concatenating training samples from both platforms. Single-platform holdout sets were z-scored separately. Z-scores or standard scores are calculated Gene expression values were standardized to the range As an additional basis for comparison, untransformed data (UN) was included in most analysis tasks. Untransformed array data is LOG data with no zero-to-one transformation. Untransformed RNA-seq data has no zero-to-one transformation.50 in which single cells became single (bulk) samples, and our array and RNA-seq data sets were the two batches or experimental conditions. Due to low sample numbers at the edges of our titration protocol, many experimental conditions could not be integrated. For those conditions with enough samples from array and RNA-seq for successful integration, we have included machine learning prediction results in table form (see Data availability).Other approaches to combining and normalizing gene expression data are commonly used in single-cell RNA-sequencing studies to integrate data sets by reducing batch effects across studies and experimental conditions. We designed an approach analogous to Seurat’s integration pipelineTraining data sets are mixtures of microarray and RNA-seq samples titrated at 10% intervals, starting with 0% RNA-seq, then 10%, 20%, …, and 100%. When RNA-seq samples are added to a training set, the matched microarray samples are removed. At each titration level, microarray and RNA-seq samples are normalized according to various methods.LOG data is the input to nonparanormal (NPN), quantile (QN), QN followed by Z (QN-Z), and z-score (Z) normalization. UN array data is LOG data without zero-to-one transformation. There is no QN (CN) or TDM for 100% array training data.2(x + 1) transformed, then array and RNA-seq values undergo gene-level zero-to-one transformation separately before being joined together. For NPN, array and RNA-seq values undergo gene-level nonparanormal normalization separately, then get combined before gene-level zero-to-one transformation. For QN, RNA-seq data is transformed to align with the quantiles of array data, array and RNA-seq undergo gene-level zero-to-one transformation separately and are then combined. For QN (CN), each array and RNA-seq column (sample) is scaled zero-to-one and then normalized using the unpaired CrossNorm algorithm, followed by gene-level zero-to-one transformation. For QN-Z, array and RNA-seq data are QN and Z normalized separately before getting combined for gene-level zero-to-one transformation. Similarly, for TDM, we transform the spread of RNA-seq data to match the spread of array data, array and RNA-seq undergo gene-level zero-to-one transformation separately and are then combined. For Z, RNA-seq and array data are z-scored separately at the gene-level, then combined for gene-level zero-to-one transformation. For UN, RNA-seq data remains untransformed and data do not undergo gene-level zero-to-one transformation.For LOG, microarray samples are LOG and RNA-seq values are logUntransformed RNA-seq data gets LOG, NPN, QN, QN-Z, and Z transformed by itself without reference to array data. It then undergoes gene-level zero-to-one transformation. For UN, RNA-seq data remains untransformed and data do not undergo gene-level zero-to-one transformation after each normalization. There is no QN (CN) or TDM applied to 100% RNA-seq training data.Like the 100% array training data, array test data gets LOG, NPN, QN, QN (CN), QN-Z, UN, and Z transformed by itself without reference to any other data, and it then undergoes gene-level zero-to-one transformation after each normalization except UN. There is no TDM for 100% array test data.For LOG, NPN, QN (CN), and Z, RNA-seq values are transformed without reference to any other data. We transform the 100% RNA-seq test data using QN, QN-Z, and TDM across the RNA-seq titration level spectrum (0–90%) to match the training LOG array data associated with each titration level (0% through 90%), just as the training RNA-seq data was transformed to match the training LOG array data at each level. QN and QN-Z use no reference for 100% RNA-seq test data, and there is no TDM applied to 100% RNA-seq test data. Transformed data undergoes gene-level zero-to-one transformation after each normalization except UN.31. Because these labels were derived from the microarray data, analyses that aim to compare the subtype prediction quality of microarray- and sequencing-based platforms would be inappropriate. Our evaluations are restricted to the relative performance of normalization methods across both platforms.Subtype classifications—derived from TCGA’s analysis of the microarray data, rather than a clinical assay—were used as the subtype labels for supervised analyses Fig. 29–31. Be51, linear support vector machine (SVM), and random forest. We used the glmnet R package implementation of LASSO52. The training of SVM and random forest classifiers was performed using the caret R package and utilizing the kernlab53 and ranger54 R packages, respectively. We used the Kappa statistic to evaluate performance on holdout data for two main reasons. The first reason is we make no assumptions about class balance in our data, so to mitigate the potential for bias due to class imbalance, we chose a metric that builds in a baseline probability of chance agreement. The second reason is to accommodate our multi-class outcome , which Kappa achieves without relying on a composite of one-vs-all comparisons, as classes were not balanced. Briefly, the Kappa statistic takes into account the expected accuracy of a random classifier and is generally considered to be less misleading than observed accuracy alone55. The formula for Cohen’s Kappa is:We performed fivefold cross-validation on training sets for model training and hyperparameter optimization using total accuracy for performance evaluation. We trained the following three classifiers: LASSO logistic regressionTP53 and PIK3CA mutation calls, including SNPs and indels, were derived from TCGA’s MC3 public call set (v0.2.8)40. Our evaluation approach changed slightly for predicting mutation status. Given the higher prevalence of TP53 and PIK3CA mutations in some expression-based subtypes, we wanted to make sure our mutation classifiers were not just predicting subtypes as a proxy for mutation status. We accounted for this by calculating the change in Kappa (delta Kappa) between a null model and a model built with true labels. Our null model was built with mutation labels randomized within each subtype. The null model Kappa represents the ability of the classifier to differentiate subtypes based on the prevalence of mutations in that subtype. Positive delta Kappa values indicate the ability of the model to predict mutation status beyond what would happen given the association of each mutation with particular subtypes.24. PLIER uses a set of biological constraints defined by gene expression pathways to decompose an expression matrix with non-negative matrix factorization. One output is a set of latent variables and loadings, and some of the input pathways may be associated with one or more latent variables, suggesting that the pathway plays a role in regulating gene expression. We ran PLIER using training data to assess the downstream benefit of creating larger data sets through cross-platform normalization. We report the proportion of significant pathways out of the global set of pathways, with higher proportions indicating a more complete capture of real biological variation. As a negative control, we identified the baseline false positive rate of significant results obtained by chance by permuting the gene-pathway relationships represented in the gene-pathway input matrix. Specifically, we permuted the gene (rows) and pathways (columns) matrix by resampling without replacement within each column such that the number of genes associated with each pathway remained constant. We compared results from single platform data sets (microarray or RNA-seq only) with normalized data comprising 50% microarray and 50% RNA-seq samples. For the single platform data sets, we included some with half of the available samples and some with the full set of samples. The normalized data sets comprised the full set of samples with half coming from either platform. The global set of biological pathways included 817 canonical, oncogenic, cell line, and blood cell pathways. For each input gene expression data set, we identified the total number of pathways significantly associated (FDR < 0.05) with at least one latent variable. To evaluate the potential benefit of increasing sample size to pathway stability and consistency, we identified oncogenic pathways that were more frequently (change in proportion > = 0.2) associated with at least one latent variable in the full sample size data sets compared to the half sample size data sets.We used PLIER to identify coordinated patterns of expression in our data, which may be the result of gene regulation56. We selected mean absolute scaled error (MASE) to assess reconstruction error based on the needs of our application. Scale invariance is a key feature enabling comparison across multiple sequencing types and normalization methods. MASE is well suited for data sets with predicted values close to zero, in contrast with other measures. MASE also measures error in absolute terms, matching our preference to weight over and under estimation of values equally. MASE is calculated on a per gene basis as follows:We hypothesized that normalization methods may distort the utility of low-dimensional projections and that this distortion may differ by the level of RNA-seq data titrated into each training set. To test this, we evaluated machine learning tasks after projecting the test data onto the low-dimensional space defined by the training data. We performed principal components analysis (PCA) on each training set using the prcomp function in R, setting the number of components to 50 Fig. . We thenWe performed supervised analysis following reconstruction to assess whether the subtype signals were retained or if features were dominated by noise introduced by combining platforms. We used the models trained for subtype classification to predict on the reconstructed holdout sets to assess how well the molecular subtype signal was retained in the reconstructed holdout data Fig. . We agaihttps://www.github.com/greenelab/RNAseq_titration_results for instructions on how to access the Docker container, download data, and get started running analyses, including a bash script to reproduce the entire analysis. We set seeds for reproducibility to account for stochastic elements in our code.Statistical analyses were performed using R version 4.1.2 in a publicly available Docker image. Please refer to our source code (v2.3) on Github at PIK3CA and TP53), 508 matched samples were available for breast cancer and 146 matched samples were available for glioblastoma.For each cancer type and predictor variable (subtype or mutation status), we created 10 replicate training and test data sets for machine learning and downstream analyses. Training and test data were kept separate from the start of each replicate. We used createDataPartition from R’s caret package to balance the distribution of classes between training and test data. For a single replicate, two-thirds of available samples were allocated to training and one-third to test data. The exact number of samples partitioned to training and test data sets may vary slightly between replicates due to stratified sampling. For subtype prediction, 520 matched samples were available for breast cancer and 150 matched samples were available for glioblastoma . Importantly, as RNA-seq samples were titrated into the training data, the corresponding array sample was removed such that one patient’s data was only represented by one sample per experimental repeat.Further information on research design is available in the Peer Review FileSupplementary InformationDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Reporting Summary


Due to the overall high costs, technical replicates are usually omitted in RNA-seq experiments, but several methods exist to generate them artificially. Bootstrapping reads from FASTQ-files has recently been used in the context of other NGS analyses and can be used to generate artificial technical replicates. Bootstrapping samples from the columns of the expression matrix has already been used for DNA microarray data and generates a new artificial replicate of the whole experiment. Mixing data of individual samples has been used for data augmentation in machine learning. The aim of this comparison is to evaluate which of these strategies are best suited to study the reproducibility of differential expression and gene-set enrichment analysis in an RNA-seq experiment. To study the approaches under controlled conditions, we performed a new RNA-seq experiment on gene expression changes upon virus infection compared to untreated control samples. In order to compare the approaches for artificial replicates, each of the samples was sequenced twice, i.e. as true technical replicates, and differential expression analysis and GO term enrichment analysis was conducted separately for the two resulting data sets. Although we observed a high correlation between the results from the two replicates, there are still many genes and GO terms that would be selected from one replicate but not from the other. Cluster analyses showed that artificial replicates generated by bootstrapping reads produce it p values and fold changes that are close to those obtained from the true data sets. Results generated from artificial replicates with the approaches of column bootstrap or mixing observations were less similar to the results from the true replicates. Furthermore, the overlap of results among replicates generated by column bootstrap or mixing observations was much stronger than among the true replicates. Artificial technical replicates generated by bootstrapping sequencing reads from FASTQ-files are better suited to study the reproducibility of results from differential expression and GO term enrichment analysis in RNA-seq experiments than column bootstrap or mixing observations. However, FASTQ-bootstrapping is computationally more expensive than the other two approaches. The FASTQ-bootstrapping may be applicable to other applications of high-throughput sequencing. Due to the overal costs of an RNA-seq experiment, technical replicates are usually omitted, especially in observational studies which already include a large number of biological replicates. The general need for technical replicates has been discussed controversely in the last years. While, for example, a high reproducibility of RNA-seq analyses has been reported, the same study points at cases were technical replicates could improve the statistical power of an experiment4. McIntyre et al.5 reported that reproducibility also depends on the size of coverage of sequencing reads across the reference genome. It has also been found that technical replicates can be useful to detect potential lane effects of the flow cell6, the solid support of the RNA material within the sequencing machine. Moreover, Li et al.7 argue for the general need of measuring the reproducibility of high-throughput experiments, since many studies have shown changes in the ranking of selected features . Such changes can finally lead to different biological interpretation, or to different outcomes of subsequent analyses, e.g. gene-set enrichment analysis9.High-throughput sequencing of RNA samples (RNA-seq) has become the standard for generating gene expression profiles of biological samples with a wide range of examples in biomedical research10. This technique does not generate new expression values but a new permutation of existing data. Thus, a replicate of the whole experiment and not for individual samples is generated. The second approach uses bootstrap samples from the sequencing reads stored in the individual FASTQ-files. Bootstrapping from a set of sequencing reads has also already been proposed in the context of transcriptomics11 and also metagenomics12. This approach generates new expression levels for each biological sample that can be regarded as an artificial technical replicate. Finally, we include the approach of mixing observations by using a weighted mean over the columns of the expression matrix, a typical technique for data augmentation in machine learning13. Usually, one expects in an experiment that the correlation of data from the same biological sample  is larger than the correlation between data from biologically independent samples. Therefore, by mixing the data of biologically independent samples, this approach appears more appropriate to generate artificially biological replicates. In the following, we denote the three approaches by FB , CB , and MO . By their nature, the CB and MO approach do not produce technical replicates for individual samples, but they still may be suited to study the reproducibility of an RNA-seq experiment.To circumvent to obstacle of additional costs for technical replicates, we study and compare three different approaches to generate replicates of RNA-seq experiments computationally. The first approach has been presented in the context of DNA microarray experiments and takes bootstrap samples from the columns of the gene expression matrixHere, we compare the three approaches of artificially generating replicates of RNA-seq experiments on a data example as a case study produced by an experiment in our own labs. The experiment consists of infected and non-infected samples each subjected twice to RNA-seq, i.e. with two real technical replicates. The two resulting data sets are denoted by R1 and R2 in the following. We observe how well analytical results obtained from artificial replicates generated from data of R1 fit to the results obtained from data of the true technical replicate R2, and vice versa. Comparisons are done on the level of it p values and log fold changes from differential expression analysis and subsequent GO term enrichment analysis. The comparison is mainly done explorative looking at similarities between results from true and artificially generated data. We also compare the gene-wise dispersion estimates for biological and technical variance in the real and artificial data.In this section, we detail first the three different approaches for generating artificial replicates of individual samples or replicates of the whole experiment Fig. . In addi14 and mapped to the mouse reference genome (Version GRCm38 from https://www.ncbi.nlm.nih.gov/) using STAR15. Each differential expression analysis was performed by the R-package DESeq23 to generate lists of it p values and log fold changes. Raw it p values were adjusted to control a false discovery rate of 16. GO term enrichment analysis was performed based on the it p value lists from differential expression analysis using Enrichr17. Differential expression and enrichment analyses were performed separately for data sets R1 and R2.Raw FASTQ-files were processed by quality-based trimming using Trimmomaticd genes in its rows and n samples in its columns. This is similar to expression matrices obtained in DNA microarray experiments, except for the difference that microarray experiments generate fluorescence values as measure of gene expression. By bootstrapping with replacement columns from the expression matrix, new realizations of the whole experiment can be generated. Consider for example the data matrix has 10 columns, a new realization of the experiment would consist a random sample of these 10 columns, e.g. columns {8, 3, 4, 6, 3, 4, 1, 1, 8, 9}. From these realizations, new lists of it p values and log fold changes can be calculated. We applied this approach 10 times on the expression matrix from R1 and 10 times on the expression matrix from R2.After mapping the reads of each original FASTQ-file to the reference genome, a matrix of read counts is generated which contains k denotes the number of reads in the original file and A FASTQ-file is typically generated as a result of a high-throughput-sequencing run and lists the sequences  of millions of DNA or RNA molecules, together with quality keys for the detection of each single nucleobase. To generate an artificial technical replicate, we draw n FASTQ-files with subsequent mapping and counting. After the differential analysis has been performed, artificially generated FASTQ-files are removed before doing the next run.Due to the generation of additional FASTQ-files, this approach requires a higher amount of storage. Consider, one original FASTQ-file has a size of 5 GByte, and one wants to generate 10 artificial replicates using the FB approach, 50 GByte of additional storage would be necessary temporarily. Therefore, our strategy is to first generate ith column of the expression matrix, i.e. a vector of length d, and n new samples by this approach and bind them together as a new Mixing observations is a typical technique of data augmentation regularly used to obtain more accurate machine learning models. To mix observations has particularly a long tradition in image classification. Here, we use a weighted mean of the columns of the following form to generate a new artificial replicates: 18 and 5 independent samples infected with Batai orthobunavirus (BATV)19. Cells used for infections as well as controls derived from the same batch were seeded in separate T25 ccmIn order to demonstrate the characteristics of each approach, we performed a small RNA-seq experiment with 4 independent control samples of the immortalized mouse dendritic cell lines DC2.4As with the data sets, we also denote the analytical results from the first sequencing run of the nine samples (5 infected versus 4 uninfected) by R1, and those from the second sequencing run of the nine samples by R2. It should be remarked that R1 and R2 were run simultaneously without a time gap. In the following, we call these also the true technical replicates, in contrast to the artificial replicates.With each of the three strategies for generating artificial replicates, we produced 10 additional series of it p values and log2 fold changes, denoted in the following by FB1 to FB10 for the FASTQ-boostrap approach, by CB1 to CB10 for the column bootstrap approach, and by MO1 to MO10 for the mixed observations approach. The series of results also include those from the GO term enrichment analyses.p values as well as the absolute log2 fold changes of all genes after differential gene expression analysis  but the nine biological replicates where put as factor in the design matrix of DESeq2. Thus, the dispersion estimates reflect the technical variance between the replicates. The analysis yields that neither of the three strategies reflects the technical variance appropriate. The FB and the MO approach under-estimate the variance for most genes, while dispersions estimated by the CB approach are mostly larger than in the true replicates  or weighted mixed observations (MO). The FB appproach can be used to better judge uncertainty and reproducibility of genes selected as differentially expressed between two groups of samples. In particular, results from differential expression analysis on the FB generated replicates can be used to judge the robustness of ranking lists obtained from the true data. In addition, the FB approach can be used for the same purpose in subsequent gene set analysis such as gene-set enrichment analysis. One drawback of the FB approach is it’s high computational cost.Supplementary Information 1.Supplementary Information 2.Supplementary Information 3.


Regulators in gene regulatory networks (GRNs) are crucial for identifying cell states. However, GRN inference based on scRNA-seq data has several problems, including high dimensionality and sparsity, and requires more label data. Therefore, we propose a meta-learning GRN inference framework to identify regulatory factors. Specifically, meta-learning solves the parameter optimization problem caused by high-dimensional sparse data features. In addition, a few-shot solution was used to solve the problem of lack of label data. A structural equation model (SEM) was embedded in the model to identify important regulators. We integrated the parameter optimization strategy into the bi-level optimization to extract the feature consistent with GRN reasoning. This unique design makes our model robust to small-scale data. By studying the GRN inference task, we confirmed that the selected regulators were closely related to gene expression specificity. We further analyzed the GRN inferred to find the important regulators in cell type identification. Extensive experimental results showed that our model effectively captured the regulator in single-cell GRN inference. Finally, the visualization results verified the importance of the selected regulators for cell type recognition. The inference of gene regulatory networks (GRNs) allows for a better understanding of transcriptional regulation and how it works in cell-type identification. The models of GRN inference based on scRNA-seq achieved success in cancer treatment , the recRecently, deep learning has brought new solutions ,8 to sinMeta-learning is adept at solving parameter initialization and insufficient data labels when using small-scale samples . The metTo address the problems above, we proposed a meta-learning framework called MetaSEM  to inferIn this work, we verified the reliability of MetaSEM from the following aspects. We first compared the model performance of MetaSEM with several state-of-the-art methods. The experimental results showed that MetaSEM was significantly better than current methods in terms of EPR, AUPR, and AUROC. Next, to analyze the robustness of MetaSEMwe, we explored essential data features in single-cell data at different data scales. We then generated cell-type-specific GRNs on the bulk RNA-seq dataset for further research. Pearson correlation analysis of GRNs and gene expression data analysis indicated that the GRNs were cell-type-specific. Finally, the visualization of cell-type-specific GRNs within the HNSCC dataset demonstrated the regulators’ importance in identifying cell types.To verify the performance of MetaSEM, we compared the model with four methods: DeepSEM , DGRN 1, GENIE3 To further evaluate the performance of MetaSEM, we trained MetaSEM on the 500 gene datasets. It should be noted that the label data in 500 gene datasets was just as 30% of that in 1000 gene datasets. The results on the 500 gene datasets in In this section, we present the results of the robustness of MetaSEM in different data scales. We first set the sampling points according to the standard deviation of gene expression and sample size. Then, we built sub-datasets using mHSC-L, mHSC-E, and mHSC-GM. In addition, we also created another dataset of the same points through random selection as a reference. We used the same hyperparameters and five-fold cross-validation during model training. As shown in To verify whether MetaSEM captures specific information, we analyzed the inferred GRN and gene expression data. We generated the cell-type-specific GRN based on eight HNSCC sub-datasets. p-values obtain an independent t-test of Fibroblast and Endothelial subsets are 0.17 and 0.37, respectively. In contrast, the Mast cell and B cell datasets are 0.05 and 0.03, respectively. These results indicate that MetaSEM can capture the specificity information, which is essential for general GRN inference tasks.Given the phenomenon reflected in Next, we analyzed how MetaSEM extracts regulators. When sorting out the output of the GRN Layer, we found that the regulatory weight of some genes was very high. Therefore, we collected the regulatory weights of these genes. The boxplots in In this section, we present the relationship between the selected regulators and cell-type identification. In order to verify the importance of the regulatory weight of cell-type recognition. We collected the cell types’ SEM matrix for each gene’s regulatory weight. Then, all genes were divided into three equal parts according to the regulatory weight for cell clustering. Then, we used the Louvain and Leiden methods to cluster cell types. As shown in The BEELINE  dataset The head and neck squamous cell carcinoma (HNSCC) dataset  is used The datasets were stored in matrices with the vertical axis gene expression and the sample as the horizontal axis. One cell type corresponds to one matrix, and the values in the matrix represent the expression values of genes on the samples. Similar to DeepSEM’s training process , we tookThe proposed MetaSEM consisted of three parts: an encoder, a meta-decoder, and a GRN layer. (i) Encoder: This part encodes the gene expression data as the feature vector using a three-layer MLP. (ii) Meta-decoder: This part models the regulatory relationship via a two-layer MLP and uses the GRN layer to find the optimal pseudo-data labels. (iii) GRN Layer: With the SEM model, the GRN layer infers the gene regulatory relationships and transforms these relationships into pseudo-data labels.A represents the adjacency matrix resulting from modeling based on the SEM, We generalized the SEM as a GRN layer to model the conditional dependencies among random variables. We employed the meta-decoder to iterate the GRN Layer for extracting regulatory information. The final output of this module was an adjacency matrix representing GRN, and the elements within the matrix described directed edge weights. The GRN Layer iteration formula is as follows:X in batches. The data i first passes through a full connection layer to learn the gene expression feature [L represents the cross-entropy loss of the network, We built a decoder to capture the data feature from gene expression. The encoder reads the natural gene expression data  feature . Then, tY. The meta-decoder learns the gene regulatory relationships from the scRNA-seq. Then, the regulatory relationships are stored in the GRN Layer. Finally, the meta-decoder outputs the pseudo-data labels to represent the potential regulatory relationships. This procedure can improve the efficiency of the encoder by pseudo-data labels.We embedded the GRN Layer into a two-layer MLP to construct the meta-decoder. Unlike the attention mechanism ,32, we eWe employed a bi-Level optimization for hyperparameter optimization. Specifically, the bi-Level optimization calculates the model’s parameters twice and compares the gradients generated. If the gradients drop, the parameters are updated. If not, the second parameter calculation is executed again . The bi-Finally, the encoder and meta-decoder outputs are extracted for model loss calculation. The overall loss calculation of the model is expressed in Equation :(8)Loss=Formula (9) represents the method of the encoder to the output feature vector.MetaSEM is mainly implemented based on the PyTorch framework. The determined model configurations are as follows: the encoder is built from a three-layer MLP model, and the meta-decoder is built from a two-layer MLP with the GRN Layer. Hyperparameter optimization is a one-step gradient-updating strategy based on bi-Level optimization. We employed the grid search method to determine the model’s architecture and hyperparameters. The number of hidden-layer neurons set 128, and the batch size set 64. The encoder and meta-decoder optimization employed the Adam optimizer method. The learning rate is K edges to evaluate the performance of MetaSEM on three indices: early precision ratio (EPR), Area Under the precision-recall curve (AUPR), and area under the receiver-operating characteristic curve (AUROC), where the value of K is equal to the number of edges of the ground-truth GRN. The above metrics were developed as indices to evaluate model performance in previous studies [Following the BEELINE framework , we appl studies ,15.This paper proposed a new GRN inference algorithm based on meta-learning to analyze the importance of the selected regulators in cell-type identification. The MetaSEM learns the potential regulatory relationships from gene expression data. Moreover, meta-learning was also used to optimize the process of feature extraction. Extensive experiments in different single-cell datasets showed that MetaSEM performed better than several advanced computational methods in GRN inference tasks. Finally, by visualizing the inferred GRN, we systematically analyzed the importance of the data dispersion. We proved the importance of the selected regulator in cell-type identification. In the future, we intend to construct GRN by fusing scATAC-seq data and scRNA-seq data to explore the effect of GRNs on single cells.