Metabolic engineers endeavor to create a bio-based manufacturing industry using microbes to produce fuels, chemicals, and medicines. Plant natural products (PNPs) are historically challenging to produce and are ubiquitous in medicines, flavors, and fragrances. Engineering PNP pathways into new hosts requires finding or modifying a suitable host to accommodate the pathway, planning and implementing a biosynthetic route to the compound, and discovering or engineering enzymes for missing steps. In this review, we describe recent developments in metabolic engineering at the level of host, pathway, and enzyme, and discuss how the field is approaching ever more complex biosynthetic opportunities. Engineering microbial cell factories for the production of useful plant natural products (PNPs) is a resource-conserving and environmentally-friendly synthesis route. Here, the authors review recent developments that enable engineering of hosts, pathways, and enzymes to make PNPs and PNP derivatives. The field of metabolic engineering endeavors to create a green manufacturing industry based on bioproduction of commodity chemicals in cell factories. Plant natural products (PNPs) are especially important targets because of their utility as flavors, fragrances, and medicines, but can be challenging to synthesize due to stereochemical complexity. PNPs are produced via specialized plant metabolism involving numerous enzymes from diverse classes that enzymatically transform central metabolites into secondary metabolite compounds such as the analgesic morphine and antimalarial artemisinin. Many PNPs are obtained from processed plant biomass, requiring substantial land, water, and time investment, and which introduces insecurity in supply chains due to variability in crop yields resulting from pests or extreme weather. Furthermore, intermediates in a PNP biosynthetic pathway are often unavailable from the host plant, thus complicating efforts to produce novel derivatives of the PNP of interest. Microbial production of PNPs can overcome these challenges by enabling (1) on-demand production capabilities associated with microbial cells, (2) scalable and controlled production in fermentation facilities, and (3) the capacity to produce PNPs and PNP intermediates at higher purity or yield than those provided by the native plant host. In addition, microbial production of PNPs can serve as a discovery platform to synthesize novel derivatives of PNPs and gain insight into enzymes involved in plant secondary metabolism.1. At the level of the host, DBT includes selection and engineering of the host to overproduce PNP-precursor metabolites in sufficient quantity; at the pathway level, a biosynthetic route to produce the PNP is determined and candidate enzymes are tested or discovered; and at the enzyme level, protein engineering may be warranted to improve function or produce derivative compounds , and are amenable to production scale-up. Other microorganisms are often employed to purposes which evolution has made them especially well-suited: Streptomyces is often used for the production of antibiotics originally derived from Streptomyces species10; Corynebacterium glutamicum is widely used for the high-titer production of amino acids11; Yarrowia lipolytica is frequently employed when using lipids as a substrate12. Yet for most applications involving the production of PNPs in a microbial host, E. coli or S. cerevisiae is employed.A first choice of host for production of PNPs may be plant cells, where plant specific subcellular compartments and protein processing are conserved, a topic recently reviewed elsewhereE. coli or S. cerevisiae. Distinct advantages of S. cerevisiae are its ease of genomic integration, owing to a high rate of homology directed recombination, and that as a eukaryote yeast contains many organelles found in plants. Some enzymes from PNP biosynthesis pathways, such as cytochrome P450s, are transmembrane proteins and require the presence of an appropriate membrane, such as the endoplasmic reticulum (ER), for proper anchoring and folding. This potential roadblock was demonstrated during the Semi-synthetic Artemisinin Project, a landmark achievement in metabolic engineering in which S. cerevisiae was engineered to produce high titers of artemisinic acid, a precursor to the important antimalarial artemisinin. In this project, both E. coli and S. cerevisiae were considered as potential hosts, and while impressive titers of the intermediate amorphadiene (25 g/L) were achieved in E. coli13, the subsequent step in the pathway is carried out by P450AMO, a plant cytochrome P450. High activity of this enzyme could not be attained in E. coli necessitating a switch to production in S. cerevisiae14. While strategies exist to modify transmembrane proteins for function in the cytosol15, using S. cerevisiae as a host for pathways containing transmembrane proteins avoids the added labor necessary to modify those enzymes. Furthermore, S. cerevisiae contains cellular microcompartments  that can be used to mimic the subcellular localization employed in PNP biosynthesis in plants16. Conversely, E. coli has a doubling time that is 3–4 times shorter than S. cerevisiae, is well suited to very high expression of enzymes, and has a different profile of native metabolites available compared to S. cerevisiae. For example, the presence of a native pathway for certain isoprenoid compounds was used to engineer E. coli strains with 2,400-fold higher production of taxadiene (a precursor to the PNP drug taxol) compared to strains of S. cerevisiae17 engineered for taxadiene production.Thus, the most immediate question for a metabolic engineer seeking to produce a compound in a heterologous host is often whether to use 21. Merits of this approach include reducing burden on the host from the heterologous pathway, the ability to utilize the species most suited to expression of specific enzymes in the pathway, and modularity associated with being able to mix pathways by growing distinct strains together. In one example, benzylisoquinoline alkaloids (BIAs) were synthesized in an E. coli and S. cerevisiae co-culture system20. E. coli were engineered for biosynthesis of the branchpoint intermediate (S)-reticuline, and S. cerevisiae strains were engineered to express membrane-bound P450 enzymes that derivatized (S)-reticuline to other PNPs. In another example, high titers of an anthocyanin PNP were achieved by splitting the metabolic burden of the pathway across four E. coli strains which were co-cultured19. Limitations of co-cultures are pathway specific and include inefficiencies in the transport and/or diffusion of intermediate metabolites between cells in the co-culture and the need to balance growth of multiple hosts in as single culture, which may differ in optimal growth conditions and rates.One additional avenue when choosing a host organism for PNP biosynthesis is to utilize multiple organisms in a co-culture with components of a metabolic pathway split between distinct organisms of the same or different speciesFollowing selection of a host species, engineering the host to increase titers of native metabolites that are biosynthetic precursors to the product of interest can greatly facilitate downstream production of PNP molecules. The core metabolic networks of model organisms are well-characterized and can be used to guide overexpression and knockout modifications for overproduction of central metabolite precursors and to address common challenges . One of the advantages of biosynthesis over chemical synthesis is how readily biosynthetic strains are distributed; once a strain has been engineered to produce a compound, researchers looking to expand on that work in the future need not repeat tedious syntheses of starting material.E. coli and S. cerevisiae that overproduce alkaloids, fatty acids, terpenes, and other valuable compound classes have been engineered (Table (S)-reticuline23 have enabled microbial biosynthesis of a wide range of BIAs produced by Papaver somniferum (opium poppy), including morphine24 and noscapine25. Likewise, strictosidine producing strains26 provide a key branch point metabolite for the biosynthesis of monoindole alkaloids (MIAs), which include vincristine, ibogaine, yohimbine, and thousands of others.Strains of ed Table . PlatforE. coli which utilizes lignocellulosic biomass27, an inexpensive waste product from agriculture and forestry, in place of expensive refined sugars. In another example, an enzyme was designed that allows for assimilation of formate into central metabolism28, potentially allowing the biosynthesis of medicines and commodity chemicals from formate, which is expected to be abundantly available from electrochemical reduction of CO2. Lastly, researchers generated a strain of E. coli that can produce its own biomass from CO2 via photosynthesis29. Although there is significant interest in utilizing natural photosynthesizing microorganisms  for the production of PNPs30, it could be advantageous to engineer the ability to fix CO2 in well studied, genetically tractable industrial microorganisms such as E. coli and S. cerevisiae. While the aforementioned strains have not yet been used for the production of PNPs, the other PNP-producing strains discussed throughout this review could potentially be integrated into these platforms to produce complex PNPs de novo from agricultural waste, formate produced with renewable energy31, or directly from atmospheric CO2. This principle has been demonstrated through the engineering of E. coli to utilize the one-carbon feedstock methanol and ultimately convert it into the flavanoid naringenin32. Such strategies could support more sustainable bioprocesses for producing increasingly diverse products, including PNPs, at industrial scale.A platform strain can be useful not only because it produces a valuable starting material, but also because the means of production of said starting material are particularly inexpensive, sustainable, or offer easy handling for the researcher or industrial producer. This is demonstrated by the engineering of an efficient simultaneous saccharification and co-fermentation (SSCF) strain for bioethanol production in tour de force33 combined all of these techniques to reprogram yeast central metabolism to overproduce acetyl-CoA for isoprenoid and fatty acid biosynthesis - molecules which are a starting point for many PNPs such as the antimalarial artemisinin. A model of the yeast reaction stoichiometries for acetyl-CoA, redox cofactors, and sugar was used to determine a more favorable reaction stoichiometry, which was defined as having a reduced ATP requirement, reduced loss of carbon to side reactions, and improved pathway redox balance. The optimal acetyl-CoA stoichiometry was implemented by augmenting acetyl-CoA biosynthesis with expression of four enzymes involved in acetyl-CoA biosynthesis in other organisms, allowing the yeast to produce 25% more of the isoprenoid farnesene with an equal supply of sugar while requiring less oxygen, an important consideration for oxygen-constrained industrial fermentation environments.After selection of a host or existing platform strain, the supply of biosynthetic precursors may be enhanced by modifications to the host, such as gene deletions, swapping of endogenous enzymes with more active homologues, or overexpression of endogenous metabolic genes Fig. . A recenp-coumaric acid overproduction, from which many PNPs including some alkaloids, polyphenols, and flavonoids are derived, has also been pursued in the context of E. coli and S. cerevisiae. For example, researchers engineered yeast producing 1.9 g/L of p-coumaric acid through a combination of six genetic modifications to yeast native metabolism. These included engineering feedback-resistant enzymes, over-expressing enzymes at bottlenecks, and removing competing side pathways34.Optimization for tyrosine and 26. Finding that geraniol, an intermediate in strictosidine biosynthesis, was metabolized by the yeast through esterification, deletions were made to ATF1 and OYE2 which reduced undesired host interactions and resulted in a 6-fold increase in strictosidine production.Deletion of competing or undesired side pathways in the host is a common strategy to increase precursor titers Fig. . In work35. Deletion of pyruvate decarboxylase genes  involved in alcoholic fermentation resulted in strains unable to grow on glucose as a carbon source. Adaptive laboratory evolution was applied to evolve FFA producing strains lacking ethanol fermentation for growth on glucose by gradually shifting the carbon source from ethanol to glucose over 200 generations. New methods like SCRaMbLE have enabled inducible control of host genetic variation36. SCRaMbLE utilizes a synthetic yeast chromosome V with recombination sites introduced in all non-essential genes such that when recombination is induced these genes are shuffled within chromosome V. SCRaMbLE was applied in S. cerevisiae and shown to improve host strain background for improved production of violacein, penicillin, and utilization of xylose as a carbon source.Finally, evolution has emerged as a powerful approach for host optimization, although it has not yet been directly applied to PNP biosynthesis. In the aforementioned work on altering yeast metabolism from alcoholic fermentation to lipogenesis, researchers also employed laboratory evolution methods to improve lipogenic growth on glucoseFollowing selection of a suitable host, a route to the desired PNP can be planned and implemented. A candidate pathway is first outlined through selection of stepwise chemical intermediates leading from host metabolism to the target compound, followed by selection of enzymes to carry out each specified reaction. For certain PNPs, detailed knowledge of the native biosynthetic pathway is available and can be used to outline all intermediates and enzymes in a pathway, facilitating pathway engineering into a heterologous host. However, such detailed knowledge can require years or even decades of dedicated research in planta and is frequently unavailable or incomplete. In such cases, candidate pathway design, enzyme selection, and pathway testing all offer distinct challenges which are discussed in the following sections.Literature on a given PNP biosynthetic route can be instrumental to outlining a pathway, although even for well-studied PNPs there are often gaps in our knowledge. One way to overcome the restriction of needing plant biochemical data for each enzymatic step is to use an approach agnostic to the natural product in question. When a reaction path to a chemical entity is unknown, retrosynthetic analysis can be used such that the target molecule is transformed into simpler precursor structures without making assumptions about starting material availability. Resulting precursors are in turn transformed into simpler structures until available starting constituents are reached. By breaking a target molecule into potential precursors, it is then possible to select enzymes which interconvert in the other direction.37 to propose biosynthetic routes to the target. Of ten available retrosynthesis-based pathway design tools38, only RetroPath39 has been experimentally tested. RetroPath takes starting compounds, a target, and reaction rules to generate potential pathways and was experimentally validated on the design of biosynthetic routes to pinocembrin40, a flavonoid four enzymatic steps from E. coli central metabolism. RetroPath narrowed down a list of nine million in silico pathways to twelve top-ranked candidates, with one providing 24 mg/L pinocembrin after construction and optimization in E. coli. Notably, RetroPath and similar tools such as BNICE.ch24 only consider the type of reaction occurring when considering enzymatic matches. If the substrate of the desired reaction is very different from that of the known reaction to which it is being compared, ranking the results by some measure of substrate similarly, such as Tanimoto distance, might be advantageous41.Retrosynthetic pathway design deconstructs a PNP one step at a time and utilizes reaction/enzyme pairs from databases such as MetaCyc42. The 10 molecules were a mix of PNPs (carvone and vincristine) and non-PNPs; the fungal metabolite epicolactone provides an example of a retrosynthetic approach that could be applied to PNPs to identify potential pathways. The genomic sequence of the native producer of epicolactone was unavailable and so the researchers based their enzymatic retrosynthesis on a previously developed eight-step non-enzymatic chemical synthesis43. Enzyme classes were assigned to each reaction manually, guided by literature and pathway databases. Multiple enzymes were identified for each of the eight steps based on reaction type, and to narrow down the candidates, enzyme hits were limited to tropolone-like biosynthetic gene clusters identified from the biosynthetic gene cluster databases MIBiG44 and antiSMASH45. However, no pathways were experimentally validated within the 90 day time frame.Retrosynthesis can also be performed manually, without the aid of automated tools. To characterize the rapidity of heterologous biosynthesis for the production of valuable compounds, a group of researchers recently performed a pressure test to produce 10 molecules of interest in 90 days46. A key question left unanswered for retrosynthetic methods is what strategies can be used for the design of long pathways when some or many steps are non-functional during in vivo testing. This is especially important for the long pathways common in plant secondary metabolism. If automated retrosynthesis tools are to gain more use for PNP biosynthesis, it would be of benefit if not only hypothetical pathways are generated, but also modules containing several enzymatic steps for orthogonal testing, as discussed in the following section “Strategies for the construction of candidate pathways”.Nature has developed a limited set of biosynthetic tools; a retrosynthetic scheme might envision chemical transformations which no known enzyme class is able to carry out, and even enzymes which are known to perform the desired chemistry may only do so on a very different substrate. Enzyme evolution to alter substrate scope is still a time-consuming endeavor, and designing enzymes capable of entirely new chemistries has had very limited demonstration thus far47 and the 1KP Project48, which leverage genomic information to prioritize biosynthetic gene clusters and enzymes for pathway discovery. Comparison of omics data between species which either produce or lack specific compounds can also help elucidate which enzymes are important for their biosyntheses  from the Erigeron breviscapus genome and divided them into 15 gene families. Previous work allowed the researchers to narrow the list from 83 to one likely candidate in the UGT88 family50. The function of this lone candidate was validated in vitro and then introduced into a yeast strain producing apigenin, the substrate of UDPGT, resulting in an engineered strain that produced apigenin-7-O-glucuronide. The P450 enzyme was discovered by narrowing down 312 putative P450s in the E. breviscapus genome to a list of 134 candidates by comparison with P450s from non-breviscapine producing plant species. Of 134 candidates, 36 were selected and screened for activity using the aforementioned strain producing apigenin-7-O-glucuronide. One P450, CYP706X, resulted in a new peak by HPLC matching the expected product scutellarin, and was subsequently verified via mass spectrometry. This work highlights the ability to identify one enzyme, incorporate it into an engineered strain, and use that new strain to discovery enzymes that perform a subsequent reaction, thus leveraging intermediate strains developed over the course of a project. As DNA synthesis costs continue to drop, one can envision simply synthesizing and testing an entire panel of candidate genes without needing to computationally prioritize the list beforehand.Demonstrating this approach, two enzymes involved in the biosynthesis of breviscapine flavonoids were discovered entirely via transcriptomic and genomic analysis and subsequently incorporated into an engineered biosynthetic pathway in Papaver somniferum. During morphinan alkaloid biosynthesis, (S)-reticuline is converted to (R)-reticuline by an epimerase. In 2015, three different groups reported the discovery of a two-component epimerase, consisting of a reductase and oxidase, using distinct discovery strategies53. The computational and synthetic biology driven approach taken by one team of researchers53 relied on the 1KP Project48 and PhytoMetaSyn25 databases to search for enzymes similar to a codeinone reductase that had been previously identified. Without accessing plant material, epimerase candidates were synthesized and expressed in an engineered yeast strain producing (S)-reticuline, thus affording conversion to (R)-reticuline and ultimately enabling de novo biosynthesis of opioids in yeast.A similar strategy was employed to discover a key enzyme required for the biosynthesis of morphine in Computational approaches to enzyme discovery based on enzyme class, expression, or phylogenetic comparison require a putative enzyme class assignment and/or detailed genomic or transcriptomic data. In the absence of this information, or when the exact nature of the reaction(s) being carried out is unclear, enzyme discovery can be performed with native plant material. This approach is especially important when a reaction involves unique metabolites and/or a catalytic mechanism that is not well represented in enzyme databases.56. The active protein fraction can then be identified using protein-mass spectrometry (protein-MS) followed by transcriptomic or genomic mapping -salutaridinol-7-O-acetate (7SOA) to thebaine can occur spontaneously57, but the potential role of an unidentified enzyme in P. somniferum had been hypothesized58. When latex extract from opium poppy was added to 7SOA an increase in thebaine was measured, indicative of enzymatic activity . Because this enzyme catalyzed a reaction previously unknown to biocatalysis, a transcriptomic search based on homologous enzymes was not possible. To isolate the enzyme, protein chromatography was used to enrich fractions with THS activity. Six major proteins were present in the THS active fractions as revealed by protein-MS and comparison with predicted translation products of opium poppy. Each candidate gene was expressed in E. coli and tested in vitro, but only one (Bet v1–1) displayed THS activity. Subsequently, the THS variant was expressed in an engineered yeast strain, demonstrating an improved biosynthetic route from fed norlaudanosoline to thebaine.One powerful approach to discovery is to isolate an unknown enzyme from native plant material via protein fractionation and functional assaying Fig. . ResearcE. coli based production of indican56, a water soluble indigo precursor with potential application for production of sustainable indigo dye. Purification of UGIG from leaves of Persicaria tinctoria, one of the highest yielding plants for indican, led to identification of a UGIG gene. The purified UGIG was analyzed via protein-MS and fragments were matched to transcriptome-predicted sequences. E. coli was chosen as a production host based on a prior platform for production of the precursor indoxyl, and UGIG expression in indoxyl producing E. coli led to accumulation of indican, validating the role of the discovered enzyme.Another team of researchers used a similar method to discover a UDP-glucose:indoxyl glucosyltransferase (UGIG) for 54. This approach was used to identify two unknown enzymes in a seven-step pathway from the MIA tabersonine to the anticancer drug precursor vindoline in Catharanthus roseus59. Researchers used tissue specific qPCR to determine that only the two terminal steps of the vindoline pathway occur outside of the leaf epidermis. To discover earlier genes in the pathway responsible for a net hydration of the substrate, candidate genes suspected to possess hydratase activity and preferentially expressed in leaf epidermis were queried and two candidate genes were selected. VIGS was used to validate the function of a new oxidase (T3O) and reductase (T3R) enzyme in planta and recombinant enzyme assay showed that product formation was only possible via the coupled action of T3O and T3R. The researchers then used the discovered enzymes to complete a seven-gene pathway in yeast producing vindoline from tabersonine, further validating the functions of T3O and T3R and providing a platform for microbial vindoline production. VIGS has also been used to characterize enzyme activities involved in the biosynthesis of noscapine from P. somniferum60 and the etoposide aglycone from Podophyllum hexandrum60 among many other examples.Putative enzyme function can also be confirmed using virus-induced gene silencing (VIGS) in planta61; these include discussions of methods for rapid multi-gene integration62, gene editing methodologies63, and techniques for combinatorial enzyme expression64. However, strategies specifically for the organization and testing of long metabolic pathways (defined here as >5 heterologous genes) have not been clearly defined. Pathway planning and enzyme identification, as described in the preceding sections, are useful for selecting enzyme candidates, but transitioning from an outlined pathway to a functional biosynthetic route expressed in a heterologous host is non-trivial42. Challenges include the proper expression of candidate enzymes, which may not be functional when expressed in a heterologous host, and the assembly and validation of multi-enzyme pathways when chemical intermediates are not commercially available.Techniques generally used to construct pathways in heterologous hosts are discussed extensively elsewhereOne approach involves breaking a pathway down into biosynthetic modules, where each module’s set of enzymes can be tested and optimized independently in a heterologous host and only combined once validated. Each module ideally begins and concludes with substrates that are commercially available, and steps requiring enzyme discovery are isolated into individual modules, such that a single module is not contingent upon multiple unknown steps Fig. .(S)-reticuline production through the use of four genetic modules containing 17 biosynthetic enzymes53. The modules focused on overproduction of pathway precursors, cofactor recycling enzymes, production of the intermediate (S)-norcoclaurine from native metabolism, and conversion of (S)-norcoclaurine to (S)-reticuline. This genetic design allowed for independent analysis of each module’s role in the pathway and any limitations. In the same work, a fifth module for thebaine biosynthesis was later designed, possessing additional enzymes that were discovered and engineered independently from the first four modules by feeding (S)-reticuline. Ultimately, module five was incorporated into the (S)-reticuline producing strain for fully de novo thebaine biosynthesis. A similar modular strategy was employed for the development of other long biosynthetic pathways in heterologous hosts, including for the production of the alkaloids noscapine65, sanguinarine67, strictosidine26, and breviscapine flavonoids49.In one example of the utility of clearly defined modules, researchers engineered yeast strains for 68—collections of wet-lab robotics and software designed to standardize the synthesis, assembly, and testing of DNA parts in microbes. It is unclear if the enzyme discovery components required for some long biosynthetic pathways can be automated, given that discovery methods are frequently tailored to an individual pathway. To date, existing foundry-based approaches have only afforded short pathways (<5 enzymes) or pathways which were already validated69.Alongside the pathway specific approach described above, a new frontier for pathway construction is the use of highly automated foundriesWhen introducing enzymes into a heterologous host, an enzyme may function suboptimally or not at all for reasons that include the new host context  or the new chemical context . Sub-optimal function of a heterologous enzyme may result in bottlenecks of carbon flux from central metabolism into the PNP pathway. Many of these modes of failure can be alleviated through enzyme engineering.70. Neomorphine accumulation results from activity of codeinone reductase (COR) on the direct product of T6ODM, neopinone, prior to a spontaneous double bond shift. ER localization tags were fused to COR, thereby sequestering it in the ER and allowing more time for cytosolic neopinone to spontaneously rearrange to codeinone before interacting with COR. The localization strategy ultimately increased morphine titers by sevenfold while decreasing production of the undesired intermediate neomorphine by fourfold. Engineering spatial organization of enzymes can also be accomplished through the use of synthetic scaffolds, which can increase product titers through enzyme clustering or increased local substrate concentration. In one example, researchers constructed a three enzyme pathway in E. coli from acetyl-CoA to the intermediate mevalonate, a precursor to the important PNP artemisinin71. Mevalonate titers were improved by 77-fold to 5 mM by using SHL/SH3 association domains to cluster the three enzymes in the pathway. RNA-based scaffolds have also been applied to cluster enzymes on RNA-scaffolds using RNA-binding domains fused to enzymes72.In plants, localization can cluster pathway enzymes, separate reaction intermediates in the pathway, and provide specific pH or substrate conditions. Localization can be a powerful tool when expressing plant enzymes in microbial contexts for the same reasons. In engineering a heterologous yeast strain for the production of morphine and its semi-synthetic derivatives, researchers observed substantial accumulation of the undesired side product neomorphine53. Western blotting indicated yeast-expressed SalSyn was present as three distinct molecular weights resulting from improper N-linked-glycosylation, indicative of improper localization to the lumen of the ER instead of the ER outer membrane. Protein engineering corrected the improper N-terminal sorting of SalSyn, allowing it to localize to the ER outer membrane and preventing N-linked glycosylation. The engineered enzyme improved conversion of (R)-reticuline to salutaridine by sixfold. In another example, researchers engineered brewer’s yeast for biosynthesis of aromatic monoterpene molecules  native to the hop plant and important components to the flavor of beer73. In plants, monoterpene biosynthesis occurs in chloroplasts and plant monoterpene synthases typically contain N-terminal plastid targeting sequences (PTSs) of 20–80 amino acids which are cleaved to yield mature protein. In the absence of PTS cleavage, enzyme function is decreased. The researchers tested truncated linalool synthases using bioinformatic and structural information to predict the PTS sites for removal. In one instance, truncation of the PTS motif resulted in a 15-fold improvement in linalool titers. In the same work, additional enzyme engineering was carried out on HMG-CoA reductase, a key rate-limiting step in the pathway for monoterpene biosynthesis. HMG-CoA reductase is controlled by an allosteric domain which responds to product accumulation by inhibiting enzyme function. The researchers truncated the yeast HMGR protein removing an inhibitory domain, thereby increasing flux towards end products.Another common problem faced when expressing plant enzymes in a new context is host misprocessing of post-translational modifications or signal peptides. In the course of engineering the biosynthesis of opioids in yeast, researchers encountered low activity in the enzyme salutaridine synthetase 74.In addition to the rational modifications discussed, additional enzyme engineering may be required for increasing yield or enabling an enzyme to act on a non-native substrate. The latter might be encountered for enzymes candidates selected through RetroPath or similar computational approaches. Techniques for engineering enzymes with higher activity or for promiscuity toward non-native substrates is a subject covered in the next section and detailed in other recent reviews of enzyme engineering75. Derivatization can enhance biological properties of a compound, but PNPs are often synthetically complex, precluding practical syntheses of novel derivatives. With an established heterologous biosynthetic pathway, one can readily replace, add, or remove enzymes or feed in alternative starting materials to synthesize functionalized derivatives. The following sections discuss strategies by which an established PNP-producing engineered microorganism can be leveraged to produce new-to-nature molecules. Each strategy is potentially complementary and in theory can afford a wealth of novel chemical entities from a single starting heterologous pathway -reticuline, demonstrating the flexibility of some enzymes to accept derivatives of their native substrates76.In addition to the ease of genetic modification of industrial microorganisms, including 77 and the enzymes for (S)-reticuline epimerization to (R)-reticuline51, or in vivo, as was done for the MIA biosynthetic enzyme strictosidine synthase78. In the former cases, the enzyme in question was purified and reacted with derivatives of the native substrate in vitro, thus elucidating which substrates are likely to be accepted in vivo. A cell-free system was developed to assay the ability of prenyltransferases to produce PNPs and novel derivatives and was demonstrated with cannabinoids from Cannabis sativa; by feeding in divarinic acid in place of olivetolic acid, cannabinoids that are typically minor products in planta were produced in high titers and prenyltransferase mutants were quickly assayed for substrate selectivity79. Promiscuity is also readily probed in vivo directly in the pathway context. Novel isoflavonoids have been produced through feeding of flavanones to engineered yeast80, while novel flavonoids and stilbenes were similarly generated from carboxylic acids fed to engineered E. coli75. In their work on the de novo biosynthesis of noscapine, researchers showed that several halogenated derivatives of the early intermediate tyrosine were accepted by seven downstream enzymes in the pathway, affording halogenated derivatives of pathway intermediates up to (S)-reticuline5. The substitution of a hydrogen atom for a halogen is relatively sterically conservative, but is a ubiquitous modification in medicinal chemistry, with nearly a quarter of all pharmaceuticals containing at least one halogen81. However, the titer of the halogenated reticuline derivatives was either too low for halogenated derivatives of further downstream intermediates to be observed, or the subsequent enzyme, berberine bridge enzyme, possesses too narrow a substrate scope. When derivatization occurs at the terminus of a pathway, enzyme promiscuity may not be required. For example, researchers produced novel betalain pigments in yeast by feeding diverse amine scaffolds82. A yeast strain was engineered for betalamic acid production, which then spontaneously condensed with the fed primary and secondary amines resulting in new-to-nature pigments.However, substrates which differ more radically from the native substrate are less likely to be accepted at high efficiency, though enzymes differ greatly in their promiscuities. Promiscuity can be assayed in vitro, as was done for the BIA biosynthetic enzyme norcoclaurine synthaseS. cerevisiae, expressing one cytochrome P450, CYP82Y1, in the absence of the preceding enzyme, an N-methyltransferase, afforded 1-hydroxycanadine in place of the usual product, 1-hydroxy-N-methylcanadine65. Similarly, swapping CYP82Y1 with CYP82X2, which is downstream in the native biosynthetic pathway, resulted in the production of N-methylophiocarpine, an isomer of the native product, 1-hydroxy-N-methylcanadine. Neither of these two products had previously been identified in the native plant host, P. somniferum65. The combinatorial space around terpene biosynthetic pathways has been similarly probed using transient expression in Nicotiana benthamiana to generate novel sesquiterpenoids derived from the parthenolide biosynthetic pathway from feverfew (Tanacetum parthenium)83. This space can be further expanded through the introduction of analogues of the native pathway enzymes. Researchers reconstituted the rebeccamycin biosynthetic pathway from the soil bacterium Lechevalieria aerocolonigenes in Streptomyces albus, which natively contains RebH, a tryptophan 7-halogenase84. By exchanging RebH with pyrH and thal, a tryptophan 5-halogenase and a tryptophan 6-halogenase, respectively, and expressing the other pathway genes in different combinations, a total of 32 different compounds were produced. A similar strategy could be applied to PNPs using S. cerevisiae or E. coli as a host to rapidly explore structure space around the native PNP.Once a heterologous biosynthetic pathway is established, the enzymes in that pathway and/or analogues of those enzymes can be employed in different combinations to afford distinct products. In this way, no unnatural substrates or novel enzymes need be introduced; all of the necessary tools to make new products are present from the initial engineering effort. For example, in biosynthesis of noscapine from canadine in 85, halogenases86, and aminotransferases87, to name only a few classes. Through similar efforts, type III polyketide synthases have been engineered to produce novel and often larger products by accepting novel substrates and performing additional chain elongation steps prior to cyclization91. Given the large effort traditionally required for protein engineering and metabolic engineering, though, relatively few examples in which the two techniques are successfully combined have been reported. In light of advances that have accelerated both and furnished a wealth of engineered enzymes and pathways with which to work, we can expect to see the two utilized in concert increasingly in the future. For example, a recent perspective discussed valuable drugs that could be produced by leveraging opioid biosynthetic S. cerevisiae strains with engineered enzymes and subsequent semi-synthesis92; these include the pharmaceuticals cisatracurium, levorphanol, and butorphanol.The chemical space accessed by heterologous pathways rebuilt from plants can be further expanded by addition of new enzymes to the pathway. In this way, a natural product can be directly transformed via halogenation, hydroxylation, methylation, prenylation, or any other chemistry available via an enzyme that will accept the natural product as a substrate. Since natural products are often large and complex, an enzyme which natively performs the desired transformation at high efficiency may not be available. However, protein engineering can be used to expand the substrate scope of existing enzymes to accept larger substrates, as has been demonstrated for cytochromes P450E. coli42, researchers identified 21 additional genes known to modify the bisindole core and predicted that combinatorial expression of those enzymes could access to a total of 540 bisindole derivatives, 98% of which have not yet been reported in PubChem. Early concrete demonstrations of addition of derivatizing enzymes to metabolic pathways mostly utilize pathways in their native host, rather than in heterologous hosts. For example, researchers added a halogenase to the marine bacterium Streptomyces coeruleorubidus, which natively produces pacidamycin, in order to produce the new-to-nature derivative chloropacidamycin93. The researchers used the introduced chlorine as a synthetic handle for cross-coupling reactions to make a range of novel products. While the two preceding examples are of bacterial natural products, the strategies could readily be applied to PNP pathways expressed in microbial hosts. In a notable achievement, researchers reported the integration of two halogenases into the medicinal plant Catharanthus roseus and observed that chlorinated catharanthine alkaloids were produced94. Both of the preceding examples rely on the halogenation of tryptophan, an early intermediate in the pathway; in C. roseus, 7-chlorotryptophan accumulation was observed in the plant and suspected to adversely affect the growth rate. To alleviate this, researchers engineered RebH to act on tryptamine, the immediate downstream metabolite in the biosynthetic pathway, rather than on tryptophan95; integration of this engineered RebH variant showed no accumulation of 7-chlorotryptophan. More recently, researchers engineered E. coli to produce resveratrol, a stilbenoid produced by several plants, and then added the halogenase Rdc2 to produce 2-chlororesveratrol96. This work utilized a heterologous pathway in an industrial microorganism with an additional enzyme introduced to produce a new-to-nature natural product derivative, albeit with a relatively simple chemical structure. Given the dramatically more complex PNPs that have been biosynthesized heterologously in recent years, future demonstrations of non-native enzyme incorporation will furnish increasingly synthetically complex novel products.To date, metabolic pathways have been more frequently modified with natural enzymes. For example, in work on the combinatorial biosynthesis of the bacterial metabolite rebeccamycin in The past 20 years of PNP metabolic engineering have seen increasingly sophisticated pathway engineering, with engineered pathways composed of two to seven enzymes in the early 2000s progressing to pathways containing 20 or more enzymes at present. The more enzymatic steps in a heterologous pathway, the more formidable the challenge for construction of the pathway, discovery of the requisite enzymatic components, and overcoming interdependencies introduced between the many enzymatic steps. Taken at face value, this should mean that long, complex PNP biosynthetic pathways take significantly longer to engineer than simple pathways. However, recent years have seen the rapid implementation of long pathways as a result of advances in DNA synthesis, sequencing, and genome engineering. These enabling technologies, along with the emergence of PNP platform strains, have allowed the discovery and engineering of increasingly long PNP pathways.Following these trends, the field of metabolic engineering leverages sequencing and synthesis to more rapidly discover pathways and enzymes and engineer those pathways into metabolic hosts. Decreasing costs of high-throughput sequencing continue to allow comprehensive profiling of plant genomes and transcriptomes, providing plentiful putative enzyme targets that can be mined via comparison with existing databases of enzymes of known function. Inexpensive DNA synthesis enables wholesale synthesis of dozens of predicted enzymes for any given step in a pathway. This approach has the advantage that hundreds of hypothesized enzymes can be tested for a given step as opposed to testing individual enzymatic hypotheses in planta. Platform strains producing important metabolites for a given specialized PNP pathway will be used to screen single enzymes and combinations of predicted enzymes to reconstitute partial metabolic pathways. Importantly, once a platform strain for a given intermediate is made, the discovery and assembly of downstream pathways are greatly facilitated.99, and early examples have demonstrated their application in enzyme evolution101. As methods for developing biosensors improve we anticipate that they will be increasingly employed in enzyme engineering to explore wider ranges of sequence space more rapidly than traditional screening methods allow. Furthermore, we expect protein engineering to be used to solve a wider range of problems in the future—not only to adapt existing enzymes to their new conditions, but also to develop completely novel activities which fill in gaps in biosynthetic pathways or expand pathways in new directions for the production of novel PNP derivatives. This will allow existing and future heterologous biosynthetic pathways to be leveraged for the production of innumerable valuable and novel chemical entities. As metabolic engineering accelerates towards increasingly complex and tailored pathways in the coming years, we expect protein engineering to become an increasingly dominant force for the production of known and novel molecules on both the research and industrial scales.Throughout this review, we have highlighted instances in which protein engineering was employed to solve challenges  encountered during the reconstruction of metabolic pathways in heterologous hosts. As the design-build-test cycles associated with both metabolic engineering and protein engineering have been greatly accelerated in recent years, we expect to see protein engineering employed more frequently during the construction of heterologous metabolic pathways. Although these cycles have accelerated, engineering enzymes for altered small-molecule production still relies on screening, typically via LC or GC, rather than selection, resulting in a disproportionate amount of project time required for sample analysis, even for limited library sizes. One exciting means to engineer selections for small molecules is through the use of genetically-encoded biosensors linking concentration of a compound of interest to an output such as fluorescence-protein expression or cell fitness. Protein and RNA biosensors can be engineered to recognize a range of small molecules and control genetic output accordingly


Signaling pathways are the key biological mechanisms that transduce extracellular signals to affect transcription factor mediated gene regulation within cells. A number of computational methods have been developed to identify the topological structure of a specific signaling pathway using protein-protein interaction data, but they are not designed for identifying active signaling pathways in an unbiased manner. On the other hand, there are statistical methods based on gene sets or pathway data that can prioritize likely active signaling pathways, but they do not make full use of active pathway structure that link receptor, kinases and downstream transcription factors.Here, we present a method to simultaneously predict the set of active signaling pathways, together with their pathway structure, by integrating protein-protein interaction network and gene expression data. We evaluated the capacity for our method to predict active signaling pathways for dental epithelial cells, ocular lens epithelial cells, human pluripotent stem cell-derived lens epithelial cells, and lens fiber cells. This analysis showed our approach could identify all the known active pathways that are associated with tooth formation and lens development.https://github.com/VCCRI/SPAGI/.The results suggest that SPAGI can be a useful approach to identify the potential active signaling pathways given a gene expression profile. Our method is implemented as an open source R package, available via The online version of this article (10.1186/s12918-018-0655-x) contains supplementary material, which is available to authorized users. A key role cell signaling  plays within biological systems is to relay extracellular signals in order to regulate intracellular gene expression. The signal transduction process is typically initiated by the binding of a ligand to a membrane-bound receptor, which triggers a cascade of intercellular signaling activities through multiple kinases - ultimately impacting on how transcription factors regulate downstream gene expression . The cooActivation of different signaling pathways can lead to numerous physiological or cellular responses, such as cell proliferation, death, differentiation, and metabolism , 5. Any In principle, it should be possible to identify the important signaling pathways of a cell by using gene expression and protein-protein interaction (PPI) data sets. Extensive, publically-available PPI data provide an opportunity to establish a general signaling pathway blueprint, to which cell type-specific gene expression data can be mapped so as to refine the general signaling pathway blueprint into a cell-type specific blueprint. In this way it should be possible to construct a set of cell-type specific active signaling pathways for any cell that summarizes the information flow from a receptor (R) to kinases (Ks), then to transcription factors (TFs).CASCADE_SCAN generates a specific pathway for a list of protein molecules using a steepest descent method. That is, the method takes the input proteins and then finds their interaction partners iteratively based on some evidences  [Pathlinker reconstructs the known signaling pathways by taking a subnetwork of PPI that consists of the Rs and TFs of interest [PathLinker App is a software tool of the Pathlinker method implemented as a Cytoscape app [PathFinder identifies signaling pathways from a specific R protein to a TF protein in PPI networks by extracting the characteristics of known signal transduction pathways and their functional annotations in the form of association rules [PPI data is a direct source of information about the structure of signaling pathways , 11. A ninterest . The Patcape app . PathFinon rules .A number of methods use PPI data alone to infer signaling pathway structure. Gitter et al. proposed a method to handle the orientation problem in weighted protein interaction graphs as an optimization problem and present three approximation algorithms based on either weighted Boolean satisfiability solvers or probabilistic assignments . Mei et There are also methods that combine PPI and genetic interaction data to identify signaling pathway structure. The activity pathway network (APN) approach utilizes high-throughput genetic interaction data and applies the Bayesian learning method to identify detailed structure of known signaling pathways . AnotherHISP uses the same approach, but in addition applies genetic algorithms with operations including selection, crossover, and mutation to select the candidate topologies of resultant signaling pathways and uses gene knockout data to get directionality of the signaling pathways [Netsearch determines networks by integrating protein-protein interaction data with microarray expression data by extracting subnetworks of the protein interaction dataset whose members have the most correlated expression profiles [A number of computational methods utilize PPI data along with gene expression data to uncover known signaling pathways , 21, 22.and the likely pathway structures for a gene expression profile . Additionally, most of the above methods were evaluated and applied to yeast PPI data, with only a few methods designed specifically to deal with the significantly greater complexity of mammalian data. Here we propose an approach to systematically identify the set of active receptor-mediated signaling pathways within any given cell, by combining PPI and gene expression data. This method is implemented as an open source packaging using the ‘R’ programming language. This open source software is called SPAGI , and is available via https://github.com/VCCRI/SPAGI/.All of the above methods aim to restructure the topologies of known signaling pathways. However, to our knowledge, no open-source methods have been reported that simultaneously and comprehensively identify the set of active signaling pathways STRING database (version 10) [combined_score) values assigned to interactions within STRING range from 0 to 999. We selected PPIs defined by STRING as ‘high confidence’ (i.e. confidence_score > = 700) to further maximise our ability to construct networks representative of true biological pathways. This thresholding yielded 16,550 and 19,502 PPIs for mouse and human respectively. After obtaining these highly scored PPIs both for the human and mouse organisms we have merged all the PPIs by assuming that the molecules have one-to-one homology mapping between the organisms. Note that after filtering and considering the presence of bi-directional interactions within STRING , the set of all known R/K/TF interactions involves 39,004 PPIs in human and 33,100 PPIs in mouse . We then took the union of all PPIs and have assigned the larger score value of a PPI if it is present in both organisms. The merged PPI network has 44,314 edges   to obtaiR (not directly connected with K) to R (directly connected with K)R to KK to K andK to TFFrom the combined high scored PPIs, we collected only the PPIs for the signaling pathways that have interactions able to make full paths from R to K to TF. This process included interactions from:combined_score value for each of the interactions. Note that for clarity, the word ‘path’ is defined as a single R/K/TF prediction, whereas the word ‘pathway’ is defined as the collection of paths that all start from the same R .Finally, we collected all the filtered PPIs from the above step, keeping their associated PPI igraph R package. As igraph considers the weight of the interaction as a cost , we have modified the PPI combined_score value as weight by calculating 1000-combined_score. After assigning the combined_score as weight we collected the reachable shortest paths from each R protein to each TF protein by utilizing the shortest_path function of the package. The shortest_path function uses the Dijkstra’s graph algorithm for the weighted directed graph. We have collected all the complete paths  that have a length from 3 to 7, allowing for at most 2 layers for RP, 5 layers for KN and 1 layer for TF. To identify cell type-specific paths, we then filtered out the complete paths where all factors were designated as housekeeping genes (see the next section for how the list of housekeeping genes was generated). As a result of these steps, the final collection of complete paths consists only of those that are not designated as housekeeping paths. These paths are used as background pathway path data for our method.To make the signaling pathway paths, we first made a directed weighted graph from the PPI data using the log2 (FPKM + 1) = 1.5 value could be used as the expression cut-off for the data sets. Using this cut-off we identified the expressed genes for all the cells and tissues. We then designated a gene as a housekeeping gene if it was found to be expressed in at least 75% of the total number of cells and tissues for that particular organism. This approach was used to identify both the mouse and human housekeeping genes. These 2 lists of housekeeping genes were then combined to generate a unique list of housekeeping genes, assuming one-to-one homology mapping between human and mouse genes. This combined list of unique housekeeping genes was used as background data.We collected the published RNA-seq gene expression data sets for different cells and tissues both for mouse and human from the ENCODE project , 28, andlog2 transformation of RPKM/FPKM/CPM values, an expression cut-off threshold to identify the expressed genes, and a high expression threshold  to calculate the activity score of the pathways.The background signaling pathway path data was used to identify the potential signaling pathways for a particular gene expression data set. As input we took the gene expression data matrix of From the gene expression data set, first we calculated the average expression value of the replicates and then identified the expressed genes by using the cut-off threshold described above.From the background path data we obtained only those paths for which all the protein factors are expressed according to the input gene expression data. This set of paths is treated as potential signaling pathway paths for the gene expression data set.Processing stepsActivity score (As) for a pathway and mathematically can be written as:For each potential signaling pathway, we first calculated the proportion of active molecules (defined as highly expressed genes based on the above high expression threshold) for each path. We then summed all the proportions of all the paths for the pathway and divided the total proportion value by the total number of paths of the pathway. This final value was termed the pi denotes the proportion of active molecules in each path and n denotes the number of downstream TFs for the pathway. Next we plotted the values of n and As to display the results of top ranked active signaling pathways in the upper positions.Where p-value. The false positive rate for the GO analysis was calculated by dividing the number of highly ranked pathways obtained via the randomly assigned data by the number of highly ranked pathways obtained from the original data.The SPAGI false positive rate was obtained by randomly assigning gene expression data and then re-performed the SPAGI analyses. The number of highly ranked active pathways for each sample was then counted. The false positive rate for highly ranked pathways was obtained by dividing the number of highly ranked pathways obtained from the randomly assigned data by the number of highly ranked pathways obtained from the original sample. GO analysis was also performed on the randomly assigned gene expression data used to determine the SPAGI false discovery rate. Each GO analysis was performed separately using the online version of Enrichr for biological processes . Resultsn = 3) [+ LEC–like cells (n = 2) [The ability of SPAGI to identify known, critical, tissue-specific signaling pathways was tested using four cell types obtained from three different gene expression data sets (two are RNA-seq and one is microarray). These four cell types were chosen as there is an extensive body of literature for them that has already identified critical pathways, thus enabling biological validation of the SPAGI output. The first data set used is from mouse dental epithelial cells at the development stage E13.5 (n = 3) . The remn = 3) ; the oth (n = 2) .Published data have shown that BMP and WNT (through FZD receptors) signaling pathways are important for embryonic mouse tooth development . Loss ofA large number of studies over decades have described the requirement for different signaling pathways during lens development. As summarized in a recent review by Cvekl and Zhang , while c+ cells) [+ cell after all filtering.Analysis of published mouse LEC gene expression data  using ou+ cells)  similarlAnalysis of published mouse LF cell data shows that, as expected, LF cell signaling pathways are very similar to LECs Fig. . We have+ cell sample, despite ROR1 being critical for capturing this population of human LECs. Similarly, the EPHA2 pathway was not identified within the mouse LF cell samples, despite this being a key pathway that leads to disease  if disrupted [To determine the breadth of PPI data coverage within the mouse and human STRING datasets, we also performed the SPAGI analysis separately for the human and mouse query data – i.e., mouse samples compared only against the mouse STRING data and the human sample compared only against the human STRING data  from the original mouse dental epithelial cell and mouse LEC data. For comparison, we also performed GO analysis based on the same random assignment used to determine the SPAGI false discovery rate described above. Each GO analysis was performed separately using the online version of Enrichr , and capIn this manuscript we described a new bioinformatics method, SPAGI, that can simultaneously and comprehensively discover the set of active signaling pathways and their putative defined path structures. Our evaluation demonstrates that the SPAGI method can accurately identify known and biologically-relevant signaling pathways from multiple gene expression data sets across different tissue types, while providing specific detail of the molecular cascades involved in these pathways. The SPAGI method therefore provides capabilities not available with other current open-source software. While some pathway analysis software is commercially available , SPAGI provides a free and open-source approach that can routinely provide updated data through updates to the STRING database.In addition to validation of the SPAGI method by comparison against known biology, the SPAGI approach was also validated by assessment of its false positive rate - both on its own and in comparison to the false positive rate obtained via GO analysis. The SPAGI approach identified few pathways when using randomly assigned gene expression data for the mouse dental epithelial cells (0.128) and mouse LECs (0.022). Moreover, the results of the randomly assign gene expression data showed the false positive rate was smaller for the SPAGI method (0–0.2) than the false positive rate obtained via the GO analysis method (0.4–1). These data provide strong support for SPAGI being both more sensitive and more specific than pathway identification via GO analysis alone.To assess whether the SPAGI method is best applied to species-specific PPI data or combined/multi-species PPI data, were performed SPAGI analysis on both single species and combined species PPI data. While large numbers of pathways were identified via the single species analyses, some biologically-relevant pathways were not identified. This included the ROR1 receptor-mediated pathway not being identified via the human PPI data, and the EPHA2 pathway not being identified in the mouse LF cell data. As both these pathways appear to be important in their respective cell types , 38, SPAIt should be noted that as currently applied, the SPAGI method detects receptor-mediated signaling pathways. Modification of the SPAGI approach could be used to identify other cellular control mechanisms involving PPIs independent of TFs. Also, at this stage it is not clear whether the other pathways highly ranked by the activity score are truly active, as protein expression and protein activation state  within a tissue cannot be determined from gene expression data. Nonetheless, the breadth of data provided by SPAGI can provide specific testable hypotheses for cell biologists to guide functional genomic studies to identify critical regulators involved in health and disease. As such, more studies are required to investigate these pathways.The SPAGI method represents a new, interesting and open-source method to comprehensively identify important receptor-mediated signaling pathways from a gene expression data set. We have applied our method to four different gene expression data sets from three different cell types and shown that the SPAGI method correctly identified all the known signaling pathways for the cells, with low false discover rate and lower false discovery than using GO analysis alone. Our results suggest that SPAGI can be a useful approach to identify the potential active signaling pathways given a gene expression profile.Additional File 1:Figure S1. The result of mouse embryonic dental epithelium cell at E13.5 with only the mouse PPI background pathway data. Figure S2. The result of newborn mouse lens epithelium cell with only the mouse PPI background pathway data. Figure S3. The result of human PSC-derived ROR1+ cell (lens epithelium cell-like) with only the human PPI background pathway data. Figure S4. The result of newborn mouse lens fiber cell with only the mouse PPI background pathway data. Table S1. False positive rate calculation for SPAGI method of randomly assigns gene expression values of new born mouse lens epithelial cell and mouse tooth epithelial cell at embryonic day E13.5. Table S2. SPAGI test result for randomly assign gene expression values of new born mouse lens epithelial cell and mouse tooth epithelial cell at embryonic day E13.5. Table S3. Identification of known pathways by SPAGI and GO analysis methods. Table S4. Summary of known pathways identification by SPAGI and GO methods for randomly assigns genes of mouse lens epithelial cell. Table S5. Summary of known pathways identification by SPAGI and GO methods of randomly assigns genes for mouse tooth epithelial cell. Table S6. False positive rate of SPAGI and GO analysis method for known pathways. (PDF 658 kb)


Because of the importance of this type of analysis, more than 70 pathway analysis methods have been proposed so far. These can be categorized into two main categories: non-topology-based (non-TB) and topology-based (TB). Although some review papers discuss this topic from different aspects, there is no systematic, large-scale assessment of such methods. Furthermore, the majority of the pathway analysis approaches rely on the assumption of uniformity of This article presents the most comprehensive comparative study on pathway analysis methods available to date. We compare the actual performance of 13 widely used pathway analysis methods in over 1085 analyses. These comparisons were performed using 2601 samples from 75 human disease data sets and 121 samples from 11 knockout mouse data sets. In addition, we investigate the extent to which each method is biased under the null hypothesis. Together, these data and results constitute a reliable benchmark against which future pathway analysis methods could and should be tested.Overall, the result shows that no method is perfect. In general, TB methods appear to perform better than non-TB methods. This is somewhat expected since the TB methods take into consideration the structure of the pathway which is meant to describe the underlying phenomena. We also discover that most, if not all, listed approaches are biased and can produce skewed results under the null.The online version of this article (10.1186/s13059-019-1790-4) contains supplementary material, which is available to authorized users. Many such experiments involve the comparison of two phenotypes, such as disease vs. control, treated vs. not treated, drug A vs. drug B, etc. Various statistical approaches are subsequently used to identify the genes which are differentially expressed (DE) between these phenotypes, such as t test , Z-score[Z-score , and ANO[Z-score . AlthougOne of the most common techniques used to address this problem is to leverage the knowledge contained in various pathway databases such as Kyoto Encyclopedia of Genes and Genomes (KEGG) , ReactomBecause of the importance of this type of analysis, more than 70 pathway analysis methods have been proposed thus far , 13. Theover-representation analysis (ORA). This approach takes a list of DE genes as input and identifies the pathways in which the DE genes are over- or underrepresented. Some widely used classical approaches from this sub-group use Fisher’s exact test [χ2 test [The first generation in the non-TB category is the act test  and χ2 t[χ2 test . Many to[χ2 test , 17, Gen[χ2 test , GeneMer[χ2 test , EASE [2[χ2 test , FuncAss[χ2 test , etc. Mo[χ2 test , GOstats[χ2 test , GOToolB[χ2 test , GoMiner[χ2 test , 26, DAV[χ2 test , WebGest[χ2 test , 29, etcfunctional class scoring methods (FCS). The hypothesis behind this approach is that small but coordinated changes in sets of functionally related genes may also be important. This approach eliminates the dependency on the gene selection criteria by taking all gene expressions into consideration [The second generation of non-TB approaches includes deration . Some ofderation , Catmap deration , GlobalTderation , sigPathderation , SAFE [3deration , GSA [35deration , PADOG [deration , PCOT2 [deration , FunClusderation , SAM-GS deration , etc.Besides ORA and FCS methods, classical statistical tests, such as Kolmogorov-Smirnov test  and WilcTopology-based methods (TB) have been developed in an attempt to include all this additional knowledge in the analysis. The impact analysis was the first such approach [In principle, considering the pathways as simple un-ordered and unstructured collection of genes—as the non-TB methods do—discards a substantial amount of knowledge about the biological processes described by these pathways. In essence, all the dependencies and interactions between genes that are meant to capture and describe the biological phenomenon are completely ignored. approach . This waapproach  includinapproach , 44, SPIapproach , NetGSA approach , TopoGSAapproach , Topologapproach , PWEA [4approach , PathOloapproach , GGEA [5approach , cepaORAapproach , 53, Patapproach , ROntoToapproach , BLMA [5approach , 57, etcEven though there are some review and benchmarking papers which provide some guidance regarding the use of pathway analysis methods, most of these review papers are limited to describing the methods and discussing various characteristics in a theoretical way , 58. Verp values are uniformly distributed under the null hypothesis , Nguyen et al. [Another significant limitation of these review papers attempting to benchmark pathway analysis methods is that they do not take into account the performance of these methods under the null hypothesis, which is the main cause of type I and type II errors in pathway analysis results. Although existing pathway analysis methods work under the assumption that the n et al. , 63 showHere, for the first time, we present a comparison of the performances of 13 representative pathways analysis methods on 86 real data sets from two species: human and mouse. To our knowledge, this is the highest number of real data sets used in a comparative study on pathway analysis methods. The second assessment investigates the potential bias of each method and pathway.X likely to be always reported as significant by method Y)? This article provides some guidance to help researchers select the right method to deploy in analyzing their data based on any kind of scientific criteria. At the same time, this article will be of interest to any computational biologists or bioinformaticians involved in developing new analysis methods. For such researchers, this article is expected to become the benchmark against which any future analysis method will have to be compared. Finally, because of the bias analysis of all known KEGG pathways included here, this article is also expected to be extremely useful to many people involved in the curation and creation of pathway databases.This article provides precise, objective, and reproducible answers to the following important and currently unanswered questions: (i) is there any difference in performance between non-TB and TB methods?, (ii) is there a method that is consistently better than the others in terms of its ability to identify target pathways, accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUC)?, (iii) are there any specific pathways that are biased ?, and (iv) do specific methods have a bias toward specific pathways  test is a statistical test that can be used to determine whether two classes of results have a non-random association [confusion matrix, includes the following four numbers: (i) DE genes belonging to the pathway, (ii) DE genes not belonging to the pathway, (iii) non-DE genes belonging to the pathways, and (iv) non-DE genes not belonging to the pathway. In R, FE test can be performed by using fisher.test function.ociation . In the WebGestalt is composed of four modules that allow users to manage the gene sets, retrieve the information for up to 20 attributes for all genes, visualize/organize gene sets in figures or tables, and identify impacted gene sets using two statistical tests, namely the hypergeometric test and Fisher’s exact test [act test , 29.GOstats uses the hypergeometric probability to assess whether the number of DE genes associated with the term  is significantly larger than expected. Similar to other non-TB methods, this computation ignores the structure of the terms and treats each term as independent from all other terms [er terms .Kolmogorov-Smirnov (KS) test compares two empirical distributions to determine whether they differ significantly [p values of the pathways.ficantly . SimilarWilcoxon rank sum (WRS) is a non-parametric statistical test generally used to determine whether or not there is a significant difference in the medians of two given populations [p values  of the DE genes inside and outside a pathway. WRS is available in R via the function wilcox.test, which takes the list of DE genes, their fold changes, and a list of genes of a given pathway as input. WRS is employed differently by some pathway analysis tools such as SAFE [ulations . In the  as SAFE  and Came as SAFE .GSEA uses a KS-like statistic test and considers the entire list of genes rather than simply relying on the cut-off to select the list of DE genes [DE genes . The GSEGSA was proposed as an improvement of GSEA [restandardized version of the maxmean statistic. of GSEA . First, PADOG hypothesizes that genes which appear in fewer pathways have a more significant effect than those which appear in many pathways [t-scores. Similar to other FCS methods, PADOG’s input is the expressions of all the genes under study. The PADOG R package is available at [pathways . Hence, lable at .impact analysis and often considered to be the state-of-the-art method in TB pathway analysis. Impact analysis methods calculate the impact of a pathway by combining two types of evidence. The first type of evidence captures the over-representation of DE genes in a given pathway. The second type captures several other important biological factors such as the position and magnitude of expression change for all the DE genes, the interactions between genes as described by the pathway, and the type of interactions. In essence, the measured fold changes of all DE genes are propagated as signals following the topology of the pathway in order to calculate a pathway-level perturbation. The first implementation of impact analysis was Pathway-Express (PE) [SPIA [ROntoTools [The first method to be able to incorporate the topological structure of the pathways in the analysis of signaling pathways was proposed in . This isess (PE) . Currentess (PE) , 68 are ess (PE) : SPIA [6E) [SPIA  and ROntntoTools .CePaGSA and CePaORA consider each pathway as a network where each node can contain one or many genes or proteins [p values derived from the six different measurements mentioned above. Since there is no indication from the original authors about which centrality measurement provides the most accurate result, in this manuscript, we choose the lowest p value of a pathway as its final p value.proteins , 53. CePPathNet relies on two types of evidence in the gene level: direct evidence and indirect evidence [p value obtained from a statistical test such as a t test when comparing two given phenotypes. Indirect evidence of a gene is calculated from the direct evidence of its neighbor genes in a so-called pooled pathway. The pooled pathway is constructed by combining all the pathways in a given pathway database. The PathNet version used in this manuscript incorporates 130 KEGG pathways that were embedded in the software. The p values obtained from these two types of evidence are then combined using Fisher’s method [p value is computed using a hypergeometric test.evidence . Direct s method  to derivOne of the main challenges in assessing pathway analysis methods is that it is difficult to assess the correctness of whatever comes out from the pathway analysis. Many times, papers describing new methods validate them on only two to three data sets followed by a human interpretation of the results. However, this approach has several problems. First, it is biased and not objective. Living organisms are very complex systems, and almost any analysis result will be supported by some references. Without a deep knowledge of the phenomena involved in the given phenotype, it is impossible to judge objectively whether such connections are really meaningful or not. Second, it is not scientifically sound. A scientific approach should formulate some hypotheses in advance, i.e., what a successful outcome of the pathway analysis should look like. Interpreting and justifying the results obtained from an experiment by searching the supporting literature as evidence are not scientifically sound.Another approach for benchmarking methods is evaluating them based primarily on their performances on simulated data sets. The problem with this approach is that any simulated data set is constructed based on a set of assumptions, few of which apply to the real data. The resulting comparison not only is difficult to reproduce, but also has some inherent bias.Here, we introduce two completely objective, reproducible, and scientifically sound approaches to benchmark pathway analysis methods. In the first subsection, we evaluate the methods based on their ability to identify the involved phenotypes using human and mouse benchmark data sets. The second subsection assesses their performances under the true null hypothesis, i.e., there is no true phenotype involved.p values of target pathways  are collected. For each experiment, a perfect method would be able to identify the target pathway as significantly impacted and rank it on top. The target pathway is chosen in advance without human interpretation. Hence, this validation is completely objective and scientifically sound. We apply each method on each of those data sets and report the ranks and ays Fig. .Fig. 1Tp values (bottom panel) of the 75 target pathways for each of the 13 competing methods.Here, we use 75 human data sets related to 15 different diseases with each disease being represented by five different data sets to evaluate the ability of methods to identify target pathways. Figure On a general note, the median rank of target pathways is within the top half for all methods studied, except for KS Fig. a. None oRegarding the performance of the individual methods, the best ranks of target pathways were obtained by PADOG (median rank = 29), followed by CePaGSA, ROntoTools, and PathNet which have median rank values of 45, 46, and 46, respectively. This result also confirms the claims in Tarca et al.  that PADp values of target pathways using the 13 methods is plotted in Fig. p values of non-TB methods are comparable to each other while those of TB methods vary considerably. Among all the methods, the median p value obtained by CePaGSA is the lowest , followed by PADOG  and CePaORA .The p values of the target pathways obtained by non-TB and TB methods. As expected, the median rank values of the TB methods are significantly lower  than those of the non-TB methods  than those of non-TB methods. These results suggest that overall, in this assessment, TB methods are superior to the non-TB methods.We also perform a higher level comparison between the ranks and p value threshold of 0.05 is used to determine whether a pathway is significantly impacted. A true positive (TP) is a positive which is correctly identified as significant. Similarly, a true negative (TN) is a negative which is correctly identified as insignificant. A false positive (FP) is a pathway that does not contain the KO gene but is reported as significant. A false negative (FN) is a pathway that contains the KO gene but is not reported as significant.Although the above assessment is better than the human interpretation approach or using simulated data sets, it still has some limitations: it focuses solely on one true positive, the target pathway. We do not know what other pathways are also truly impacted and therefore cannot evaluate other criteria such as the accuracy, specificity, sensitivity, and the AUC of a method. Here, we use knockout data sets that involve using knockout experiments (KO), where the source of the perturbation is known, i.e., the KO gene. We consider pathways containing the KO gene as positives and the others as negatives. After performing the pathway analysis method on this data set, a Subsequently, we calculate the accuracy, sensitivity, specificity, and AUC of methods studied using 11 KO data sets. Since CePaGSA, CePaORA, and PathNet do not support mouse pathways, they are left out from these comparisons. The comparisons of accuracy, sensitivity, and specificity are illustrated in Additional file p value = 0.009).Among those four statistical measures, the AUC is the most comprehensive and important one because it combines both the sensitivity and specificity across all possible thresholds Fig. . Again, p values of target pathways, and the AUC. Moreover, the results suggest that there is still room for improvement since the ranks of target pathways are still far from optimal in both groups.In conclusion, TB methods outperform non-TB methods in all aspects, namely ranks and p values resulting from those 2000 runs  or bimodal (biased toward 0 and 1)  shows that this method has a tendency to yield low p values and therefore report the pathway as significantly impacted even when it is not . By contrast, a null distribution of p values of a pathway skewed to the left (biased toward 1) indicates that the given method tends to produce consistently higher p values thus possibly report this pathway as insignificant when it is indeed impacted . The results of this null-hypothesis analysis may explain why some methods work well for certain diseases while they perform poorly for others. If a method is biased to report more often a given cancer pathway as significant, that method may be perceived to perform better in experiments involving that particular type of cancer.In this benchmark, we conduct a deeper investigation into the behavior of these methods under the null hypothesis. Here, we create a true null hypothesis by using simulated data sets that are constructed by randomly selected healthy samples from the 75 aforementioned data sets. We apply each method more than 2000 times, each time on different simulated data sets. Each pathway then has an empirical null distribution of  0 and 1 , 72. Howp values of all the pathways generated by GSEA is uniformly distributed under the null  produced by these methods are compared in Fig. The number of pathways biased toward 0 produced by 13 methods are shown in Fig. Similarly, the numbers of pathways biased toward 1 produced by different methods are shown in Fig. The goal of pathway analysis is to translate the list of genes that are differentially expressed across the given phenotypes  into meaningful biological phenomena. Over the last few years, more than 70 pathway analysis methods have been proposed. A real problem in the field is the annotation of the pathways. The pathways evolve as more knowledge is gathered. Essentially, at any moment in time, the knowledge captured by the pathways is both incomplete and perhaps partially incorrect. Regardless of the imperfections of today’s pathways, one still needs to identify which of these pathways are significantly impacted in the given phenotype. Hence, extensive benchmarking results will be very useful even though the annotations of the pathway will be imperfect at any one particular time. Although there have been already a few publications guiding the users by comparing these methods, they are collectively limited in the following ways: (i) they only discuss the methodological aspects of the methods, (ii) the assessment of the methods is based on simulation data sets which often fail to capture the complexity of real biological phenomena, (iii) they do not compare the performance of the methods under the null, (iv) they do not take into account the systematic bias of a method introduced by the imbalanced number of data sets for one disease, and (v) they do not take the quality of annotation of the pathways into account, which is one of the real challenge in the field. These limitations may cause significant bias in the conclusions . Here, wIn order to avoid the potential bias in the comparison, we consider several important factors. First, we utilize an equal number of data sets for each disease in our experiment. This is a crucial factor because if a method tends to unsuccessfully identify some pathways associated with some particular diseases as significantly impacted (type II error), then having too many data sets of these diseases will undermine the rank and the performance of this method.p values and absolute log-fold changes. However, due to the heterogeneity present in the individual experiments, the number of DE genes obtained from different studies of the same condition often differ significantly [p values of 5%, 21 out of 75 human gene expression data sets studied do not have any DE genes. At the same time, one of the data sets has more than 1000 DE genes . The classical approach to obtain a list of DE genes from a given gene expression experiment involves applying thresholds based on ficantly –77. For In addition, we apply the use of KO data sets in assessing pathway analysis methods, which has never been used in any comparative study in the field. This approach avoids the shortcoming of the target pathway approach which focuses on the only one true positive, the target pathway. However, a knockout is a severe perturbation of a complex organism, and in some sense, most if not all pathways will be affected to some degree. Given this, the problem becomes philosophical: given that most of all pathways will be affected to some degree, which pathways we want the analysis to identify? Our proposed answer to this is that we want the analysis to identify the pathways that contain the cause of the phenotype, i.e., the KO gene. We feel that this definition is reasonable because it satisfies two conditions: (i) all “interesting” pathways according to the definition above are truly interesting and (ii) there is no other way to define “interesting” pathways without including all other pathways or without using a completely arbitrary decision threshold.p values of target pathways, as well as the AUC.Our assessment using both human and mouse KO data sets shows that the TB methods consistently provide better results than the non-TB methods in terms of ranks and We also evaluate the performances of pathway analysis methods under the null hypothesis. It is interesting to see that the total number of pathways biased toward 0 is almost double the number of pathways biased toward 1 (696 pathways biased toward 0 versus 356 pathways biased toward 1). In other words, majority of the pathway analysis methods (except GSEA) tend to consider a given pathway as significantly impacted when it is not truly impacted .More importantly, benchmarking methods based on their performances under the null overcome the problem of currently poor annotation of the pathways. In other words, when analyzing two groups of healthy samples (the true null hypothesis), a sound method  should not identify any pathway as significantly impacted, regardless of its quality of annotation.p values of the target pathways, but that might be due to its intrinsic bias toward 0. For example, PADOG achieves the lowest median rank of the target pathways  performed in this manuscript should be considered. A method might perform better than other comparative methods in terms of ranks and ays Fig. a whereasues Fig. b. Howeveues Fig. . In othenumber of biased pathways; (ii) ranking of the target pathways; (iii) AUC, accuracy, sensitivity, and specificity; and finally (iv) p values of the target pathways. The number of biased pathways is the most important factor since a less biased method would yield fewer false negatives and fewer false positives in the result. The second important factor is the ranking of the target pathways. In contrast to the ranking, an assessment of a method based on the derived p values of the target pathways is not as trustworthy because the p values are extremely sensitives to these factors. For example, the low median p value achieved by CePaGSA is due to the fact that this method reports the majority of the pathways (61.82% in average) as false positives in any given condition.To choose the best pathway analysis method, one should consider the following four crucial factors in order of importance: (i) p value produced by all methods studied. If the null distribution of p values of a target pathway is not available, knowing the probability of that pathway being biased toward 0 or 1 is also helpful. In an attempt to provide this information, for each pathway, we calculate the number of methods (out of the 13 methods investigated) biased toward 0 or 1  methods provide a better ability to identify pathways that contain genes that caused the phenotype or are closely related to it . A topology-based analysis is also recommended when (i) it is important to consider how various genes interact, (ii) one wishes to take advantage of the sizes and directions of measured expression changes, (iii) one wishes to account for the type and direction of interactions on a pathway, (iv) one intends to predict or explain downstream or pathway-level effects, and (v) one is interested in understanding the underlying mechanisms. The topology-based approach that provided the best AUC across our 11 KO data set was the impact analysis, as implemented in ROntoTools . The samA non-TB method may be more useful when one needs to analyze arbitrarily defined sets of genes, rather than pathways. In this category, GSEA provided the highest AUC in our extensive testing. GSEA was also the most unbiased method out of the 13 approaches benchmarked in our studies.The Fisher’s exact (FE) test or hypergeometric test is arguably the most widely used method for enrichment analysis. However, our results show that FE is not very suitable in the context of pathway analysis. Figure p values using the two sample t test. Subsequently, we select the genes that have p values less than 5%. Finally, the top 400 genes (around 10% number of genes present in KEGG) with the highest unsigned log-fold changes are considered as DE genes.In order to select the DE genes, we first calculate the gene-level target pathway. Ideally, a good pathway analysis method would rank the target pathway on top with a small p value. We perform each method on the 75 data sets and put the resulting ranks and p values in the violin plots for the comparison , true negatives (TN), false positives (FP), and false negatives (FN) described in the “p value cutoff, where sensitivity is on the y-axis and FPR is on the x-axis. The AUC, the area under the ROC curve, is one of the most important evaluation metrics since it measures a test’s discriminative ability.The receiver operating characteristic curve (ROC curve) is a graphical representation of the relationship between the sensitivity and the false positive rate (FPR=1−specificity) for every possible control sample pool. It is important to stress that this set only contains samples from healthy individuals. For each analysis, we create a simulated data set by randomly choosing 15 samples as “disease” and 15 samples as “control” from the pool. For each of the 13 pathway analysis methods, we create 2000 such simulated data sets and perform pathway analysis separately on each of them, resulting in a list of 2000 p values for each pathway under the null  or 1 (left-skewed or negatively skewed), since they are responsible for type I and type II errors. We use Pearson’s moment coefficient to determine the skewness of a distribution . It is tμ is the mean, σ is the standard deviation, E is the expectation operator, and μ3 is the third central moment.where γ1≃0, then the distribution of p values is symmetric, i.e., it is unbiased. To decide whether a distribution is biased toward 0 or 1, we set a threshold of ±0.1. To be more specific, γ1>0.1 indicates the distribution is right-skewed (biased toward 0) and γ1<−0.1 means it is left-skewed (biased toward 1).If Additional file 1Figure S1. Distribution of numbers of DE genes of 75 human gene expression data sets in the first experiment using the thresholds of corrected p-values <0.05 and log|FC|>1.5. Figure S2. Distribution of numbers of DE genes of 11 mouse gene expression data sets using different thresholds of 1.5 and 5% for log|FC| and corrected p-values, respectively. Figure S3. Comparison of 8 methods using 11 KO data sets in term of accuracy, sensitivity, and specificity. Figure S4. Examples of pathways that have empirical null distributions of p-values biased toward 0. Figure S5. Examples of pathways that have empirical null distributions of p-values biased toward 1. Figure S6. Aggregate p-values of all the pathways generated by GSEA are uniformly distributed under the null. Table S1. 75 benchmark data sets of 15 diseases used to compare 13 methods in this paper. Table S2. Eleven knockout benchmark data sets used to compare 8 methods in this paper. Table S3. Number of methods biased for each pathway. (PDF 6475 kb)Supplementary figures and tables. Additional file 2Review history. (PDF 116 kb)