Antibiotics are frequently prescribed in nursing homes; national data describing facility-level antibiotic use are lacking. The objective of this analysis was to describe variability in antibiotic use in nursing homes across the United States using electronic health record orders.A retrospective cohort study of antibiotic orders for 309,884 residents in 1,664 US nursing homes in 2016 were included in the analysis. Antibiotic use rates were calculated as antibiotic days of therapy (DOT) per 1,000 resident days and were compared by type of stay (short stay ≤100 days vs long stay >100 days). Prescribing indications and the duration of nursing home-initiated antibiotic orders were described. Facility-level correlations of antibiotic use, adjusting for resident health and facility characteristics, were assessed using multivariate linear regression models.2 version 0.24 software).In 2016, 54% of residents received at least 1 systemic antibiotic. The overall rate of antibiotic use was 88 DOT per 1,000 resident days. The 3 most common antibiotic classes prescribed were fluoroquinolones (18%), cephalosporins (18%), and urinary anti-infectives (9%). Antibiotics were most frequently prescribed for urinary tract infections, and the median duration of an antibiotic course was 7 days . Higher facility antibiotic use rates correlated positively with higher proportions of short-stay residents, for-profit ownership, residents with low cognitive performance, and having at least 1 resident on a ventilator. Available facility-level characteristics only predicted a small proportion of variability observed (Model RUsing electronic health record orders, variability was found among US nursing-home antibiotic prescribing practices, highlighting potential opportunities for targeted improvement of prescribing practices. The Centers for Disease Control and Prevention’s (CDC) Core Elements of Antibiotic Stewardship highlight the importance of tracking antibiotic use to optimize prescribing practices. The Centers for Medicare & Medicaid Services (CMS) revised the Requirements for Participation in Medicare for Long-Term Care in 2016, requiring antibiotic stewardship programs that include a system to monitor antibiotic use. However most nursing homes lack the infrastructure and expertise to effectively analyze antibiotic use data. Unlike acute-care hospitals, few studies describe antibiotic use in this setting, and benchmarks to evaluate antibiotic use across nursing homes are not available. Electronic health records (EHR) are increasingly available in US nursing homes and are used for medication orders, and they may be a useful data source to track and report antibiotic use in nursing homes. The objectives of this analysis were to use EHR antibiotic orders to describe national and facility-level antibiotic use rates and assess facility-level variability in a large cohort of US nursing homes.Antibiotics are frequently prescribed inappropriately in nursing homes,PointClickCare is a health technology company that provides cloud-based software to >21,000 long-term and acute/post-acute care providers and covered 8,200 providers in 2016. A retrospective observational cohort study design was used to describe antibiotic use using EHR orders in 1,664 nursing homes that previously consented to de-identification and aggregation of their data from January 1 to December 31, 2016. Resident demographics were reported.S1). If the end date for the antibiotic order was missing, the discontinuation date was used, if both were missing, the order was excluded (0.8% of orders). The number of resident days was determined by census data, and antibiotic rates were calculated as DOT per 1,000 resident days. Nursing home stays were defined as continuous if the gap between discharge and readmission was <3 days. Nursing home stays were classified into short stays (≤100 days duration) and long stays (>100 days duration). Antibiotic use rates were reported by agent, class, route, type of nursing home stay, and time from nursing home admission to the first antibiotic order. An antibiotic course combined orders for the same drug with a prescription gap of ≤1 day. Duration of antibiotic courses were reported. Nursing home–initiated courses were defined as courses ordered on or after day 3 of admission.The proportion of unique residents receiving antibiotics was reported. One antibiotic day of therapy (DOT) represents the order of a single antibiotic on a given day. Systemic routes of administration were included . Antibiotic DOT were classified by class  diagnoses. We were unable to directly determine which indications were prescribed for prophylaxis or treatment; antibiotic courses ≤1 day and >42 days in duration were considered prophylaxis and were excluded from classification under specific infection type. Data mining of all remaining indications was conducted to classify 96.7% of antibiotic courses into 5 infection categories: genitourinary; respiratory; skin, soft-tissue, and musculoskeletal; gastrointestinal or intraabdominal; and other. For example, terms such as “UTI,” “URIN,” “bladder,” “kidney,” “renal,” “prostate,” and “pyelo” in the indication field were grouped under genitourinary infections. Dental prescribing was classified under skin, soft-tissue, and musculoskeletal infections. If >1 indication was identified, the antibiotic course was classified into the infection category other than genitourinary infection. Prescribing practices by infection type were described by drug, number of courses, median, and interquartile range (IQR) of course durations initiated in nursing homes, excluding prophylaxis.Indications were transcribed into the EHR by the prescriber when an antibiotic was ordered; 80% of the indications were entered as free text and 20% were entered using  and CMS Nursing Home Compare were identified and described for all nursing homes with available data (Supplementary Table S2). Facility-level characteristics of nursing homes included in this analysis were compared to all national nursing homes available in the data sets. Continuous covariates were categorized at the median. Comparison between individual covariates and facility-level antibiotic use was performed using the χ2 test or the Fisher exact test where appropriate for categorical variables and the Wilcoxon-Mann-Whitney test for continuous variables. Reported P values with α (significance level) <.05 were included in the multivariate linear regression model for facility-level antibiotic use rates as the outcome and the proportion of short-stay residents as the main exposure. Interaction was assessed using the Breslow-Day method. Because “facility direct care hours per resident day” was found to have an association with type of nursing home stay, it was included as an interaction term in the model for antibiotic use rates. Collinearity of covariates was also assessed. A final model was built using stepwise selection methods. Model R2 was reported to assess the proportion of variance of facility-level antibiotic use explained by the variables included in the model. Statistical analyses were performed using SAS version 9.4 software .A univariate and multivariate linear regression analysis was performed to evaluate the variation in facility-level antibiotic use rates. Medians and IQRs of facility-level antibiotic use rates and proportion of short-stay residents are reported. The main exposure variable was proportion of short-stay residents and was categorized at the median (75% or higher). Facility-level correlates of resident health and facility characteristics publicly available through LTCfocusPointClickCare utilized data from clients that had previously consented  to both deidentification and aggregation of their data, without any restrictions. Under the terms of the BAAs, PointClickCare complied with obligations under Health Insurance Portability and Accountability (HIPPA) and the Health Information Technology for Economic and Clinical Health (HITECH) Act regarding the use and disclosure of protected health information. No protected health information was shared with the CDC, and the CDC did not have access to resident-level data. Summarized aggregated antibiotic use data were shared with CDC collaborators. The National Center for Emerging and Zoonotic Infectious Diseases reviewed the protocol and determined that the project did not meet the definition of research and was exempt from human subject review.Resident demographics and antibiotic use data for the 1,664 study nursing homes are listed in Table S2). However, 14.4% of all nursing home–initiated courses were classified as prophylaxis (duration ≤1 day and >42 days) and contributed to 23.3% of total DOTs (Supplementary Table S2).During 2016, 49% of short- and long-stay nursing home residents receiving an antibiotic were prescribed 1 antibiotic course, 35% received 2–3 courses and 16% received ≥4 courses. Most antibiotics were administered via oral route (81%), 15% were intravenous, and 4% were intramuscular. Fluoroquinolones, cephalosporins, and urinary anti-infectives were the most commonly prescribed antibiotic classes; levofloxacin, trimethoprim-sulfamethoxazole, and ciprofloxacin were the most commonly prescribed agents . The median proportion of short-stay residents at a facility was 75% . The median antibiotic prescribing rate in facilities with a proportion of short-stay residents ≥75% was 90 DOT per 1,000 resident days compared to 38 DOT per 1,000 resident days in facilities where the proportion of short-stay residents was <75%. Aggregated resident- and facility-level covariates significantly associated with facility-level antibiotic use are shown in Table –140. TheAntibiotic orders captured in EHR were used to describe antibiotic use in nursing homes at the facility and national levels. Median facility-level antibiotic use rates varied considerably and were only partially explained by facility and facility-level resident characteristics, highlighting potential opportunities for targeted improvement of prescribing practices. Although antibiotic use rates were higher in short-stay residents, a larger cumulative proportion of long-stay residents receive antibiotics over time. Almost one-third of antibiotic prescriptions were prescribed on admission, highlighting the opportunities for discharge stewardship. A recent analysis reported that 23% of patients discharged from hospital to long-term care facilities were discharged on antibiotics, and antibiotic use was significantly associated with a 30-day emergency department visit and C. difficile infection within 60 days.More than half of all nursing-home residents were prescribed an antibiotic, as reported previously. and fluoroquinolones are commonly prescribed. A high frequency of asymptomatic bacteriuria in combination with inappropriate culturing practices in nursing homes has been associated with antibiotic overprescribing. Overuse of antibiotics has been reported for nursing home residents with respiratory symptoms. Increasing awareness of adverse events associated with antibiotic use, review of medications in nursing home emergency kits, and implementation of antibiotic prescribing protocols and decision support tools can help optimize the treatment of infections in nursing home settings. After excluding courses classified as prophylaxis, the duration of almost half (48%) of all nursing home-initiated courses exceeded 7 days, similar to prescribing in nursing homes in Ontario, Canada, where 45% of prescriptions exceeded 7 days. With a reported 0.4% increase in the risk of antibiotic-related harm with every additional day increase of antibiotic use, and increasing evidence for the effectiveness of shorter antibiotic courses, optimizing treatment duration is critical for resident safety. Implementing standardizing antibiotic reviews when the clinical picture is clearer and more diagnostic information is available can optimize the selection and duration of antibiotic treatment and should be further adapted to nursing home workflows. Further evaluation of the effect of individual prescribing practices and the number of unique prescribers in a facility on the frequency of antibiotic prescribing is needed. Behaviorally targeted assessment and feedback interventions should be evaluated in nursing home settings.Urinary and respiratory tract infections are the most common indications for antibiotic use in nursing homes, as shown previously, Median antibiotic use ate in facilities where cumulative proportion of short-stay residents ≥75% was almost 2.5 times that of facilities with a lower proportion of short-stay residents. These residents are at increased risk for infections following hospital discharge and should be adjusted for when developing facility-level benchmarks. Another notable finding is that having at least 1 ventilator bed in the facility was a strong predictor of antibiotic use. Ventilator-capable skilled-nursing facilities have been associated with increased risk of resident colonization of multidrug-resistant organisms underscoring the importance of infection control and antibiotic stewardship. Different measures of functional status have been evaluated in relation to development of infection and possible antibiotic use. In this analysis, higher measures of nursing care requirements were associated with higher rates of antibiotic use on univariate but not multivariate analysis. The case-mix index, which is used to determine reimbursement for nursing home care, has been associated with antibiotic use, and its role may have been minimized by the high proportion of short-stay residents in this analysis. Overall, facility-level characteristics only explained 24% of antibiotic use variability highlighting the need for additional analyses to further explore different variables and opportunities for improving antibiotic use.Even after adjusting for facility-level characteristics, the proportion of short-stay residents was found to be associated with higher facility-level antibiotic use rates, as shown previously.This study had several limitations. Although some resident characteristics were captured at the facility level, these variables were not available at the resident-level and further evaluation of the influence of specific resident characteristics is needed. Also, prescriber-level data were not available and could not be assessed. A general description of indications was based on provider documentation and no validation was done. Misclassification of infections may limit the use of these data at a facility level without additional provider training and validation. Including general infection categories as part of EHR orders rather than free text may help identify opportunities for improvement and provide feedback for providers on their prescribing practices. Emergency room transfers or hospital admissions that were <3 days were not captured in this analysis; thus, the proportion of antibiotic courses that were initiated outside the nursing home may have been underestimated. Antibiotic orders, which may not always reflect actual administration, were described in this analysis. However, antibiotic orders may be a good proxy for prescribing behavior, especially for audit-and-feedback interventions. Although this analysis reflects data from 2016 and some prescribing practices may have changed over time, the analytic methods used in the analysis of EHR orders and identifying facility-level characteristics remain valuable and can serve as a baseline for assessing the effect of stewardship regulatory requirements. Although this analysis provides a description of a large subset of nursing homes, it may not be representative of national practices. Further evaluation and validation of EHR antibiotic use data and measures can support nursing homes in using these data in tracking and reporting antibiotic use. Other important potential next steps include evaluating data elements for risk adjustment of facility-level benchmarks, streamlining the generation of antibiotic use reports for individual facilities and assessing the impact of prescriber audit and feedback in this setting.Tracking antibiotic prescribing in nursing homes is critical to assess opportunities for improvement, guide practice change, evaluate the impact of stewardship interventions and improve resident outcomes.


Autoimmune diseases (AIDs) are a class of chronic disabling diseases characterized by inflammation and damage to muscles, joints, bones, and internal organs. Recent studies have shown that much progress has been made in the research of exosomes in AIDs. However, there is no bibliometric analysis in this research field. This study aims to provide a comprehensive overview of the knowledge structure and research hotspots of exosomes in AIDs through bibliometrics.Publications related to exosomes in AIDs from 2002 to 2021 were searched on the web of science core collection (WoSCC) database. VOSviewers, CiteSpace and R package “bibliometrix” were used to conduct this bibliometric analysis.Frontiers in immunology is the most popular journal in this field, and Journal of Immunology is the most co-cited journal. These publications come from 473 authors among which Ilias Alevizos, Qianjin Lu, Wei Wei, Jim Xiang and Ming Zhao had published the most papers and Clotilde Théry was co-cited most often. Studying the mechanism of endogenous exosomes in the occurrence and development of AIDs and the therapeutic strategy of exogenous exosomes in AIDs are the main topics in this research field. “Mesenchymal stem cells”, “microRNA”, “biomarkers”, “immunomodulation”, and “therapy” are the primary keywords of emerging research hotspots.312 articles from 48 countries led by China and the United States were included. The number of publications related to exosomes in AIDs is increasing year by year. Central South University, Sun Yat Sen University, Tianjin Medical University and University of Pennsylvania are the main research institutions. This is the first bibliometric study that comprehensively summarizes the research trends and developments of exosomes in AIDs. This information identifies recent research frontiers and hot directions, which will provide a reference for scholars studying exosomes. AIDs are a class of diseases caused by the loss of the body’s autoimmune tolerance leading to damage to its own tissues, affecting about 10% of the world’s population .Exosomes are extracellular vesicles (EVs) with a diameter of about 30~150 nm, and their contents are rich, including: DNA, RNA, lipids, proteins, cytokines and other substances. The other two extracellular vesicles with larger diameters are microvesicles and apoptotic bodies, but the immunomodulatory effect of exosomes is the strongest . In addiBibliometrics is a literature analysis method that analyzes the output and status of publications in a particular research field from a quantitative and qualitative perspective , 16. Durhttps://www.webofscience.com/wos/woscc/basic-search) on 15 April 2022. The search formula is ((TS = (Exosomes)) AND TS = (Autoimmune Diseases)) AND LA = (English), and the type of documents is set to “articles “ and “review” (We conducted a literature search on the Web of Science Core Collection (WoSCC) database  is a bibliometric analysis software that can extract the key information from numerous publications , which iCiteSpace (version 6.1.R1) is another software developed by Professors Chen C for bibliometric analysis and visualization , 30. In https://www.bibliometrix.org) was applied for a thematic evolution analysis and to construct a global distribution network of publications of exosomes in AIDs (The R package “bibliometrix” (version 3.2.1) , Period II (2009-2016) and Phase III (2017-2021). As shown in These publications came from 48 countries and 122 institutions. The top ten countries are distributed in Asia, North America, and Europe, mainly in Asia (n=4) and Europe (n=4) Table 1.The top 10 institutions are located in 3 countries, with three-fifths of them located in China. The four institutions that published the most relevant papers are: Central South University , Sun Yat Sen University , Tianjin Medical University  and University of Pennsylvania . Subsequently, we selected 44 institutions based on the minimum number of publications equal to 3 for visualization, and constructed a collaborative network based on the number and relationship of publications of each institution Figure 4Frontiers in Immunology published most papers , followed by International Journal of Molecular Sciences , Stem Cell Research & Therapy  and Scientific Reports . Among the top 15 journals, the journal with the highest impact factor is Cellular & Molecular Immunology (IF=11.53), followed by Molecular Therapy (IF=11.46). Subsequently, we screened 48 journals based on the minimum number of relevant publications equal to 2 and mapped the journal network  was the most cited journal, followed by International Journal of Molecular Sciences (Co-citation=651), Blood (Co-citation=568) and Frontiers in Immunology (Co-citation=514). In addition, the impact factor of Nature reviews Immunology is the highest (IF=53.11), followed by Nature (IF=49.93). Journals with the minimum co-citation equal to 150 were filtered to map the co-citation network  and Iran . We noticed the close cooperation among four countries: the United States, China, Italy, and Iran. In addition, Iran has active collaborations with the United States, Spain and Germany. When it comes to research institutions, there is a good cooperative relationship between some of them, such as Sun Yat Sen University, Tianjin Medical University and University of Pennsylvania. However, we also found Central South University, while publishing the most papers, had little collaboration with other institutions, which will be detrimental to the long-term development of academic research. Although there are cooperative relations between some countries, the breadth and intensity of cooperation between institutions are not ideal. For example, there is only a small amount of cooperation between institutions in the United States and China. Clearly, this situation will hinder the development of the research field in the long run. Therefore, we strongly recommend research institutions in various countries carry out extensive cooperation and communication to jointly promote the development of exosomes in AIDs.Frontier in immunology , indicating it is currently the most popular journal in this research field. Among the journals, the journal with the highest impact factor is Cellular & Molecular Immunology , followed by Molecular Therapy . Regarding the co-cited journals, we could find most of them are high-impact Q1 journals. Obviously, these journals are high-quality international journals, providing support for the study of exosomes in AIDs. What’s more, the current research of exosomes in AIDs is mainly published in Molecular, Biology, and Immunology related journals, and very few studies are published in clinically related journals, indicating that most of the current research is still in the stage of basic research.Most of the research of exosomes in AIDs was published in + T cells, CD8+25+ regulatory T cells  is the most frequently cited author, followed by Raposo G (citation=87) and Kim SH (citation=85). In 2001, Théry C conducted a detailed identification of the protein components in dendritic cell-derived exosomes, and found that the components in exosomes were significantly different from those of apoptotic vesicles . The nexesponses , and thiesponses , 68. In ssengers . Subsequssengers . In recessengers . ObviousThe Journal of experimental medicine in 1996, which first proposed that EB virus-transfected B cells can perform antigen presentation by secreting exosomes, which stimulated people’s interest in the role of exosomes in antigen presentation , MSC-Exos are complex mixtures of factors targeting different therapeutic pathways, so their immunomodulatory and tissue repair effects are much stronger . BesidesAlthough scholars have achieved good results in the use of MSC-Exos in the treatment of AIDs animal models, there is still a long way to go before its clinical application in the treatment of AIDs. Firstly, the manufacturing and preparation processes of MSC-Exos are tedious and complicated, which are still to be optimized. We should seriously consider standardized methods for MSC-Exos isolation, purification, quantification, and quality control, which can reduce batch effects and establish quality control for its clinical applications. Secondly, we can try to explore ways to increase the half-life of exosomes [such as MSC-Exos with an injectable hydrogel ] and detmiRNAs are currently the most studied substances in the contents of exosomes Table 7,Many miRNAs can be biomarkers for disease diagnosis and prognosis. The results of miRNA microarray showed that the level of miR-548a-3p in serum exosomes of RA patients was significantly reduced, and was negatively correlated with inflammatory indicators such as serum CRP and rheumatoid factor (RF), so miR-548a-3p may serve as a biomarker for predicting disease activity in RA . In addiOn the other hand, due to the unique lipid bimolecular structure, exogenous exosomes can deliver miRNAs from the outside into the organism, and have an intervening effect on the target cells to treat AIDs. In addition to the aforementioned MSC-Exos, other cells-derived exosomes can also deliver miRNAs to treat AIDs. For example, dendritic cell-derived exosomes can deliver miR-146a to promote the formation of Treg cells, ultimately alleviating clinical symptoms in Myasthenia gravis .in vitro.More importantly, exogenous exosomes can also exert immunomodulatory effects on other cells , 116. Rein vivo and high delivery efficiency) in the treatment of AIDs compared to traditional drugs and cell therapy  have been widely used in the field of bibliometrics , so our Of course, this study also has some shortcomings. Firstly, the data of this study are only from the WoSCC database, and other databases are ignored, which may miss some relevant studies. Secondly, we filtered studies published in English, which may mean non-English writing papers were underestimated. What’s more, publications in 2022 were not included due to insufficient data.Exosomes have important research value and application prospects in AIDs. The rapidly increasing number of publications shows that the research of exosomes in AIDs is increasingly valued by scholars from the whole world. The leading countries are China and the United States; however, cooperation and communication among various countries and institutions remain to be strengthened. On the one hand, studying the mechanism of endogenous exosomes in the occurrence and development of AIDs will help us to analyze the reasons for the immune imbalance of AIDs and be beneficial to the diagnosis of the disease process of AIDs. On the other hand, compared with traditional drugs and cell therapy, exogenous exosomes have great advantages in the treatment of AIDs, so studying the therapeutic strategy of exogenous exosomes will be of great application value for the precision treatment of AIDs in the future. Notably, in addition to basic research, we should also pay attention to the transformation of research results, that is, the clinical application of exosomes to AIDs patients.The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.FW and JG wrote the manuscript. FW and JG have contributed equally to this work and share first authorship. JK, XW, QN and JL analyzed the data and drew pictures. LZ reviewed and revised the manuscript. All authors read and approved the final manuscript. All authors contributed to the article and approved the submitted version.This work was supported by the National Natural Science Foundation of China [grant number 81771768] and by the applied basic research project of Shanxi Science and Technology Department [grant number 201901D111416].The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Noteworthy, the Saturated fatty acids (SFA)content of MT was reduced from 61.46% to 25.49% (p < 0.05), although SL was found to be more prone to oxidation during storage. The clearest discrepancy in volatile compounds was the increase of heterocyclic compounds in SL. In summary, improving the edible properties of animal fats by adding vegetable oils is an effective solution, and SL may have a great potential to be developed into a high-quality product with improved nutritional composition of animal fat.This study evaluated the physio-chemical properties of a structural lipid (SL) obtained by the enzymatical incorporation of flaxseed oil into mutton tallow (MT). By measuring the melting point, colour, safety, fatty acids, apparent viscosity, shear stress and volatile compounds of the samples, the results showed that compared to MT, SL exhibited lower L∗(lightness) value, melting point, apparent viscosity and shear stress ( Structured lipid; Transesterification; Fatty acids; Rheological properties; Volatile compounds. These specialized lipids can be designed to contain desired fatty acid compositions with a variety of applications, to variety of application in pharmaceutical and food industry . AlthougMutton tallow (MT) is composed of 62% of saturated fatty acids of which more than 57% are MCFA, such as C16:0 and C18:0. Intake of dietary saturated fatty acids has been well-known to be associated with increasing low-density lipoprotein cholesterol, thus, associated with increasing the risk of cardiovascular diseases .This study aimed to develop and characterize MT substitutes by constructing healthier animal fat blends. The results of the study may help determine the feasibility of using enzymatic interesterification to produce structured lipid (SL) with balanced fatty acid composition and modified physical and chemical properties. Our results have shown that SL may have a great potential to be developed into a high-quality product with improved nutritional composition of animal fat.22.1FO and MT were purchased from Ningyang supermarket in Yinchuan, Ningxia, China, and Novozym 435  was from Novozymes Biotechnology Co., Ltd . Hexane was of liquid chromatography grade. Other chemicals and reagents not specifically described in this study are of analysis pure reagent.2.2SL samples were prepared as described by the previous authors with a slight modification . Lipase-2.3The color of the sample was measured with Commission International Eclairage (CIE) for L∗(lightness), a∗ (redness) and b∗ (yellowness) using a CR400 Minolta colorimeter . The light source of C with an aperture of 8 mm and the additional closed cone was calibrated utilizing a white plate . The whiteness was calculated utilizing the following formula:The sample was poured into a glass bottle, and the color of the sample was evaluated at 5 separate positions against a standard whiteboard background , 10.2.4A 2 mm diameter capillary tube was immersed into the melted oil sample to be tested until the oil in the tube reached a height of 10 mm. The tube was removed, placed quickly into liquid nitrogen to cool and solidify, and freezed in a refrigerator (−18 °C to -20 °C) for 12 h. The capillary tubes were then put into a water bath (ramp-up procedure: 0.1 °C/min), and the temperature of the fat column sliding upwards in each capillary tube was recorded.2.5GC-MS was performed as reported by the previous authors with minor modifications , 11. The2 (≥99.99%). MS conditions were as follows: EI ion source, emission current 200 amu, scan range 20–550 amu and electron energy 70 eV. Identification of FAs by comparison of retention times and FAME criteria and the relative contents expressed as the weight percentage  were calculated. Each sample was analyzed three times.One microliter of fatty acids (FA) methyl esters was injected into an ODS-SP chromatographic column   with the separation mode of 1:12. The temperature program of the column was as follows: 2 min at 180 °C, ramp up to 230 °C at a rate of 3 °C/min and finally held at 230 °C for 10 min, and the temperatures of the injector and detector were 250 °C. The detection voltage was 350 V, and the carrier gas was He2.62H5OH boiling method [v/v), and 1 mL of saturated potassium iodide solution was added. After the flask was away from light for 10 min, deionized water (30 mL) was added. The sample solution was titrated against 0.01 M sodium thiosulphate solution, with 1% starch solution (1 mL) as the indicator. The peroxide value was expressed as milliequivalents of active oxygen per kg of samples (meqO2/kg).The acid value, peroxide value and saponification value of the oils and fats were determined by cold solvent indicator titration, KOH–Cg method , 13. Theg method . The sam2.7−1 to 400 s−1. To test the effect of temperature (55–95 °C) on viscosity and shear stress, the shear rate and heating rate were set at 50 s−1 and 5 °C/min respectively.The rheological properties of FO, MT and SL were measured using an AR1500ex rheometer (TA Instruments (Shanghai) Co. New Castle, USA). The procedure was set up with reference to the method of the previous authors [2.8As reported by the previous authors with minor modifications . To extrThe organic phase was analyzed using gas chromatography , with the DB-WAX chromatographic column  . The extraction head was inserted into the sample inlet, and the fiber head was pulled out and analyzed for 5 min. Oven temperature was kept at 60 °C for 2 min initially, then increased to 100 °C at a rate of 3 °C/min and held for 5 min, then from 100 to 180 °C at 5 °C/min and held for 5 min, and reached a final temperature of 230 °C at a rate of 8 °C/min and held for 10 min. Mass specialty conditions were ionization mode EI, ion source temperature 200 °C, electronic energy 70 eV, filtration emission current 200 μA, interface temperature 250 °C and scanning mass range (32–402 amu).2.9P < 0.05 was considered to be statistically significant.The results were expressed as mean ± standard deviations (SD). The figures and principal component analysis (PCA) were plotted with Origin 2018b for Windows . Data were analyzed by one-way ANOVA, followed by Duncan's multiple range test using SAS 8.2 for Windows . 33.1p < 0.0001), indicating that the prepared SL can effectively improve the edible quality of MT. The change in the melting point may be mainly caused by the changes in the contents of triglyceride with a high melting point, medium melting point and low melting point in the mixture [The differences in the melting points reflect the existing state and edible quality of oils and the differences in the fatty acid compositions. FO showed the characteristics of typical vegetable oil, with a lower melting point of -25.3 ± 0.9 °C . MT show mixture .Table 1Tp < 0.05), indicating that SL and MT have great differences in color , 1.130 ± 0.001 g/100g (p < 0.0214) and 184 ± 1 mg/g (p < 0.0001).There were significant differences in oxidative stability values of FO, MT and SL among all samples . The phy3.2p < 0.002). In contrast, PUFA (50.09 ± 1.06%) was predominant in SL, followed by SFA (25.51 ± 0.22%) and MUFA (23.17 ± 0.08%) (p < 0.0004). Stearic acid (C18:0) was the predominant SFA in all the samples, followed by palmitic acid (C16:0). Oleic acid (C18:1) was the predominant MUFA in all the groups. Concerning PUFA, of note was the new production of eicosatrienoic acid  (41.63 ± 0.92%) in SL.As shown in SL was produced by the transesterification of FO and MT using lipase. The enzymatic catalytic site and activity of lipases were found to result in differences in the extent of acyl migration . As expe3.32/kg), followed by SL (406 ± 4 meqO2/kg) and MT (85 ± 2 meqO2/kg). The peroxide values of FO, MT and SL seemed to be too high, which may be because they were stored at a high temperature (65 °C) . Notably, the increased rate of the peroxide value in SL was significantly lower than that in FO, and the increased rate of the peroxide value in MT was significantly lower than those in FO and SL. The growth rate of FO peroxide value was lower than that in SL in 0–8 days but higher than that in SL in 8–14 days. There is evidence that the loss of nutrients, shortened shelf life, off-flavours and discolouration of lipids are closely linked to lipid oxidation. However, as the primary stage of lipid oxidation ends, deeper oxidation of lipids leads to the accumulation of lipid peroxides , which are potentially toxic and carcinogenic [The peroxide values of the three samples were monitored for 14 days at 60 °C to investigate the effect of lipase-catalyzed transesterification on the oxidative stability of the SL. The peroxide values of the three samples tended to increase with an increase in the storage time . After tinogenic . The hydinogenic , 22. Thiinogenic , which minogenic , 25. The3.4As shown in Indeed, the viscosity and shear force generally increased with the triacylglycerol FA chain length and declined with the degree of unsaturation. These results were consistent with previous reports , 27. At Similarly, melting point was found to be highly corrected with viscosity, C18:1 and C18:2 content, and the ratio of SFA to USFA and MUFA to PUFA of three different oils and fats . FAs wit3.5A total of 73 VCs were identified in the three samples . A totalMost of the heterocyclic compounds are produced by the oxidative decomposition of lipids. Heterocyclic compounds as a portion of fat and fresh meat flavor have an important role in the sensory system, and the effect of transesterification on flavor compounds is beneficial , 34. Mor4This study was undertaken to provide the lipid industry with information on the physicochemical properties of SL to improve its application as edible oil and fat and value-added food ingredient. The results showed that SL had the color and apparent viscosity similar to vegetable oils, low melting point and reasonable fatty acid ratio. The physicochemical indexes  of SL were lower than the maximum values provided by Codex Alimentarius (2005), which met the safety standards of edible oils and fats. Future research should further explore the combination of various vegetable oils and animal fats and the selection of materials for other food ingredients to improve the safety of SL in the food industry. As the enzymatic transesterification formulation can be easily translated to large-scale production, this modification and reuse of animal fat may have great application potential in the food industry.Jun Liu: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.Weiyi Zhang: Conceived and designed the experiments; Performed the experiments.Dunhua Liu: Conceived and designed the experiments; Performed the experiments; Contributed reagents, materials, analysis tools or data.Wei Zhang: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.Lu Ma: Contributed reagents, materials, analysis tools or data.Shuzhe Wang: Conceived and designed the experiments.This work was supported by Funds for Independent Innovation in Agricultural Science and Technology in Ningxia Hui Autonomous Region  and The national Spark Program of China (2015GA880005).The data that has been used is confidential.The authors declare no conflict of interest.No additional information is available for this paper.


The Centers for Disease Control and Prevention identify antibiotic prescribing stewardship as the most important action to combat increasing antibiotic resistance. Clinicians balance broad empiric antibiotic coverage vs. precision coverage targeting only the most likely pathogens. We investigate the utility of machine learning-based clinical decision support for antibiotic prescribing stewardship.In this retrospective multi-site study, we developed machine learning models that predict antibiotic susceptibility patterns  using electronic health record data of 8342 infections from Stanford emergency departments and 15,806 uncomplicated urinary tract infections from Massachusetts General Hospital and Brigham & Women’s Hospital in Boston. We assessed the trade-off between broad-spectrum and precise antibiotic prescribing using linear programming.p = 0.11). In the Boston dataset, the personalized antibiograms coverage rate is 90.4%; a significant improvement over clinicians (88.1% p < 0.0001). Personalized antibiograms achieve similar coverage to the clinician benchmark with narrower antibiotics. With Stanford data, personalized antibiograms maintain clinician coverage rates while narrowing 69% of empiric vancomycin+piperacillin/tazobactam prescriptions to piperacillin/tazobactam. In the Boston dataset, personalized antibiograms maintain clinician coverage rates while narrowing 48% of ciprofloxacin to trimethoprim/sulfamethoxazole.We find in Stanford data that personalized antibiograms reallocate clinician antibiotic selections with a coverage rate (fraction of infections covered by treatment) of 85.9%; similar to clinician performance . Antibiotic selection driven by personalized antibiograms achieves similar coverage rates to those seen in actual clinical practice using fewer broad spectrum antibiotics. Increasing antibiotic resistance is a natural and inevitable consequence of regular antibiotic use, raising the looming threat of a post-antibiotic era that could cripple routine medical care with higher infection-related mortality and costs of care5.The World Health Organization (WHO) estimates that 700,000 people already die annually due to antibiotic resistant infections, and expects this number to exceed 10 million per year by 20506. For example, sixty percent of hospitalized patients receive antibiotics despite the fact that half of antibiotic treatments are inappropriate—meaning antibiotic use was unwarranted, the wrong antibiotic was given, or the antibiotic was delivered with wrong dose or duration7. A key challenge is that antibiotics must often be prescribed empirically, before the identity of the infecting organism and antibiotic susceptibilities are known. Microbial cultures are the definitive diagnostic tests for this information, but may take days to confirm final results, far too long to delay initial therapy8.The Centers for Disease Control and Prevention (CDC) identify improving antibiotic prescribing through antibiotic stewardship as the most important action to combat the spread of antibiotic resistant bacteria10. Yet, it is precisely the excessive use of antibiotics that increases drug resistant organisms11. Overuse of broad-spectrum antibiotics can thus have severe immediate and indirect consequences ranging from increasing antibiotic resistance to drug-specific toxicities and secondary infections such as Clostridioides difficile colitis14.Broad-spectrum antibiotics help ensure coverage of a range of organisms that would lead to rapid clinical deterioration if left untreated17. An institution’s antibiogram might report for example that 1000 Escherechia Coli were isolated in the prior year, and that 98% were susceptible to meropenem, while only 89% were susceptible to ceftriaxone. These approaches may not consider many or any patient-specific features. Microbial culture results found within the electronic health record can be used to objectively measure not only whether chosen antibiotics were appropriate, but if alternatives would have sufficed. Here we hypothesize that the standard of care may benefit from machine learning-based clinical decision support for personalized treatment recommendations.Existing standards of care for selecting empiric antibiotics involve referring to clinical practice guidelines combined with knowledge of institution-specific antibiograms—an annual report from an institution’s microbiology lab that tracks the most common organisms isolated by microbial cultures and the percentages that were found susceptible to different antibiotics19. Though promising, neither system was widely adopted by clinicians as they were not easily integrated into their medical workflow or adaptable to constantly evolving local antibiotic resistance patterns20. With modern day hospital IT and electronic medical record software it is now possible to integrate clinical decision support into medical workflows and dynamically train models with real world clinical data streams21.The development of computerized clinical decision support for antibiotic prescribing stems back decades to the likes of MYCIN and Evans et al.—rule-based systems that guide clinicians through empiric antibiotic selection24. Limitations in most of these prior studies is that positive microbial culture results were used as a proxy for the outcome of infection, despite their being both false positive and false negative microbial cultures with respect to an actual clinical infection. Moreover, these studies do not address the question of which antibiotics should have been administered.Literature concerning modern day data-driven approaches to antibiotic decision support fall into two distinct categories. One category of studies predict infection status at the time microbial cultures were ordered, offering promising consideration for when antibiotics are needed at all28. These studies address the challenge of selecting the right antibiotic. Antibiotic prescribing policies that leverage machine learning predictions were simulated and benchmarked against retrospective clinician prescribing and suggested improved performance. Optimizing patient coverage rates, however, is only one important objective that could be naively addressed by prescribing maximally broad antibiotics to all patients without consideration for adverse effects on the individual or population. Further critical research needs to systematically evaluate the trade-off between maximizing antibiotic coverage across a population of patients and minimizing broad-spectrum antibiotic use.The second category of studies predict antibiotic susceptibility results for positive microbial cultures29. We examined precision-recall curves of these models and highlighted thresholds that separated subgroups of patients with probability of coverage with narrower-spectrum antibiotics equal to antibiogram values of broader-spectrum antibiotics. Here we substantially extend our work on personalized antibiograms to generalize beyond species identity, introduce a linear programming optimization framework to simulate optimal antibiotic allocations across a set of patients, conduct a sensitivity analysis to estimate model performance on patients with negative microbial cultures, and assess the generalizability of our findings with data from an external site. Specifically, our objective in this study is the following.In a previous work we demonstrated that machine learning models could predict antibiotic susceptibility results when conditioned on microbial speciesWe (1) train and evaluate personalized antibiograms—machine learning models that use electronic health record data to predict antibiotic susceptibility results; (2) evaluate the performance of antibiotic selections informed by personalized antibiograms relative to selections made by clinicians; and (3) systematically evaluate the trade-off in performance when fewer broad-spectrum antibiotics are selected across a population of patients. We complete this objective using a cohort of patients who presented to Stanford emergency departments between 2009 and 2019 and then replicate our process on an external cohort of patients who presented to the Massachusetts General Hospital and Brigham & Women’s Hospital in Boston between 2007 and 2016.p = 0.11). We find in the Boston data that personalized antibiograms reallocate antibiotic selections with coverage rate of of 90.4%—significantly higher than the coverage rate clinicians achieve (88.1% p < 0.0001). In the Stanford data we find that antibiotic selections guided by personalized antibiograms achieve a coverage rate as good as the real world clinician prescribing rates while narrowing 69% of the vancomycin + piperacillin/tazobactam selections to piperacillin/tazobactam, 40% of piperacillin/tazobactam prescriptions to cefazolin, and 21% of ceftriaxone prescriptions to ampicillin. In the Boston data we find that personalized antibiograms can replace 93% of the total ciprofloxacin prescriptions with nitrofurantoin without falling below the real world coverage rate. Similarly 48% of the total ciprofloxacin and 62% of nitrofurantoin prescriptions can be exchanged with trimethoprim/sulfamethoxazole.In our Stanford cohort we find that personalized antibiograms are able to reallocate antibiotic selections made by clinicians with a coverage rate (defined as the fraction of infections covered by the antibiotic selection) of 85.9%, similar to the clinician coverage rate , ValleyCare hospital  and Stanford University Healthcare Alliance affiliated ambulatory clinics. We included patient encounters from both the Stanford and ValleyCare emergency departments. Structured electronic health record data include patient demographics, comorbidities, procedures, medications, labs, vital signs, and microbiology data.We used the STAnford Research Repository (STARR) clinical data warehouse to extract de-identified patient medical records31. Our study was approved by the institutional review board of the Stanford University School of Medicine. Project-specific informed consent was not required because the study was restricted to secondary analysis of existing clinical data.STARR microbiology data contain information about microbial cultures ordered within the hospital, emergency departments, and outpatient clinics. Our microbiology data included source of culture, order timestamp, and result timestamp. Microbial culture data also included resulting organism name and antibiotic susceptibility pattern, which indicated whether each organism was either susceptible, intermediate, or resistant to a set of tested antibiotics. Microbiology data collected from ValleyCare and Stanford emergency departments were analyzed at separate microbiology labs. Both follow standardized national procedures to measure antibiotic susceptibility as defined by the Clinical & Laboratory Standards Institute (CLSI)32.We replicated the analysis on electronic medical record data of patients from Massachusetts General Hospital and the Brigham and Women’s Hospital in Boston, MA—a dataset made available through Physionet28. Observations between the years 2007 and 2016 were included in the study.The unit of observation in this analysis was a patient-infection. In the Stanford data, analysis was restricted to patients who presented to the emergency department with infection between January 2009 and December 2019. We included patients 18 years or older and patients who required hospital admission. We further restricted the cohort to patients where an order for at least one positive blood, urine, cerebral spinal fluid or fluid microbial culture; and, at least one order for intravenous or intramuscular antibiotics were placed in the first 24 h after presentation to the emergency department. We excluded observations where antibiotics or microbial cultures had been ordered within the 2 weeks prior to the presentation to the emergency department. We incorporated admissions with negative cultures in a sensitivity analysis. Figure Coagulase-Negative Staphylococci sometimes represent true infections warranting antibiotic treatment, we excluded Coagulase-Negative Staphylococci cases as they frequently represent non-infectious contaminants. We trained models for eight commonly administered single antibiotic choices  and four combination therapies . We defined our prediction time to be the time at which the first intravenous or intramuscular antibiotic was ordered for the patient following admission to the emergency department. Using Boston data, we similarly trained personalized antibiogram models that predicted whether four antibiotics commonly administered for urinary tract infection  would provide activity against the target infection.Using Stanford data we trained 12 binary machine learning models to estimate the probability that common antibiotic selections would provide activity against infections at the point in time empiric antibiotics were chosen. An antibiotic selection was said to provide activity against a patient’s infection if all microbial organisms that grew in the patient’s microbial cultures were listed as susceptible to at least one of the antibiotics in the selection. While microbial cultures growing Pseudomonas aeruginosa would be resistant to ceftriaxone and cefazolin; Gram negative rods would be resistant to vancomycin; and Streptococcus agalactiae is susceptible to cephalosporins35. Antibiotic susceptibility was also inferred from observed results of related antibiotics. For example, if an organism was susceptible to a first-generation cephalosporin, it was assumed that it would also be susceptible to a second, third or fourth-generation cephalosporin36.Not all antibiotics were tested against all organisms in the microbiology lab which resulted in missing labels for some of our observations. Antibiotics are only tested if they could plausibly be active against a specific organism. We received consultation from the Stanford microbiology lab to generate a set of rules to impute missing labels. For example, our imputation rule assumed that 37. Categorical features included diagnosis codes (ICD9 and ICD10 codes), procedure orders, lab orders (including microbiology lab orders), medication orders, imaging orders and orders for respiratory care. For prior microbial culture results, categorical features were constructed based on the antibiotic susceptibility pattern of extracted isolates. Bag of words feature representations for each admission were generated such that the value in each column was the number of times that feature was present in the patient’s record during a pre-set look back window. For diagnosis codes, the look back window was defined as the entire medical history. For other categorical features, the look back window was defined as the year leading up until prediction time. If a feature was not present for a patient (for example a diagnosis code was never assigned or a lab test was never ordered), the value in the corresponding column was zero. This allowed us to implicitly encode missing values into our representation without having to impute our data.Using Stanford data, a feature matrix was constructed with data in the EHR with timestamps up until the prediction time. Though observations were restricted to infections that required hospital admission, features were constructed based on data from all forms of available medical encounters (including ex: prior primary care visits). We used a bag of words featurization technique similar to Rajkomar et al. to construct our feature matrixNumeric features included lab results and vital signs from flowsheet data. We binned the values of each unique numerical feature into categorical buckets based on the decile cutoffs of their distributions in the training set. We used the training set to identify thresholds for each decile in a feature’s distribution and then applied these thresholds to patients in our validation and test set to prevent information leakage. To create the bag of words representation, we created columns in our feature matrix where the corresponding value represented the number of times the feature with a value in a particular decile was observed within a look back window. For lab results and vital signs, the look back window was 14 days prior to prediction time. Features were not standardized, and rather left as counts.In addition to these categorical and numeric features, we included patient demographics , insurance information, and institution . Sex, race, ethnicity, insurance payer, and institution were one-hot encoded. In total, the sparse feature matrix contained 43,220 columns.28 Features included prior microbiology data, antibiotic exposures, comorbidities, procedures, lab results and patient demographics. The total number of features used in this portion of the analysis was 788.With Boston data, a feature matrix was generated as in Kanjilal et al.38. This is particularly important so that we can take into consideration changes in the data generating process  that occur over time when estimating model performance. We did not re-weigh or re-sample our training data according to class balance in an attempt to preserve model calibration on our test set39.With Stanford data we split the cohort by year into training (2009–2017), validation (2018), and test (2019) sets to mimic distributional shifts that occur with deployment40.We selected from four model classes: L1 (lasso) and L2 (ridge) logistic regressions, random forests, and gradient boosted trees. These were specifically chosen so that we could search over model classes with different biases and variances. The L1 and L2 penalized logistic regressions assume the outcome is a linear function of the features, are less flexible, but also less prone to overfitting the data. The random forests and gradient boosted trees can model nonlinear interactions between features and outcomes, are more flexible, but more prone to overfitting. Random forests perform inference by averaging predictions from a collection of trees, and gradient boosted trees perform inference by summing the predictions of a collection of trees that each fit the residuals at the prior boosting roundk = 5 fold cross validation grid search over the training set. Hyperparameters that led to the highest mean area under the receiver operating characteristic curve (AUROC) were selected for each model class. We then fit the final model for each model class using the entire training set and evaluated each on the validation set. The best model class was chosen by selecting the model with the highest AUROC in the validation set. After choosing the best model class, hyperparameters were re-tuned on the combined training and validation set using a stratified k = 5 fold cross validation grid search.The training and model selection procedure we used for the logistic regressions and random forest is as follows. First, hyperparameters for each model class were selected by performing a stratified The training procedure was altered for the gradient boosted tree models so that we could regularize with early stopping. The training procedure was as above except that for each model fit, 5% of each training fold was held out and used as an additional validation set for early stopping. We set the maximum number of boosting iterations to 1000 and a tolerance of ten boosting rounds for the early stopping criteria.41. The gradient boosted tree models were fit using the lightgbm python package42. We computed the area under the receiver operating characteristics curve (AUROC) and average precision, with 95% confidence intervals estimated by bootstrapping the test set 1000 times43. We list all tested hyperparameter configurations in Supplementary Note The final model was then trained using the combined training and validation set and final performance was evaluated on the test set. The logistic regressions and random forest models were fit using the sci-kit learn python packagek = 5 fold cross validation grid search over the training set28.The Boston data was split into training (2007–2013) and test (2014–2016) sets by time . The optimal model class and hyperparameter setting for each of the four binary models was chosen with a We used the out of sample predicted probabilities from each of our binary classifiers to optimize antibiotic selections across patients in the test set and benchmarked against: (1) random antibiotic selections and (2) the observed clinician selections. Clinician performance was measured by extracting which antibiotics were administered to patients using information stored in the medication administration records. In the Stanford data, we restricted this analysis to admissions in the test set where one of the twelve antibiotic selections we trained models for were administered. In the Boston data, analysis was similarly restricted to patients who were prescribed one of the four antibiotics we trained personalized antibiogram models for.44. For each admission we selected an antibiotic option that maximized the predicted probability of choosing an antibiotic listed as susceptible subject to the constraints that (1) only one antibiotic option could be selected for a given patient infection and (2) the total number of times certain antibiotic options were selected across patients matched a fixed budget. In the initial simulation, this budget was defined to be the number of times particular antibiotic choices were actually administered by clinicians in the real-world data. Thus, if ceftriaxone was allocated 100 times in the data, the optimizer was similarly forced to allocate ceftriaxone 100 times. In further simulations, these budget constraints were perturbed to empirically estimate the trade-off between maximally selecting antibiotics with activity against a patient’s infection and reducing the use of broad-spectrum antibiotics. An illustrative example of how patient feature vectors are converted into antibiotic assignments is shown in Fig. The optimized antibiotic selections were generated by solving a constrained optimization formulation using linear programming tion and  the totaN be the number of admissions in our held out test set and M be the total number of antibiotic selections. Let S ∈ RN×M be a matrix of binary variables indicating antibiotic options, where sij = 1 represents that the jth antibiotic option is allocated to the patient in the ith admission. Let Φ ∈ RN×M be a matrix of out of sample probability estimates from our machine learning models, specifically ϕij represents the predicted probability that the patient in the ith admission would be covered by antibiotic option j. Finally, let Kj be the total number of times the jth antibiotic must be used over the set of our N admissions (the budget parameters). For our initial simulation we let the budget parameters match the number of times clinicians actually allocated the jth antibiotic selection over the set of N admissions. Our problem formulation specified below was implemented using the PuLP python package and solved with the CBC solver45 [Eq. In technical detail, let We performed a sensitivity analysis to estimate model performance on the full deployment population, including patients with negative microbial culture results. This is a important because at prediction time, whether a microbial culture will return positive is unknown. Further, negative microbial cultures do not preclude infection at a site not tested. Some patients with negative microbial cultures will have a latent undetected infection with an antibiotic susceptibility profile (set of labels) that goes unobserved. This can skew model performance estimates if patients with censored labels have a covariate distribution different from those with observed labels. To address this, we (1) constructed an electronic phenotype to identify patients with negative microbial cultures that truly lacked infection, (2) re-trained a new set of personalized antibiogram prediction models that include patients flagged by the electronic phenotype and (3) used inverse probability weighted estimates of AUROC to evaluate performance on the deployment population, the union of patient admissions with positive and negative microbial cultures.Coagulase-negative Staphylococci were considered negative. Antibiotics were either never administered, or they were stopped within 24 h of them starting. Antibiotics were not restarted for an additional 2 weeks if they were stopped. No ICD codes related to bacterial infection were associated with the hospital admission  and the resulting organism(s) was susceptible to the antibiotic, or the admission resulted in negative cultures and was flagged by the electronic phenotype (bucket 2). A negative label was assigned if the admission resulted in a positive microbial culture (bucket 1) and a resulting organism(s) was not susceptible to to the antibiotic.Models were trained using patient admissions in buckets 1 and 2. The covariate distribution of patient admissions in bucket 3 was used to estimate model performance on the full deployment population  using inverse probability weighting.We used inverse probability weighting to account for patient admissions whose class labels we did not observe (bucket 3) in our estimates of model performance (AUROC). In a theoretical deployment scenario, we would deploy our model on a population of patient admissions that include the union of buckets 1, 2, and 3. If the covariate distribution of patients admissions in buckets 1 and 2 differs from the covariate distribution of patient admissions in bucket 3 and our models perform better or worse in regions of this covariate distribution that are more common for patient admissions in buckets 1 and 2, then we run the risk of over or underestimating how well our models would perform in deployment.i, i’s class label, and t is the probability cut-off threshold to map predicted probability to a predicted class label. The inverse probability weighted ROC curve and area under it can be estimated by varying the probability cut-off thresholds of these estimators [Eqs. To estimate performance on a population of patients that includes a set of patients whose labels we do not observe, we weigh each patient admission whose label we do observe (bucket 1 and 2) by the inverse probability of observing it. We obtain this probability by fitting a binary random forest classifier  to predict whether the patient admission would land in bucket 1 and 2, or bucket 3. The inverse probability weighted estimates of sensitivity and specificity for each of our 12 models are shown below. 46. Specifically, an empirical distribution of random coverage rates was created by repeatedly randomly re-assigning antibiotic selections to different patient-infections 10,000 times. A pvalue was calculated by taking the fraction of coverage rates in this empirical distribution that equaled or exceeded the coverage rate achieved with personalized antibiograms. If no value in the empirical distribution equaled or exceeded the observed value, the pvalue was reported as p < 0.0001. The clinician coverage rate was compared to the personalized antibiogram coverage rate using a similar procedure, except that the empirical distribution of clinician coverage rates was generated by performing a stratified bootstrap (stratified by antibiotic selection) 10,000 times.Coverage rates of random antibiotic selection in the two cohorts were statistically compared to coverage rates achieved with personalized antibiograms with one-sided permutation testsFurther information on research design is available in the N = 8342 infections from 6920 adult patients who presented to the Stanford  emergency department or ValleyCare  emergency department in Pleasanton, California between 2009 and 2019 and were subsequently admitted to the hospital for an infection. Some patients in our dataset have multiple instances of infection spanning their medical records. We include patients with multiple infections to mimic the deployment scenario where models would perform inference on patients with new infections who had prior infections seen during training. Demographic information is listed in Table The Stanford cohort includes We train binary machine learning models  using structured electronic health record data to estimate the probability that infections would be susceptible to 12 common empiric antibiotics choices (four of which were combination therapies). Labels are derived from the antibiotic susceptibility reports of the microbial cultures. Observations are assigned a positive label when organisms isolated are deemed susceptible (not intermediate or resistant) to the respectively tested antibiotics based on microbiology lab standards.Ntrain = 5804 patient-infections from 2009 to 2017, Nval = 1218 patient-infections from 2018, and Ntest = 1320 patient-infections from 2019. In Table 40. Best model class refers to the model type that performed best in the validation set and was thus used in the final evaluation and downstream personalized antibiotic selection. The positive class prevalence  ranged from 0.23 (vancomycin) to 0.98 (vancomycin and meropenem). Average precision ranged from 0.46 to 0.99. AUROC ranged from 0.61 to 0.73.Our dataset is split by time into training, validation and test sets containing In Supplementary Fig. N = 770 observations. Antibiotic selection guided by personalized antibiograms in the Stanford data demonstrates a coverage rate of 85.9%; significantly better than random treatment selection (79.2% p < 0.0001) and comparable to clinician performance (84.3% p = 0.11). In Supplementary Fig. To estimate the potential clinical impact of prescribing guided by personalized antibiograms, we employ a linear programming based optimization procedure to simulate reassigning antibiotic selections to patients in our held out test set using personalized antibiogram estimates. We track the coverage rate (fraction of infections covered by the antibiotic selection) achieved by personalized antibigorams and compare to coverage rate achieved by clinician and random selections. The personalized antibiogram guided treatment selection and random selection are constrained such that the frequencies at which specific antibiotics are assigned to patients match the frequencies used by clinicians. We call these budget constraints. Without such a constraints, assignment of the broadest spectrum antibiotics possible would result in the highest coverage rate. We restrict this analysis to the subset of infections in our test set where one of the 12 antibiotic selections were administered. This results in a total of To assess the potential for reducing broad-spectrum antibiotics with personalized antibiograms, we sort antibiotic selections according to their antibiogram value (fraction of infections listed as susceptible) and repeat the linear programming based optimization simulations under revised constraints where the personalized antibiogram based treatment selection is forced to use fewer broad-spectrum  antibiotics in place of more narrower-spectrum  antibiotics. We track the point at which the coverage rate of the personalized antibiogram based antibiotic selections matched the real world coverage rate, and the point at which it matched the coverage rate of a random antibiotic selection. Results of these simulations are shown in Fig. We find that antibiotic selections guided by personalized antibiograms can achieve a coverage rate as good as the real world clinician prescribing rate while narrowing 69% of the vancomycin + piperacillin/tazobactam selections to piperacillin/tazobactam, 40% of piperacillin/tazobactam prescriptions to cefazolin, and 21% of ceftriaxone prescriptions to ampicillin.N = 15,806 uncomplicated urinary tract infections from 13,862 unique female patients who presented to Massachusetts General Hospital and Brigham & Women’s Hospital in Boston between 2007 and 201647. Mean age for the cohort is 34.0 ± 10.9 years, and 63.6% of infections were from Caucasian women. Further information on the exact cohort definition can be found in Kanjilal et al.28.We replicate our experiments on a held out dataset consisting of Ntrain = 11,865 patient-infections while the test contained Ntest = 3941. We train personalized antibiogram models to predict the probability that four antibiotics administered to patients in this dataset with urinary tract infection would go on to be listed as susceptible for the patient’s underlying infection. As in Kanjilal et al. the four antibiotic options are ciprofloxacin, levofloxacin, trimethoprim/sulfamethoxazole, and nitrofurantoin. Table We use the training and test set split used in Kanjilal et al. to train and evaluate personalized antibiogram models. The training set contained p < 0.0001) and real world clinician prescribing (88.1% p < 0.0001). Replications with varying linear programming budget parameters demonstrate that 93% of the total ciprofloxacin prescriptions could have been exchanged with nitrofurantoin without falling below the real world coverage rate. Similarly we show that 48% of the total ciprofloxacin and 62% of nitrofurantoin prescriptions could have been exchanged with trimethoprim/sulfamethoxazole.In Fig. 48.We conduct a sensitivity analysis with the Stanford cohort to incorporate patients with negative microbial culture results into our model performance estimates. This is done by (1) constructing an electronic phenotype to flag patients with negative microbial cultures that lacked infection, (2) training twelve new personalized antibiogram models  that include patients with negative cultures, and (3) using inverse probability weighting model performance estimatesIn Table 49. Our optimization simulations demonstrate that even with modest AUROCs, antibiotic selection informed by personalized antibiograms can match or exceed clinician performance. Furthermore, antibiotic selection guided by personalized antibiograms achieved similar coverage rates to those seen in the real world with fewer broad-spectrum antibiotics, an ongoing and vital antibiotic stewardship challenge52. An example of which is the goal to reduce empiric vancomycin utilization. While vancomycin provides effective Gram positive coverage including methicillin resistant Staphylococcus aureus (MRSA), reducing unnecessary empiric vancomycin reduces antimicrobial resistance pressure and toxicities including ototoxicity and renal toxicity53. In 2019, the proportion of Staphylococcus aureus classifying as MRSA was 22% at Stanford, which likely explains the high frequency of vancomcyin + piperacillin/tazobactam empiric antibiotic use in the emergency department54 in our test set. We demonstrated that personalized antibiogram guided antibiotic selection could reduce vancomycin + piperacillin/tazobactam use in favor of piperacillin/tazobactam monotherapy by 69% without falling below the coverage rate achieved by clinicians. Another antibiotic stewardship goal is to reduce use of fluoroquinolones for treatment of urinary tract infection55. Despite criteria outlined in the Infectious Disease Society of America guidelines, fluoroquinolone use across the United States remains high at 40.3% to treat cases of uncomplicated urinary tract infection56. In the Boston test set, flouroquinolone was used by clinicians across 33.6% of cases. We demonstrated that personalized antibiogram guided antibiotic selection could reduce ciprofloxacin use by up to 48% in favor of trimethoprim/sulfamethoxazole while maintaining the real world coverage rate. These results point toward the broader vision and impact of this work. The CDC identifies antibiotic stewardship as the most important intervention to combat the larger ecological costs of increasing microbial resistance6. In this study we empirically addressed the double-edged sword of ensuring patients receive immediate value in the clinical care they receive while minimizing overexposure of excessive broad-spectrum antibiotics. Our results indicate that personalized antibiograms could simultaneously promote antibiotic stewardship goals while ensuring, if not improving, patient safety.Antibiotic susceptibility classifiers  demonstrated modest to moderate discriminatory power in terms of AUROC, consistent with prior state-of-the-art literature. AUROC alone however is not a good determinant of clinical utilityClinician performance however is not the only baseline worth benchmarking against. In additional experiments outlined in Supplementary Notes The normal antibiogram approach, which also used our linear programming optimization procedure, outperformed the personalized antibiogram approach—though we note the comparison was not apples-to-apples as the normal antibiogram leveraged information about species identity to generate probability estimates of antibiotic susceptibility. At the time empiric antibiotic selection, the species is only suspected but not known. This analysis nevertheless demonstrates the utility of our linear programming based optimization framework in the advent of rapid diagnostic technology that can more quickly determine gram stain and species identity.The personalized antibiogram approach outperformed the guideline based approach on the sub-population of patients in our cohort hospitalized with urinary tract infection. In the same subset, the guideline based approach outperformed clinician performance. We note however the comparison of guidelines to clinicians was also not apples-to-apples because in the subgroup of patients with urinary tract infection clinicians used a different set of antibiotics than what the guideline based approach allowed. This is likely due to the fact that at the point in time clinicians ordered empiric antibiotics, the exact syndrome, to which guidelines are tailored, was only suspected and not known. This further demonstrates the need for technology to enable guided antibiotic selection that models the uncertainty of the pathogen and syndrome—like our personalized antibiograms.57. We’ve designed personalized antibiograms to fit in with existing clinical workflows. When a patient presents with potential infection, clinicians first order a set of microbial cultures and provide empiric therapy based upon personal experience, clinical practice guidelines, or institutional standards59. Personalized antibiograms are designed to provide individualized antibiotic susceptibility predictions at the point in time clinicians are already seeking external support in the existing workflow. Prediction models could be embedded in electronic health records, such as through Epic cognitive compute environments, and in real time gather patient specific features, compute and display a susceptibility probability score for each antibiotic considered21 A key direction for future research is thus to prospectively evaluate the effect of personalized antibiogram based decision support on successful antibiotic selection.The success of clinical decision support relies heavily on whether it can be successfully implemented into existing clinical workflows. Systems need to be designed such that the “right information needed to make the right decision for the right patient at the right time.” is conveyed appropriatelyOur linear programming based optimization procedure requires a specified fixed “budget” of each antibiotic choice across an entire population. In practice, patients present with infection sequentially one at a time. The optimization procedure simulated and presented here is thus not explicitly a decision algorithm intended for direct sequential use. Nevertheless, once solved with a particular set of constraints, the solution space is broken into regions that would allow antibiotic recommendations for future patients in a sequential manner.We acknowledge limitations associated with the retrospective nature of our analysis, which leveraged electronic medical record data collected in an observational manner. Though the Stanford cohort is derived from a dataset representing a comprehensive primary/secondary healthcare system, we note that there still exists a possibility that patients included in this study had medical encounters elsewhere that went unobserved. We note however that our model performance estimates reflect any error that may be induced by this phenomenon.Not all microbial cultures return positive, and those that are negative do not always indicate lack of bacterial infection. We attempted to address potential selection bias in our model performance estimates induced by this phenomenon in a sensitivity analysis estimating model performance on the full population  using inverse probability weighting. We acknowledge that this analysis relied on diagnosis codes to rule in possible infection, and requires assumptions about lack of unmeasured confounders. Nevertheless, the results of this analysis reassuringly revealed no considerable change in performance between the two settings, within the constraints of the observed data.Coagulase-Negative Staphylococci is a common contaminant in blood cultures which we handled by excluding them from our set of infections. Enterococcus is sometimes considered a colonizer in urine cultures, especially when patients lack symptoms. To address this limitation, we only included in our analysis patient encounters where prescribers clinically determined hospitalization and empiric antibiotic treatment was needed based on patient presentation. We also acknowledge that microbial culture results can yield imperfect labels due to the presence of ESCHAPPM organisms60. In-vitro these organisms may appear susceptible to cephalosporins but in-vivo can induce resistance.We acknowledge limitations associated with the use of microbial cultures as a gold standard. Positive microbial cultures do not always indicate a true clinical infection. By the nature of differing antibiotic susceptibility patterns across institutions, individual models are likely not robust to distributional shifts. We expect model performance could degrade if applied to different patient populations, though many of the risk factors for drug resistant organism infection are likely transferable to different settings . Specifically, we would expect our models trained on Stanford data to perform worse on Boston data and vice-versa. Future work could characterize the degree to which model performance decays across sites and the utility of retraining. Here however we demonstrate that the process of training personalized antibiogram models and using them to inform antibiotic selection is generalizable by repeating our analysis on held out data from an external site. We would also expect model performance to naturally degrade over time, which is why we separated our training and test sets by time to offer a more realistic appraisal of their potential future performance.In the Stanford cohort, antibiotic selection guided by personalized antibiograms achieved coverage rates similar to clinicians, while in the Boston dataset coverage rates increased. It should be noted that in the Boston dataset, clinician performance is closer to random chance. In the Stanford cohort the diversity of antibiotics commonly ordered was much greater as there was more variation in the types of cases presented as opposed to the more narrow uncomplicated urinary tract infection case in the Boston cohort. Analysis results can be sensitive to the details of any cohort definition. Nevertheless, personalized antibiograms demonstrate promise in multiple settings.Lastly, not all infections present the same risk to patients if treated with an inappropriate antibiotic regimen. Future work may benefit from leveraging the idea that some infections are more critical to treat than others.Machine learning classifiers trained using electronic health record data can predict antibiotic susceptibility for patients with positive microbial cultures. Antibiotic selection policies guided by personalized antibiograms could maintain or improve infection coverage rates while using fewer broad-spectrum antibiotics than are seen in real-world practice. Machine learning driven antibiotic selection could improve antibiotic stewardship without sacrificing, and potentially even improving, patient safety.Supplementary Data 1Supplementary Data 2Supplemental MaterialPeer Review FileDescription of Additional Supplementary FilesReporting Summary