In June 2023, text-based multiple-choice questions were submitted to GPT-4.0 and GPT-3.5. The AAO provides the percentage of humans who selected the correct answer, which was analyzed for comparison. All questions were classified by 10 subspecialties and 3 practice areas . Out of 1023 questions, GPT-4.0 achieved the best score (82.4%), followed by humans (75.7%) and GPT-3.5 (65.9%), with significant difference in accuracy rates . Both GPT-4.0 and GPT-3.5 showed the worst results in surgery-related questions (74.6% and 57.0% respectively). For difficult questions (answered incorrectly by > 50% of humans), both GPT models favorably compared to humans, without reaching significancy. The word count for answers provided by GPT-4.0 was significantly lower than those produced by GPT-3.5 ; however, incorrect responses were longer (P < 0.02). GPT-4.0 represented a substantial improvement over GPT-3.5, achieving better performance than humans in an AAO BCSC self-assessment test. However, ChatGPT is still limited by inconsistency across different practice areas, especially when it comes to surgery.To compare the performance of humans, GPT-4.0 and GPT-3.5 in answering multiple-choice questions from the American Academy of Ophthalmology (AAO) Basic and Clinical Science Course (BCSC) self-assessment program, available at  In order to understand and “generate” human-like language, LLMs are “pretrained” on a substantial volume of text data from the internet, employing a neural network architecture called “Transformer”. The latest versions of the model are GPT-3.5 and GPT-4.0, launched in March 2022 and March 2023 respectively. Both models are available and optimized for natural conversation in the ChatGPT web app; however, GPT-4.0 is currently accessed under a paid monthly subscription2. In the ophthalmology setting, ChatGPT has been tested for triaging patient symptoms3 and for answering patient and care giver questions about specific ocular diseases5. To our knowledge, four papers analyzed the performance of LLMs in answering multiple-choice questions from simulations of board certification exams11, but only one included direct comparison between humans, GPT-3.5 and GPT-4.0 in a limited sample of American Academy of Ophthalmology (AAO) Basic and Clinical Science Course (BCSC) questions9.The growing interest around ChatGPT has led to a considerable number of recent publications investigating the impact of AI language models in the medical environment, from both an educational and a workflow perspectivehttps://www.aao.org/education/self-assessments to all the subscribed members; furthermore, the comparative performance of these three respondents according to different categorization of the questions, including subspecialties and practice areas, was calculated.The purpose of this study was to compare the overall performance of humans, GPT-4.0 and GPT-3.5 on a larger sample of multiple-choice questions of the AAO BCSC self-assessment program available at https://www.aao.org/education/self-assessments) were submitted in June 2023 to ChatGPT, using both GPT-4.0 and GPT-3.5 language models.In this comparative cross-sectional study, the multiple-choice questions included in the self-assessment test from the clinical education section of the AAO official website  Cataract/Anterior Segment, (2) Cornea/External Disease, (3) Glaucoma, (4) Neuro-Ophthalmology/Orbit, (5) Ocular Pathology/Oncology, (6) Oculoplastics/Orbit, (7) Pediatric Ophth/Strabismus, (8) Refractive Mgmt/Intervention, (9) Retina/Vitreous, (10) Uveitis. Additionally, a masked investigator further classified questions according to practice area: (1) Diagnostics/Clinics, (2) Medical treatment, (3) Surgery. Since ChatGPT cannot analyze multimedia content, questions containing images were excluded from the study. Each consecutive multiple-choice question was entered in separate chat-rooms. In almost every response, ChatGPT discussed the topic of the submitted question, and then selected an option. If ChatGPT did not choose any of the provided answers, investigators resubmitted the question in a new chat, adding the prompt “You must choose one” after the question mark, to force a definite choice.Upon submitting the multiple-choice option selected by ChatGPT in the self-assessment test, the correct answer is revealed. Results for humans were obtained from past completed examinations stored in the AAO website, rather than through direct engagement of human participants: after answering each question, the percentage of humans who chose each distinct option are revealed on the website. Regrettably, the AAO does not provide any information about the number of human respondents, their level of education or degree of proficiency in ophthalmology. Questions answered incorrectly by more than 50% of human users were categorized as difficult, while those answered correctly by at least 95% of humans were considered easy.AAO questions, ChatGPT outputs and chosen answers, along with the percentage of humans who selected each multiple-choice option, were recorded. As secondary outcome, the word counts for questions and for responses by ChatGPT were calculated to assess (1) AI verbosity, (2) word count response differences for correct or incorrect answers and (3) correlation between the length of the questions and the length of the discussion provided by the AI.All data were entered into an electronic database via Microsoft Office Excel 365  and analyzed with IBM SPSS Statistics . Ethical approval and informed consent were not required, since the study did not involve human participants.P value < 0.05 was considered statistically significant.The number of correct answers from humans, GPT-4.0 and GPT-3.5 was compared by Chi-square  exempted this study from the requirement for ethical approval.https://www.aao.org/education/self-assessments included 1073 multiple-choice questions; 50 (4.7%) were excluded from the analysis because they incorporated images. In total, 1023 questions were submitted to both GPT-4.0 and GPT-3.5. Table As of June 2023, the self-assessment test at 9. The analysis indicated that to achieve 0.80 power at a 0.05 significance level, the study should include a sample size of 252 questions for comparisons between GPT-4.0 and GPT-3.5, 404 questions for GPT-4.0 vs humans, and a notably larger sample of 5566 questions for GPT-3.5 vs humans.To ensure that our study was adequately powered to detect meaningful differences between groups of responders, we conducted a power analysis based on the percentage of questions answered correctly by GPT-4.0 (84.3%), GPT-3.5 (69.5%) and humans (72.9%) in a previous study by Lin et al., who also employed the multiple-choice questions of the AAO BCSC programP = 0.25). Upon resubmitting the questions with the additional prompt “You must choose one”, a multiple-choice option was always selected.In some cases, ChatGPT did not select an answer on the first query  and GPT-3.5 (P < 0.0001). Conversely, GPT-3.5 compared unfavorably to humans (P < 0.0001).Overall, GPT-4.0 and GPT-3.5 answered correctly to 843 (82.4%) and 674 (65.9%) questions respectively, while the mean percentage of questions answered correctly by humans was 75.7 ± 17.2%, corresponding to 774 ± 176  questions. The number of correct answers by GPT-4.0 was significantly higher than both human users (P = 0.0002). In particular, GPT-3.5 obtained the highest percentage of correct answers in Ocular Pathology/Oncology  and the lowest in Pediatric Ophthalmology/Strabismus . Conversely, GPT-4.0 and humans showed more consistent results, with no significant difference across subspecialty groups (P = 0.15 and P = 0.96 respectively).GPT-3.5 performance was significantly variable across the 10 different subspecialties (P = 0.02) and Oculoplastics/Orbit (P = 0.04). A post-hoc power analysis was performed to check whether the lack of significant differences in other subspecialties could be determined by an insufficient sample size. The power analysis confirmed this suspicion, showing low statistical power for the comparisons in most subspecialty groups.Despite the fact that GPT-4.0 performed better than humans in all subspecialties, the difference was statistically significant only in Glaucoma (P = 0.002), Glaucoma (P = 0.02), Neuro-Ophthalmology/Orbit (P = 0.003) and Pediatric Ophthalmology/Strabismus (P = 0.0004). Table GPT-3.5 showed worse performance, obtaining significantly lower scores than humans in Cataract/Anterior Segment (entclass1pt{minimaP = 0.03 and P = 0.04 respectively). GPT-4.0 and GPT-3.5 obtained the best scores in Diagnostics/Clinics  and the worst in Surgery . Conversely, human users achieved better consistency, with no significant difference across practice areas (P = 0.86). More specifically, GPT-4.0 performed better than humans in Diagnostics/Clinics (P = 0.0002) and Medical treatment (P = 0.08), while GPT-3.5 obtained significantly lower scores than humans in all practice areas . Table GPT-4.0 and GPT-3.5 showed significant difference in performance among practice areas (entclass1pt{minimaentclass1pt{minimaP > 0.05). Difference between GPT-4.0 and GPT-3.5 was also not significant (P = 0.46).In total, 92 out of 1023 (9.0%) questions were answered incorrectly by more than 50% of human users. The majority of these questions were included in the Refractive  and the Ocular Pathology/Oncology  subspecialties. In these difficult cases, the mean percentage of correct answers for humans was 40.1 ± 7.5%. Both GPT-4.0 and GPT-3.5 performed better than humans in these questions , but without reaching significancy , however the difference between GPT-4.0 and GPT-3.5 almost reached a significant level (P = 0.059). Figure Overall, 120 out of 1023 (11.7%) questions were answered correctly by 95% or more humans. To evaluate major mistakes, the number of times ChatGPT chose the wrong answer to these questions was tracked. The mean percentage of correct answers for humans in this subset was 96.4 ± 1.2%; only 2 questions were answered incorrectly by GPT-4.0 (98.3%), and 9 by GPT-3.5 (92.5%). There was no significant difference between both GPT models and humans (P < 0.0001). Furthermore, there was a significant increase in word count in responses generated for questions that were answered incorrectly compared to those that were answered correctly, for both GPT-4.0  and GPT-3.5 . According to univariate analyses, the word count for responses of both GPT-4.0 and GPT-3.5 was positively correlated to the word count of the questions .On average, the word count for answers provided by GPT-4.0 was significantly lower than those produced by GPT-3.5 .This study compared the performance of humans, GPT-4.0 and GPT-3.5 in a large ophthalmology test based on multiple-choice questions. Instead of running a simulation inside the AAO BCSC web app, which includes 260 random questions extracted from a database of 4500 + entries, the more comprehensive self-assessment test available at The difference in correct answers among the three study groups was statistically significant: GPT-4.0 achieved the best score (82.4%), followed by humans (75.7%) and GPT-3.5 (65.9%). Interestingly, both GPT-4.0 and GPT-3.5 showed less consistent results across different practice areas compared to humans. Diagnostics/Clinics was the easiest practice area (83.7% and 68.1% respectively), with Medical treatment following closely (83.4% and 65.1%). Instead, Surgery was the hardest area for both models (74.6% and 57.0%). It is possible to hypothesize that answers for clinical questions are easier to deduct from web resources, while surgery may involve highly specialized techniques and protocols that may not be well-represented in the data AI language models were trained on. Likewise, surgical knowledge often relies on images or videos, which were not included in the training database. Concerning subspecialties, GPT-3.5 also demonstrated significant inconsistencies, whereas GPT-4.0 achieved a degree of consistency much closer to human performance.The performance of GPT-4.0 and GPT-3.5 was further analyzed by considering only difficult questions, defined as questions answered incorrectly by more than 50% of human users. Remarkably, both models performed better than humans in this challenging subset, but the difference was not statistically significant . The outcomes for the easiest questions, defined as those answered correctly by 95% or more humans, was comparable among AI models and human users .Although in multiple-choice questions the word count of ChatGPT responses has no impact on the selection of a correct answer, succinctness still plays a significant role in user experience, especially in the healthcare environment. Clear and concise answers are generally preferred, as long as the complexity of the topic is respected. Overall, the length of ChatGPT answers was directly proportional to word count of the questions, with GPT-4.0 outputs being significantly more succinct than GPT-3.5 (160 ± 56 vs. 206 ± 77 words). On the other hand, responses were significantly longer for questions answered incorrectly in both language models. Although explaining the technical reasons behind this behavior goes beyond the scope of this study, it was observed that excessive verbosity was often associated with evasive and generic responses, hinting at potential uncertainties in the resolution of the questions.9.Other authors have performed similar research to assess the potential of ChatGPT in ophthalmology. Lin et al. compared the performance of GPT-4.0, GPT-3.5 and humans in the setting of the AAO BCSC Self-Assessment Program Simulated Exam (260 questions). The three groups of respondents scored 84.3%, 69.5% and 72.9% respectively. These results are consistent with our findings ; however, Lin et al. marked non-answers as incorrect, while we included the additional prompt “You must choose one” to force an answer. Such difference in design limits the relatability of the comparison between the two studies. Furthermore, Lin et al. took an alternative approach to assess the difficulty of each question: instead of evaluating difficulty based on the percentage of human users who answered correctly to the questions, a masked investigator classified questions as first-order  or higher-order  problem-solving12. A limitation of this study lies in the fact that the AAO self-assessment test exclusively reveals the percentage of humans who selected a certain multiple-choice option, without disclosing the number of respondents and their degree of knowledge. Additionally, the reproducibility of ChatGPT answers was not assessed herein, although a previous study by Antaki et al. reported almost perfect repeatability6.More recently, Moshirfar et al. evaluated the performance of GPT-4.0, GPT-3.5 and humans in answering 467 questions from the StatPearls question bank, respectively scoring 73%, 55% and 58%. While a direct comparison is not completely appropriate, since two different datasets of questions were used, scores for all respondents were lower than what we found . In this study, the difficulty score, ranging from 1 to 4, was provided by StatPearls. Results for humans were not affected by difficulty grading, while GPT-3.5 showed a notable decline in performance as the level of difficulty increased. This trend was much less evident for GPT-4.013. Low quality data may introduce algorithmic bias, possibly reinforcing old, outdated knowledge. Finally, AI can generate hallucinations, consisting of completely fabricated information, factual inaccuracies, logical inconsistencies or nonsensical responses14. Malevolent uses of ChatGPT are also conceivable, posing a serious threat for the integrity of scientific research. For instance, many authors reported how ChatGPT could produce seemingly authentic scientific manuscripts or abstracts, avoiding plagiarism checks18.Despite the great potential of generative AI, many challenges and outstanding issues are emerging. First, the source of the massive amount of data used to train LLMs was not subjected to rigorous quality control, with high risk of false or misleading information. In this regard, it is crucial to understand that ChatGPT does not really “understand” questions; instead, it answers based on probability correlations of text found within the training datasets and statistical patterns5, as well as assisting ophthalmologists in clinical decision-making, research and administration20. Acknowledging ChatGPT shortcomings, such as hallucinations, and potential abuses, will be crucial for the successful integration of this technology in the medical setting21.In conclusion, GPT-4.0 represented a substantial improvement over GPT-3.5, achieving higher performance compared to humans in the self-assessment tests for the American Board of Ophthalmology examination. However, ChatGPT is still limited by performance inconsistency across different practice areas, especially when it comes to surgical/practical knowledge. Currently, the utility of ChatGPT extends beyond that of a comprehensive repository of ophthalmological knowledge. AI based language models demonstrated remarkable capabilities for real-time data interpretation, with the potential to revolutionize teleophthalmology services by providing almost instant responses to patient queries


Multi-parameter control of light is a key functionality to modulate optical signals in photonic integrated circuits for various applications. However, the traditional optical modulators can only control one or two properties of light at the same time. Herein, we propose a hybrid structure which can modulate the amplitude, wavelength and phase of surface plasmon polaritons (SPPs) simultaneously to overcome these limitations. The numerical results show that when the Fermi level of graphene changes from 0.3 to 0.9 eV, the variation of optical transmission, wavelength and phase are 32.7 dB, 428 nm and 306°, respectively. The demonstrated structure triggers an approach for the realization of ultracompact modulation and has potential applications in the fields of optical switches, communications and photo-detection. Effectively manipulating of these parameters at micro/nanoscales is of great significance and has been widely exploited in optical communication8, optical sensing11, photo-detection12 and so on. Surface plasmon polaritons (SPPs), originating from the interaction between free electrons of metal and the electromagnetic waves, hold a more potential applications at micro/nanoscales because they can break the conventional diffraction limit14. Herein, the active control of SPPs has attracted great attention of researchers16.As the essential property of electromagnetic field, polarization, wavelength, amplitude and phase reflect a wealth of applications in optical perception and operation19. For example, Zhuang Ren’s group demonstrated an active and smart electro-optic THz modulator, which is based on a strongly correlated electron oxide vanadium dioxide (VO2). With milliampere current excitation on the VO2 thin film, the transmission, reflection, absorption and phase of THz waves can be modulated efficiently. In particular, the antireflection condition can be actively achieved and the modulation depth reaches 99.9%, accompanied by a 180° phase switching20. Ali Forouzmand’s group proposed an electrically tunable amplitude and phase modulators, which are designed by the hybridization of indium tin oxide (ITO) into a guided-mode resonance mirror. A gate-tunable amplitude modulator with a modulation depth as high as ∼ 0.80 is realized, and the phase variation of ∼ 210° is accomplished when the applied bias voltage alters from − 15 V to + 24 V21. However, these proposed modulators can only control one or two parameter of light despite its superior performance, which greatly hinders their widely application23. Herein, a modulator which can control more properties of light will attract much attention of experts.Currently, much work has been done in multi-parameter modulation to meet the increasing demand for various application scenarios25.In this paper, a hybrid silicon-dielectric-graphene-grating structure is proposed, which can modulate the amplitude, wavelength and phase of SPPs simultaneously. The SPPs is stimulated by the grating and propagating on the graphene, the Fermi level of the graphene is controlled by the voltage applied between silicon substrate and graphene. The numerical results show that when the Fermi level changes from 0.3 to 0.9 eV, the variation of optical transmission, wavelength and phase are 32.7 dB, 428 nm and 306°, respectively. This structure has the potential application in the fields of optical switches, communications and photo-detection2) exhibit an excellent performance such as low leakage current, high dielectric strength and low amounts of defects, we select the CaF2 as the dielectric layer, which could improve the performance of modulator effectively26.In Fig. 0 and ɛr are the charge constant, the permittivity of vacuum and the relative permittivity of CaF2 respectively, V and d are the voltage and thickness of CaF2. The relation between Fermi level and the carrier density is presented in Eq. , and the two-dimensional 2D) simulation model is shown in Fig. D simulatFor the model of anisotropic graphene, its out-of-plane permittivity is set to 2.5, and the in-plane conductivity can be obtained from random-phase approximation, including the effect of finite temperature (T = 300 K):g and kB are the conductivity of graphene and the Boltzmann’s constant. Ef and ɷ are the Fermi energy level and radian frequency, respectively. The carrier relaxation time τ = μEf/evf2, where the Fermi velocity vf = 106 m/s and carrier mobility μ = 10,000 cm2/(V s). The effective permittivity of graphene ɛg can be described by means of the following expression:0 and tg = 0.33 nm represent the vacuum permittivity and thickness of graphene respectively27.Here σFigure 5 V/m, while it is 1.37 × 105 V/m when the Fermi level increases to 0.6 eV. The electric field intensity reaches 1.53 × 105 V/m as the Fermi level increases to 0.8 eV, and it has changed about 23.3%. It can be concluded that with the increase of the Fermi level, the amplitude of the SPPs increases gradually.Figure Figure 32. The wavelength changes by 428 nm when the Fermi level of graphene increase from 0.3 to 0.9 eV (226 nm at 0.3 eV while 654 nm at 0.9 eV). The results verify that the wavelength of the SPPs can also be modulated by the demonstrated structure effectively.Figure Figure d is 1.6 μm and 1.8 μm, respectively (the operating wavelength is fixed at 8 μm). The phase is − 139° when the Fermi level of graphene is 0.3 eV, and the phase is 167° when the Fermi level increases to 0.5 eV at the propagation length of 1.6 μm. The phase changes at right port changes can reach to 306°. . Thus, the amplitude, wavelength and phase of propagating SPPs in our model can be simultaneously controlled through the Fermi level of graphene by changing the applied voltage, which will have a potential application in modern integrated optics and communication33.The phase of SPPs can be affected by both the refractive index and the propagation length of graphene due to the change of optical path. Figure In conclusion, we have demonstrated a silicon-dielectric-graphene-grating hybrid structure. The amplitude, wavelength and phase of SPPs can be modulated simultaneously when the SPPs is propagating along the graphene. The numerical results show that when the Fermi level of graphene changes from 0.3 to 0.9 eV, the optical transmission, wavelength and phase are changed by 32.7 dB, 428 nm and 306°, respectively. The realization of multi-parameter modulation in SPPs will have a promising application in the field of optical communication, sensor and photo-detection.2 is set as 6.76. The mesh size is 1/5 of one wavelength.The simulation has been performed using the commercial finite element method (FEM), trial version of software “COMSOL Multiphysics 5.5”. In simulation, the scattering boundary condition and user-defined port are used. The module of radio frequency is used to investigate the relationship between voltage and optical transmission, wavelength and phase. The surface conductivity model and the transitional boundary condition are used in the model of graphene. In addition, the permittivity of CaF


Inflammatory breast cancer (IBC) is one of the most lethal subtypes of breast cancer (BC), accounting for approximately 1–5% of all cases of BC. Challenges in IBC include accurate and early diagnosis and the development of effective targeted therapies. Our previous studies identified the overexpression of metadherin (MTDH) in the plasma membrane of IBC cells, further confirmed in patient tissues. MTDH has been found to play a role in signaling pathways related to cancer. However, its mechanism of action in the progression of IBC remains unknown. To evaluate the function of MTDH, SUM-149 and SUM-190 IBC cells were edited with CRISPR/Cas9 vectors for in vitro characterization studies and used in mouse IBC xenografts. Our results demonstrate that the absence of MTDH significantly reduces IBC cell migration, proliferation, tumor spheroid formation, and the expression of NF-κB and STAT3 signaling molecules, which are crucial oncogenic pathways in IBC. Furthermore, IBC xenografts showed significant differences in tumor growth patterns, and lung tissue revealed epithelial-like cells in 43% of wild-type (WT) compared to 29% of CRISPR xenografts. Our study emphasizes the role of MTDH as a potential therapeutic target for the progression of IBC. Inflammatory breast cancer (IBC) is a very aggressive and rare subtype of breast cancer (BC) with a poor prognosis and accounts for 1–5% of all cases of BC. The 5-year survival rate for people with IBC is 41% compared to non-IBC (nIBC) ,3. HowevPreviously, our group characterized the cell surface proteome of IBC. We identified metadherin (MTDH) overexpression in IBC cell lines vs. non-cancerous mammary epithelial cells, and nIBC cell lines . SubsequMetastasis is the leading cause of death in BC patients . IBC patThis study investigates the role of MTDH in IBC by assessing its function in cell proliferation, migration, tumor spheroid formation, signaling of oncogenic pathways, tumor progression, and metastasis. We develop isogenic models of the SUM-149 and SUM-190 IBC cell lines using MTDH-CRISPR/Cas9 base knockout for in vitro and in vivo studies. To validate the oncogenic role of MTDH, we also overexpressed MTDH in the non-cancerous mammary epithelial cell line MCF-10A. Our findings show that the absence of MTDH significantly decreases IBC cell proliferation, migration, invasion, and tumor spheroid formation and decreases the expression of various signaling molecules. In addition, we demonstrate that MTDH affects invasion and metastasis. To our knowledge, this study is the first to assess the role of MTDH in the invasion and metastasis of IBC. Thus, this study provides significant new evidence for the field of IBC, contributing to studying possible new targets involved in IBC progression.In a previous study, we demonstrated that MTDH is overexpressed in IBC cells and patient tumor tissues compared to normal tissues and in the tumor emboli. Furthermore, we reported that MTDH overexpression was found in the plasma membrane of IBC cells, specifically in SUM-149 cells, compared to the non-cancerous mammary epithelial cell line, MCF-10A . MoreoveSeveral publications have highlighted MTDH as an important target in the treatment of BC, including its role in cancer onset and progression ,32. TherPrevious studies have shown that IBC cells can spontaneously form 3D tumorspheres in vitro ,34,35. TMTDH overexpression has been associated with increased cancer cell metastasis . TherefoSTAT3 and NFKB1 genes in both IBC cell models. For SUM-149, there is a trend (p = 0.10) of STAT3 downregulation (−2.00 f.ch.) in sg-MTDH cells. Moreover, the expression of the NFKB1 gene is significantly reduced by −2.11 f.ch.  and MCF-10A cells (Ctrl and OE-MTDH) cells C. DensitMTDH has been correlated with BC progression and poor overall survival in patients , but itsFirst, the results showed that the health of the mice was not compromised by the absence of MTDH A. A reduTo validate that SUM-149 sg-MTDH cells remained with a low expression of MTDH after 10 weeks of study, we performed a qPCR for MTDH gene expression. A significant reduction (−5.52 f.ch.) was observed E in mousHigh expression of MTDH has been associated with tumor metastasis, decreased survival outcomes, and higher mortality in female reproductive cancers such as breast, ovarian, and cervical . We prevOur results demonstrated that MTDH depletion decreased the proliferation of the TN and HER2+ IBC models and their ability to form colonies. At the same time, MTDH overexpression promoted the proliferation of non-cancerous mammary cells. Although MTDH has been correlated with BC proliferation by showing high levels of Ki-67 in tissues, previous studies demonstrated that knocking down of MTDH did not affect the proliferation of MDA-MB-231-LM2 cells, a subcell line of nIBC with a high propensity to lung metastasis ,46,47, iHerein, we also established that depletion of MTDH decreased the migration and invasion of IBC cells while overexpression of MTDH promoted the migration of non-cancerous cells. Studies demonstrate that MTDH is relevant in cancer cell invasion, since Matrigel invasion assays have shown that different cancer cells, such as hepatocellular carcinoma and glioma cells, display an increased invasive ability through MTDH overexpression . AdditioNFKB1) and the expression of NF-κB, AKT, ERK, and STAT3 in a TN IBC model. In a HER2+ IBC model, no changes in gene expression were observed and only a down-regulation of NF-κB, AKT, and ERK proteins was detected. The contrasting results in gene expression between the IBC models could be explained by previous studies which have demonstrated that the amplification or overexpression of HER2 promotes the activation of NF-κB and STAT3 in contrast to TNBCs  as described in [Female severe combined immunodeficient mice (SCID)  between 21 and 28 days of age were housed under specific pathogen-free conditions. Mice received an irradiated AIN 76-A phytoestrogen-free diet  and water ad libitum. To test the effects of MTDH silencing in IBC tumor formation and progression, we injected 1.5 × 10ribed in ,73. Grouribed in . At the TM, General Data Company Inc., Cincinnati, OH, USA). Sectioning was performed using a Leica CM 1860 cryostat  by cutting 8 μm sections of the left and right lung of mice injected with WT SUM-149 or MTDH-edited cells (n = 7/group). The sections were mounted on slides with a subbing solution as described in [Sectioning: The excised lungs were left in 10% neutral-buffered formalin (NBF) overnight and then immersed in a solution of 20% sucrose in 1X PBS. The lungs were then placed in optimal cutting temperature compound (OCT) . A pathologist analyzed H&E-stained slides and the micrographs were captured at a magnification of 20×. Hematoxylin & Eosin (H & E) staining: First, tissues were stained with Mayer hematoxylin solution (Sigma-Aldrich) for 10 min at RT and then rinsed with running tap water for 10 min until the water was colorless. The tissues were then rinsed in 95% alcohol for 10 s. Then, the staining was performed with Eosin Y with Phloxine B (Sigma-Aldrich) solution for 30 s and rinsed in ascending series of ethanol . Finally, tissues were processed in xylene 1 min twice and mounted using VectaMountp-value for the in vitro studies was calculated using Student’s t-tests, ordinary-one way or two-way analysis of variance (ANOVA) with the Bonferroni multiple comparison test. Gene expression studies for each cell line or tumor were evaluated using the 2−ΔΔCt formula by comparing their relative gene expression to the reference genes. Data are expressed as Mean ± S.E.M. Each experiment was carried out in three or more independent biological replicates. Statistical analyses were performed using Graph Pad Prism v.9.0 , and differences were considered significant when p < 0.05.The In vivo studies: Summary statistics were performed to describe the tumor volume and weight. The normality evaluation was performed using the Shapiro–Wilk test. The presence of outliers was verified using a generalized extreme studentized deviation (ESD) test. Mean changes were evaluated over time using a general linear model with repeated measures. Model diagnostics were performed as well, i.e., Cooks distances and DF betas were ascertained to access influential values. A pairwise comparison was performed to determine the difference between the groups and each week. Only if the data did not follow a normal distribution, Wilcoxon–Mann–Whitney tests were performed. The significance level (α) was set to 0.05. Analyses were performed using the statistical software R version 4.1.0.Our findings identify for the first time a potential role of MTDH in promoting IBC proliferation, migration, invasion, and metastasis by regulating NF-κB and STAT3. Furthermore, we showed that MTDH plays a role in the invasive capacity of IBC promoting metastasis to the lung. Altogether, these findings open the possibility of a new target to better understand the molecular biology of IBC. MTDH has the potential to serve as a diagnostic target to diagnose and treat the progression of this deadly disease in IBC patients.


We analyzed the X-ray photoelectron spectra (XPS) of carbon 1s states in graphene and oxygen-intercalated graphene grown on SiC(0001) using Bayesian spectroscopy. To realize highly accurate spectral decomposition of the XPS spectra, we proposed a framework for discovering physical constraints from the absence of prior quantified physical knowledge, in which we designed the prior probabilities based on the found constraints and the physically required conditions. This suppresses the exchange of peak components during replica exchange Monte Carlo iterations and makes possible to decompose XPS in the case where a reliable structure model or a presumable number of components is not known. As a result, we have successfully decomposed XPS of one monolayer (1ML), two monolayers (2ML), and quasi-freestanding 2ML (qfs-2ML) graphene samples deposited on SiC substrates with the meV order precision of the binding energy, in which the posterior probability distributions of the binding energies were obtained distinguishably between the different components of buffer layer even though they are observed as hump and shoulder structures because of their overlapping. Especially, due to the short escape depth of photoelectrons excited by soft X-rays, XPS has been applied to various surface and interface analysis such as film thickness3, chemical states at surfaces or interfaces6, and atomic distortion at the interface7. The elemental information measured by XPS is revealed by line-shape analysis for the obtained spectrum. Although the regression analysis of XPS spectra is a non-linear regression, the least squares method has been used to minimize the fitting error until now. In an analysis using the least-squares method, it is not always possible to obtain a globally optimal solution, and its reproducibility is frequently low because it depends on the initial values. It is also difficult to incorporate known physical property information such as constraints on the parameters, and it is impossible to evaluate the accuracy of the parameter estimation. In the analysis of XPS, it is essential to develop a highly reproducible analysis method that properly incorporates known physical property information9. In this paper, to solve these problems, we apply Bayesian spectroscopy to analyze the XPS spectra of one monolayer (1ML), two monolayers (2ML), and quasi-freestanding 2ML (qfs-2ML) graphene layers grown on SiC substrates and attempt to extract changes in the chemical states of the graphene layers by chemical modification at the interface.X-ray core-level photoelectron spectroscopy (XPS) is a popular and powerful tool for investigating the elemental composition of materials5. Riedl et al. proposed that graphene C 1s spectra have four peaks, in addition to SiC and graphene (Gr), two additional components S1 and S25. S1 comes from the C atoms bound to one Si atom on the surface of SiC(0001) and to three C atoms in the buffer layer. S2 comes from the remaining sp10, O12, Ge13, Si14, Au15 and Bi16 have been intercalated between the buffer layer and SiC. When atoms are intercalated beneath the buffer layer, dangling bonds of Si are terminated by intercalated atoms to break covalent bonds between the buffer layer and SiC. Since the charge transfer into the graphene layer is also modified by the intercalation, the charge neutrality level can be controlled around the Dirac point artificially. However, the transport properties are often degraded after the interface modification. To enhance the transport property of graphene, precise control and characterization of the chemical states should be performed at the intercalated interface. The bonding at the modified interface would differ depending on the intercalated atoms and on the treatment conditions.High-quality, large-scale graphene can be formed by thermal decomposition of SiC at elevated temperatures. It is known that buffer layer are formed between graphene and the SiC substrate and have an equivalent structure to graphene, although the buffer layer adhere strongly to the SiC substrate, while dangling bonds remain with the SiC substrate due to lattice mismatchIn such cases, the reliable structure model for the intercalated interface would often be lacking. Thus, an alternative method is strongly required to decompose core-level photoelectron spectra even in the case where a reliable structure model or a presumable number of components is not known. However, we have been forced to analyze the lineshape with the number of components and the constraint parameters assumed to validate a plausible structure and the experimental setting. Thus, there might be a concern of containing subjective and empirical arbitrariness in the fitting results in the XPS spectra. In addition, if the line-shape analysis was treated as a black-box tool due to its complexity, that would yield incorrect results. Therefore, a reproducible and reliable approach without arbitrariness is required for XPS spectral analysis.17. 3d core-level XPS spectra of 17. In Japan, efforts to realize spectral decomposition based on Bayesian inference have begun in synchrotron radiation facilities18.Recently, the result of Bayesian spectral deconvolution for core-level XPS has been reported to realize automatic analysis of core-level XPS spectra by incorporating the effective Hamiltonian into the stochastic model of spectral deconvolution9. However, even in such a situation, scientists can achieve high-precision spectral analysis by the following scenario. They first analyze the spectral data without assuming prior knowledge. By reviewing the results of this analysis, they quantify physical constraints that were previously unnoticed or unquantifiable. Utilizing such knowledge, for example, by constraining the range of regression parameters, they can achieve the analysis of spectral data with the desired accuracy. The purpose of this study is to propose a framework for incorporating this natural flow of spectral data analysis conducted by scientists into Bayesian spectroscopy20.The difficulty in performing such Bayesian inference is in designing the prior probability distribution. Since prior probability distributions can restrict the range of parameters, if the constrain condition for their parameters is known in advance, it can be incorporated into the prior probability distribution. Otherwise, a distribution that does not affect parameter estimation, such as a wide uniform distribution, is used as the prior probability distribution. In that case, the posterior probability distribution may exhibit multimodality due to exchange among spectral components. As a result, parameter estimates are not precise, are poorly reproducible, and may not be physically reasonable. When analyzing the XPS spectra, it is important to overcome these difficulties and quantify knowledge of the target materialIn this article, core-level spectra in both pristine and oxygen-intercalated graphenes grown in SiC(0001) have been analyzed by Bayesian spectroscopy with constraints on the values of the spectral parameters based on knowledge of the physical properties.n-doped 6H-SiC(0001) using the face-to-face method21, where two SiC substrates were placed one on top of the other with a gap of 20 22. The core-level and valence-band photoelectron spectra were measured using a photon energy of 680 and 40 eV, respectively. The Fermi energy and the energy resolution were confirmed by measurements for the Fermi level of the gold reference. The overall energy resolutions were estimated to be 0.69 and 0.04 eV for core-level and valence band measurements, respectively.Graphene layers were grown on 23 for the 1ML and 2ML samples, which are shown in red for the SiC substrate, blue for graphene (Gr), green for S1 and magenta for S2, respectively. Although the peak positions are shifted about 0.2 eV comparing the vertical dashed lines with the peak positions observed in Fig. Figure 23) gives the strongest peak at 283.8 eV, and the peak structures of the graphene (284.67 eV23) and of the two components S1 (285.04 eV23) and S2 (285.53 eV23) for buffer layer atoms are observed as broad peaks from 284.5 to 286.0 eV without separation. In the 2ML sample shown in Fig. 23) gives a second intense peak at 283.3 eV, which is the almost same as that of 1ML. However, the spectral structure in the high-binding energy region changes markedly, giving a dominant peak at 284.8 eV. Although this peak is considered to be chiefly attributed to graphene (284.56 eV23), and also includes the components of the buffer layer S1 (285.01 eV23) and S2 (285.50 eV23) because it has a shoulder structure on the high energy side.In the 1ML sample depicted in Fig. 24.The surface cleanliness and the thickness of graphene were checked by the core-level and valence-band photoemission measurements. No contamination-related features such as an O 1s peak were found in the core-level photoemission spectra. Angle-resolved photoemission spectra (ARPES) showed clear dispersions originating from the 26, it is not known whether the Si dangling bonds remain in the sample qfs-2ML after annealing. The XPS spectrum in the qfs-2ML sample is similar to that in the 2ML sample, as seen in Fig. 24 and the objective is to confirm the validity of our proposed framework.The buffer layer at the interface between SiC and graphenes consists of a carbon layer in a graphene-like honeycomb arrangement that bonds covalently to the Si-terminated substrate partially. For the 1ML and 2ML samples, not all Si atoms can bond to carbon atoms due to the different lattice constants between SiC and graphene and due to the 27. However, applying uniform distribution of the prior probability of energy E, the width of the red component is large and the obtained solution is not reasonable as a result of XPS as shown in the left of Fig. E is inhibited because the posterior distribution of the red component obtained by the exhaustive search becomes two-modal and one distributed near the blue component as shown in Fig. E. When we have constrain conditions for E: each energy difference is greater than Figure entclass1pt{minimaentclass1pt{minimaentclass1pt{minimaentclass1pt{minimaK is a subscript for model identification. Based on Bayesian spectroscopy is a spectral decomposition analysis method that incorporates a Bayesian inference framework. Let 29, the posterior probability distribution b is the quasi-inverse temperature27 defined as an inverse variance i according to a normal distribution with zero mean and variance Z19 is obtained by marginalizing the numerator of Eq. (19 (BFE) is defined as F, the estimation of the noise intensity 30 method according to Eq.  and Gaussian G(x) shapes, with their intensity A, binding energy E, spectral width w at full width at half maximum (FWHM) and a mixing ratio C in Eq.  and estimation of the binding energies has been achieved with high precision of the order of meV. From their binding energies, we estimated the existence ratio of the buffer layers S1 for S2 with its standard deviation, which is consistent with previous studies. We performed model selection to determine the number of components in the XPS spectrum of the qfs-2ML sample. The four-peak model is selected, however, the contribution of S2 is quite small and this is probably considered due to the heterogeneity of the qfs-2ML graphene sample. These results demonstrate that the appropriate design of the prior probability distributions based on the information of physical properties and the insights gained from the exhaustive search is effective to perform the spectral decomposition with high precision.


Artificial Intelligence chatbots such as ChatGPT are increasingly being utilized for medical information by the public as well as by medical professionals, including researchers. We investigated the accuracy of information and citations generated by this search engine for a common medical condition, UTI.This was a qualitative pilot evaluation of an artificial intelligence chatbot for answering and referencing clinical questions on UTI. We created 5 clinical questions (Table) related to UTI diagnosis and treatment. Two clinical infectious diseases experts graded the responses on a Likert scale as to accuracy of the response. The gold standard for responses was derived from IDSA guidelines and/or evidence based electronic resources (e.g. UpToDate). Citations were checked for accuracy using PubMed searches, Google Scholar and also, in one case, direct communication with the purported author.ChatGPT Performance on UTI QuestionsChatGPT answered 3 of the 5 questions mostly correctly . The response to one question was rated as only half correct . One question was answered 100% correctly. Incorrect information was primarily related to diagnostic approaches to urine cultures for both asymptomatic and symptomatic patients (Table), and did not reflect answers found in current guidelines.When prompted for supportive evidence of answers, ChatGPT created plausible, yet fabricated studies. The authors were real and well-known experts in the field, but the combinations of authors, journal title and paper title were often false. Citations were incorrect for all 5 questions and fabricated in 12 of 24 provided citations.ChatGPT performed surprisingly well on correctly answering basic diagnosis and management questions on UTI, a commonly encountered condition with a large body of literature to train from. The lack of current information from guidelines hampered accuracy and could be a barrier to appropriate use of this technology for real-time learning or clinical decision-making. The inaccuracy of citations limits its use for supporting evidence-based research and publications.Kalpana Gupta, MD MPH, GSK: Advisor/Consultant|Iterum Therapeutics Limited: Advisor/Consultant|Qiagen, Inc.: Advisor/Consultant|UpToDate Wolters Kluwer Health: Royalties


BackgroundThe application of artificial intelligence (AI) in education is undergoing rapid advancements, with models such as ChatGPT-4 showing potential in medical education. This study aims to evaluate the proficiency of ChatGPT-4 in answering Saudi Medical Licensing Exam (SMLE) questions.MethodologyA dataset of 220 questions across four medical disciplines was used. The model was trained using a specific code to answer the questions accurately, and its performance was assessed using key performance indicators, difficulty level, and exam sections.ResultsEasy and Average questions, but accuracy decreased for Hard questions. Performance was consistent across all disciplines, indicating a broad knowledge base. However, an error analysis revealed areas for further refinement, particularly with category (Option) A questions across all sections.ChatGPT-4 demonstrated an overall accuracy of 88.6%. It showed high proficiency with ConclusionsThis study underscores the potential of ChatGPT-4 as an AI-assisted tool in medical education, demonstrating high proficiency in answering SMLE questions. Future research is recommended to expand the scope of training and evaluation as well as to enhance the model’s performance on complex clinical questions. OpenAI introduced ChatGPT, a comprehensive large language model (LLM), to the public in November 2022. Often referred to as GPT-3.5, this model underwent training on vast text datasets using both supervised and unsupervised learning methods and then refined using reinforcement learning based on human input. A distinguishing feature of ChatGPT is its inability to browse the web, setting it apart from other chatbots with internet access. While OpenAI’s proprietary version of ChatGPT has online querying capabilities, the public version does not currently offer this feature, though it is anticipated in forthcoming releases. On March 14, 2023, OpenAI unveiled GPT-4, another LLM trained in a manner consistent with its previous model [Lately, several teams have assessed ChatGPT’s efficacy on national medical licensing tests, encompassing exams from countries such as China, Japan, and Germany -8. AlthoThe Saudi Medical Licensing Exam (SMLE) is a comprehensive evaluation mechanism implemented by the Saudi Commission for Health Specialties to qualify physicians, whether domestic or international, to practice medicine in Saudi Arabia, or to permit postgraduate medical students to apply for local residency programs. It is a compulsory examination for all prospective doctors during their internship as part of their bachelor’s degree. The exam’s structure was created by the SMLE steering committee, which includes members from the Saudi Medical College Deans. The exam is designed to test a specific learning domain, encompassing not only knowledge but also cognitive abilities  and attitude . The SMLThe objective of this study is to evaluate the performance of ChatGPT-4, a state-of-the-art AI language model developed by OpenAI, in answering questions from the SMLE. This assessment aims to understand the model’s accuracy, proficiency, and reliability in interpreting and responding to complex medical questions that are used to gauge the competency of medical graduates in Saudi Arabia. The results from this study will offer insights into the potential applications of using advanced AI models such as ChatGPT-4 in medical education and testing.This study was designed to evaluate the efficacy of the ChatGPT-4 model in answering multiple-choice questions (MCQs) with four options, i.e., A, B, C, and D, from the SMLE.Data collectionhttps://canadaqbank.com/smle.php). This dataset was accessed on July 9, 2023. Four researchers collaborated in the data collection process to ensure comprehensive coverage of questions and eliminate potential bias in question selection. The questions were derived from various sections of the SMLE, including Medicine, Pediatrics, Obstetrics and Gynecology, and Surgery, to ensure diversity in subject matter. Each question was accompanied by four potential answers , with only one correct response.A total of 220 questions were extracted from the SMLE test bank available on the CanadaQBank website . This decision was made to ensure that our study accurately reflected the conditions and challenges of the real examination. Furthermore, the sections from which we distributed were as follows: 30% of the questions were from Medicine, 25% were from Obstetrics and Gynecology, 25% were from Pediatrics, and the remaining 20% were from Surgery. The questions were chosen based on the guidelines provided in the Commission’s handbook . This apModel training and evaluationThe ChatGPT 4.0 model was trained using the given codes, which defined its role as a medical professional with the knowledge required to answer SMLE questions accurately and coherently. We gave ChatGPT 4.0 the following training code: “You are a medical professional, I will give you Saudi Medical License Exam (SMLE) questions and four multiple choice for each question. You have the knowledge required to choose one single answer in an accurate, coherent manner. I will provide you with the questions from multiple tests on various topics such as medicine, surgery, obstetrics and gynecology, and pediatrics. I will provide you with the questions and I want you to choose the correct answer.” Without this training code, the ChatGPT 4.0 model would not be able to answer the questions accurately. Instead, it might default to its general behavior, which includes referring the user to a specialist doctor for consultation. This training code is crucial as it provides the model with specific information about the nature of the questions and their source, and instructs it to select one correct answer from the four options provided. This prompt provided the model with information about the nature of the questions and their source. It also directed the model to select one correct answer from the four options provided.Following the training phase, the model was evaluated on its ability to select the correct answer for each question. Performance metrics, such as accuracy, precision, and recall, were computed to assess the model’s overall performance. The model’s performance was further evaluated across different question difficulty levels and sections.Statistical analysisData analysis was performed using SPSS, version 26.0 . Given the descriptive nature of our study, we primarily focused on descriptive statistics to gain insights from our dataset. Specifically, frequencies and percentages were calculated for all categorical variables to offer a comprehensive understanding of the distribution and characteristics of the data.We calculated key performance indicators to gauge the model’s performance. The metrics we focused on included accuracy and precision for each response category . Accuracy was determined as the proportion of total correct answers provided by the model out of all the questions it attempted. This gave us a general sense of how often the model was correct in its predictions. Precision, on the other hand, was calculated for each category. It represented the ratio of true positives  to all instances that the model classified as positive for that category. This gave us an understanding of how precise the model was in its predictions for each specific category.Easy, Average, and Hard question categories. This analysis was intended to discern any possible correlation between the intricacy of a question and the model’s propensity to answer it correctly. Finally, the performance was assessed according to the different sections of the SMLE. This analysis revealed how well the model responded to questions related to Medicine, Pediatrics, Obstetrics and Gynecology, and Surgery, thus providing an indication of its strengths and areas for improvement in different medical disciplines. Through these detailed statistical analyses, a comprehensive understanding of ChatGPT-4’s capabilities in answering SMLE questions was attained.Furthermore, an error analysis was undertaken to discern the types and frequencies of mistakes made by the model. This was crucial in pinpointing the specific areas that might benefit from further refinement. We also evaluated the model’s performance based on the difficulty level of the questions. We computed the mean accuracy for the We utilized a dataset comprising 220 questions, divided as per the SMLE blueprint, to evaluate the performance of the ChatGPT-4 model. The results of this evaluation are organized by key performance indicators, performance across different levels of question difficulty, and performance based on the section of the exam.ChatGPT’s key performance indicatorsThe overall success rate of ChatGPT-4 in correctly answering the questions was found to be 88.6%. We also measured how well ChatGPT-4 did for each question category (options), A, B, C, and D. The performance of the ChatGPT-4 model was evaluated across four response options . For option A, out of 52 instances, the model correctly answered 43 (82.7%) questions and incorrectly answered nine 17.3%). In the case of option B, out of 45 instances, the model correctly answered 42 (93.3%) questions and incorrectly answered three (6.7%). For option C, out of 42 instances, the model correctly answered 37 (88.1%) questions and incorrectly answered five (11.9%). Lastly, for option D, out of 81 instances, the model correctly answered 76 (93.8%) questions and incorrectly answered five (6.2%). These results provide a comprehensive understanding of the model’s performance across the different response categories  difficulty-level questions, out of 53 instances, the model correctly answered 50 questions, resulting in a success rate of 94.3%. It incorrectly answered three questions, yielding an error rate of 5.7%. For Easy difficulty-level questions, the model showed exceptional performance. Out of 54 instances, it correctly answered all questions, achieving a success rate of 100%. For Hard difficulty-level questions, out of 113 instances, the model correctly answered 94 questions, achieving a success rate of 83.2%. It incorrectly answered 19 questions, resulting in an error rate of 16.8% (Figure We next examined how ChatGPT performed for questions of different difficulty levels. To ascertain the difficulty level of each question, we turned to the historical performance metrics provided by the CanadaQBank website. For every question within their database, CanadaQBank not only indicates the proportion of users who answered the question correctly but also categorizes the questions based on this success rate. This categorization, rooted in real-world performance, serves as a reflection of each question’s complexity. CanadaQBank’s method ensures that the difficulty categorization of each question is based on actual user performance, eliminating subjective biases and offering a consistent, objective measure of a question’s challenge level. For Performance based on exam sectionWe then examined how well ChatGPT-4 did based on the exam section the questions came from. The model did well across all sections. In the Medicine section, out of 66 instances, the model correctly answered 57 questions. It incorrectly answered nine questions, yielding an error rate of 13.6%. In the Obstetrics and Gynecology section, out of 55 instances, the model correctly answered 53 questions. It incorrectly answered two questions, resulting in an error rate of 3.6%. In the Pediatrics section, out of 55 instances, the model correctly answered 49 questions. It incorrectly answered six questions, yielding an error rate of 10.9%. In the Surgery section, out of 44 instances, the model correctly answered 39 questions. It incorrectly answered five questions, yielding an error rate of 11.4% (Figure Error analysisFinally, we examined the types of errors made by ChatGPT-4. An error analysis was conducted to identify the types and frequencies of errors committed by the ChatGPT-4 model across different sections and response categories.In the Medicine section, the model made errors primarily in category A, with an error rate of 26.7%. Categories C and D also saw error rates of 13.3% and 11.5%, respectively. Category B had no errors. In the Obstetrics and Gynecology section, the model made a few errors in categories B and C, with error rates of 8.3% and 7.1%, respectively. There were no errors in categories A and D. In the Pediatrics section, the model made the most errors in category A, with an error rate of 20%. Categories B, C, and D saw fewer errors, with rates of 6.7%, 12.5%, and 5.9%, respectively. In the Surgery section, the model made errors primarily in category A, with an error rate of 16.7%. Categories B and C also saw errors, with rates of 12.5% and 20.0%, respectively. Category D had a low error rate of 5.3% Figure .This error analysis provides insights into specific areas where the model could benefit from further refinement. It appears that the model struggles the most with category A questions across all sections, suggesting that future training could focus on improving performance in this area.Our findings demonstrate the substantial potential of ChatGPT in assisting medical education, particularly in the context of preparing for exams such as the SMLE. This study presents a comprehensive evaluation of ChatGPT’s performance on a set of 220 SMLE questions, covering four major medical disciplines, namely, Medicine, Pediatrics, Obstetrics and Gynecology, and Surgery.Easy and Average questions suggests that it could be a valuable asset for foundational learning and ongoing review. As we continue to refine and train the model, we anticipate that its applicability in medical education and assessment will broaden, eventually revolutionizing the way we prepare for exams and engage in lifelong learning.The overall accuracy of the model was 88.6%, showing a high degree of proficiency in answering the SMLE questions correctly. While our study showcases the high potential of ChatGPT-4 as a supplementary educational tool, the margin of error of 4-16% emphasizes that it should be used cautiously and in conjunction with traditional study methods and expert consultation. Given the present limitations, ChatGPT-4 may not yet be ready for standalone use in high-stakes exam preparation or medical decision-making. However, its high level of accuracy in answering Easy and Average questions, suggesting a strong grasp of foundational medical knowledge. However, the accuracy decreased for Hard questions, indicating that the model might benefit from additional training on complex clinical scenarios or rare diseases. When analyzed based on the subject matter of questions, the model’s performance was fairly consistent across Medicine, Pediatrics, Obstetrics and Gynecology, and Surgery. This suggests that the model has been successful in acquiring a broad knowledge base in diverse medical fields, supporting its utility as a comprehensive learning tool. The integration of AI into medical education, as noted by Davenport and Kalakota [Our observations are consistent with the findings of Kung et al. , who conKalakota , has sigKalakota . The preOur study does have certain limitations. First, our study trained ChatGPT-4 using a limited number of SMLE questions. Increasing the volume of questions for training might improve the model’s performance. Second, while our study included scenario-based MCQs, it did not incorporate other formats like open-ended or short-answer questions. Therefore, our findings may not be fully generalizable to assessments that use diverse question formats. Third, we evaluated the performance of ChatGPT-4 on questions from only four medical disciplines. Including questions from other medical and surgical sub-specialties could provide a more comprehensive understanding of the model’s capability.Hard questions suggests it may benefit from exposure to more complex and nuanced medical datasets. By training ChatGPT with a higher proportion of intricate clinical scenarios, it may become better attuned to understanding and answering challenging questions. Incorporation of Feedback Loops: Implementing a feedback mechanism where medical professionals can review and correct the model’s outputs, especially on the Hard questions it gets wrong, can help it learn from its mistakes. Over time, this iterative feedback can help the model improve its accuracy in identifying the nuances of tougher questions. The model demonstrated high accuracy with Easy and Average questions but had reduced accuracy with Hard questions. Analysis of the model’s performance by subject matter indicated consistent efficacy across all four disciplines. The significance of this study extends beyond merely quantifying the performance of an AI language model in answering medical examination questions. In the field of medical education, there is a growing need for innovative, scalable, and time-efficient assessment tools, particularly for formative assessments that facilitate learning in addition to measuring performance. Our study’s implications are broad and relevant for both medical students and educators. For students, ChatGPT-4 can act as a real-time, supplementary self-assessment tool that helps identify both strengths and areas for improvement. For educators, the model offers a highly accurate and efficient means to design and validate assessments. While promising, we recommend additional studies and refinements before this technology is widely incorporated into medical education and assessments. Its high accuracy in Easy and Average questions could be leveraged to create baseline assessments, freeing educators to focus on designing more complex and nuanced questions to challenge their students. The study opens a promising avenue for further research into how AI-assisted tools can be effectively integrated into medical education and assessment practices. While our findings indicate strong potential, we recommend additional refinement and validation before the model can be widely adopted in educational settings.This study evaluated the performance of ChatGPT-4, an AI language model, in answering SMLE questions. The study involved a set of 220 SMLE questions encompassing four medical disciplines, namely, Medicine, Pediatrics, Obstetrics and Gynecology, and Surgery. Statistical analyses using R software revealed that the overall accuracy of ChatGPT-4 in answering the questions was 88.6%. An error analysis identified areas where the model could benefit from further refinement. Here are some potential pathways for refinement: Focused Training on Complex Scenarios: The model’s reduced accuracy with


Create a timeline of diagnosis and treatment for IPF in the US.A retrospective analysis was performed in collaboration with the OptumLabs Data Warehouse using an administrative claims database of Medicare Fee for Service beneficiaries. Adults 50 and over with IPF were included (2014 to 2019).To focus on IPF, the following diagnoses were excluded: post-inflammatory fibrosis, hypersensitivity pneumonitis, rheumatoid arthritis, sarcoidosis, scleroderma, and connective tissue disease.Data were collected from periods prior, during, and following initial clinical diagnosis of IPF. This included prior respiratory diagnoses, number of respiratory-related hospitalizations, anti-fibrotic and oxygen use, and survival.A total of 44,891 with IPF were identified. The most common diagnoses prior to diagnosis of IPF were upper respiratory infections (47%), acute bronchitis (13%), other respiratory disease (10%), chronic obstructive pulmonary disease and bronchiectasis (7%), and pneumonia (6%). The average time to a diagnosis of IPF was 2.7 years after initial respiratory diagnosis. Half of patients had two or more respiratory-related hospitalizations prior to IPF diagnosis. Also, 37% of patients were prescribed oxygen prior to diagnosis of IPF. These observations suggest delayed diagnosis. We also observed only 10.4% were treated with anti-fibrotics. Overall survival declined each year after diagnosis with median survival of 2.80 years.Our retrospective cohort demonstrates that IPF is often diagnosed late, usually preceded by other respiratory diagnoses and hospitalizations. Use of available therapies is low and outcomes remain poor.The online version contains supplementary material available at 10.1186/s12890-023-02565-7. Idiopathic pulmonary fibrosis (IPF) is the most common type of fibrosing interstitial lung disease affecting an estimated 2.8–9.3 cases per 100,000 people per year [Differentiating IPF from other causes of ILD is important as it can lead to different approaches to treatment and altered prognosis . TreatmeAs it is widely held that patients with IPF experience diagnostic delays based on data gathered from academic centers, we sought to describe the timeline to diagnosis and initiation of treatment for patients with IPF using real world datasets. Our retrospective cohort analysis is unique because we used data from the Center for Medicare Services Fee for Service database, which consists of over 50,000 patients with IPF across the United States. Therefore, the current study aimed to investigate the identified patients’ initial respiratory diagnosis, subsequent time to clinical diagnosis of IPF, timing of oxygen initiation and prescription of anti-fibrotic agents. We also obtained information regarding respiratory-related hospitalizations and survival.To refine our IPF code identification strategy, an initial local cohort analysis of 200 consecutive patients coded for IPF were first analyzed by medical record review. This retrospective cohort study included patients > 18 years of age who were present within our institution’s electronic record system between 2011 and 2019. ICD 9 and 10 codes for IPF (516.3 and J84.122) were used to identify the validation study cohort. Each patient’s clinical history, laboratory results (including serologies for connective tissue disease and hypersensitivities), exposure history, CT scan pattern, lung biopsy results, and competing diagnoses were reviewed independently by three physicians , and consensus diagnoses derived  data. The database contains longitudinal health information on enrollees, representing a diverse mixture of ages, ethnicities, and geographical regions across the United States . Since tWe included all adult patients 50 years or older who had their first coded clinical diagnosis of IPF between January 1, 2014, and December 31, 2019. We then constructed an incident cohort by requiring individuals to have five years of continuous enrollment prior to their IPF diagnosis. IPF was identified using the following International Classification of Diseases, Ninth Edition (ICD-9) diagnosis codes: 516.31; and International Classification of Diseases, 10th Edition (ICD-10) diagnosis codes: J84.112, as previously described [Descriptive variables of interest at index IPF diagnosis included age, gender, race/ethnicity, region of residence, and history of smoking . We further evaluated respiratory-related diagnoses and hospitalizations at baseline and after IPF diagnosis. Respiratory-related hospitalizations were identified using AHRQ’s Clinical Classification Software .The data from this observational cohort analysis are reported using descriptive statistics including frequencies and percentages or average mean values. We used the Fisher’s exact test for multiple group comparisons . In addiThe funding source for this study played no role in study design; collection, analysis, or interpretation of the data; writing of the report; and decision to submit this paper for publication.Our initial local cohort analysis described above allowed us to exclude other competing diagnoses and focus more specifically on IPF  of patients with IPF had other initial respiratory diagnosis in their baseline period before the index diagnosis of IPF. The most common respiratory-related diagnoses were upper respiratory infections (47%), acute bronchitis 13%), other upper respiratory disease (10%), chronic obstructive pulmonary disease and bronchiectasis (7%), and pneumonia (6%). Additional diagnoses can be found in Table %, other The average time from initial respiratory-related hospitalization to diagnosis of IPF was 786 days or 2.2 years. Most patients with IPF were hospitalized with a respiratory-related diagnosis between baseline and follow-up . A majority had a respiratory-related hospitalization prior to a diagnosis of IPF  and many had additional hospitalizations after their diagnosis of IPF . Furthermore, many patients had multiple hospitalizations.  used oxygen at some point in their disease course. Overall, 37% of the cohort received oxygen prior to their diagnosis of IPF , with the remainder initiating oxygen therapy either on or after their diagnosis of IPF . For those who initiated oxygen use prior to their IPF diagnosis, the average time between first oxygen use and IPF diagnosis was 814 days or 2.2 years Fig. . Those wSurvival declined drastically each year after respiratory hospitalizations, with an approximately 50% survival rate 2 years after the first respiratory related hospitalizations Fig. . The medIt has been suggested that the diagnosis of IPF has been increasing in prevalence over time . DespiteOur examined data (from 2014 to 2019) found a total number of patients with IPF = 44,891, yielding a prevalence estimate of 3 per 100,000 persons, per year. However, our cohort of patients with IPF  is certainly a significant underestimate of the total numbers with IPF. There are several reasons that indicate that the total of 44,891 patients in our cohort is an underestimate of the total numbers of patients with IPF. First, we required five years of consecutive enrollment and data to better understand the diagnostic timeline. In addition, we employed a comprehensive exclusion of competing diagnoses based upon our initial review of complete records on consecutive patients in our local electronic health records. Indeed, the available data initially searched yield over 90,00 patients with a code for IPF. However, our intent was to have a rigorously defined cohort of IPF with adequate timeline data to analyze diagnostic delays, treatment use, and survival. For these reasons, this estimate of IPF prevalence cannot be directly compared to other studies .Our data demonstrated that many patients received other respiratory diagnoses prior to their coded clinical diagnosis of IPF. The majority of these were pulmonary fibrosis unspecified, post-inflammatory fibrosis, chronic obstructive pulmonary disease unspecified, and interstitial pulmonary disease unspecified. It is also interesting that 19% of patients had three or more respiratory-related hospitalizations prior to diagnosis as well. Furthermore, roughly 37% of patients were already prescribed oxygen at the time their coded IPF diagnosis, with 27% of those receiving oxygen having been prescribed this therapy for 4 or more years prior to their diagnosis. These observations further support the conclusion that the diagnosis of IPF is rendered extremely late in the disease course.It is concerning that only a small percentage of patients ultimately received anti-fibrotic treatment. Prior studies indicate that pirfenidone and nintedanib use is associated with slowing of lung function deterioration, as well as reduced short-term mortality and reduction in hospitalizations –6, 12. IPrevious studies have also suggested gender differences in the initiation of anti-fibrotic therapy, with women less likely to receive anti-fibrotic therapy compared to male patients with IPF , 12. In The prognosis for IPF from this real-world patient cohort was similar to other cohorts described, with a mean 2.88-year prognosis following clinical diagnosis . This agOur stated goal was to evaluate the timeline to diagnosis and therapy in patients with IPF. To accomplish this, we excluded patients with coexisting diagnostic codes for other fibrotic lung diseases including chronic hypersensitivity pneumonitis and connective tissue disease-associated lung fibrosis (Supplementary Figure Additional studies using different methodologies have also documented delays in the diagnosis of interstitial lung disease and idiopathic pulmonary fibrosis. The INTENSITY study reported a survey of 600 subjects with diagnosed interstitial lung disease, finding that over half of patients had received at least one misdiagnosis, with delay to the current diagnosis occurring a median of seven months after the onset of symptoms and with 43% being delayed over a year . AnotherThis study does have several limitations. In a claims-based cohort dataset, we have no way to review the clinical notes and records to determine whether the medical team considered IPF earlier than when the code appeared in the claims-based record system. Furthermore, the respiratory related hospitalizations were determined using AHRQ software. We do not claim or report that the hospitalizations were solely due to IPF. They could have been coded as due to pneumonia, asthma exacerbation, post inflammatory fibrosis, COPD, or other diagnoses. It is not possible to retrospectively determine whether the hospitalizations were due to IPF alone from review of this claims-based data. That noted, we employed the greatest precision possible to determine the timing of diagnosis of IPF and related clinical events such as oxygen use, antifibrotics, hospitalizations and death.In addition, as an observational retrospective dataset review, our findings rely on the validity of accurate recording and diagnostic coding and miscoding is a possibility which would impact our analysis. To mitigate this, we performed a local cohort code validation study to exclude concurrent competing codes to refine our study group to patients with IPF. Despite our concerted efforts to refine our study cohort, it remains possible that some patients with competing fibrotic diagnoses such as fibrotic NSIP, may have been included. Furthermore, as a descriptive survey study, no control group was required or reported and confounding factors could have impacted the results. We would also note that it also remains possible that some of prior respiratory diagnoses did represent simple infections rather than a true misdiagnosis of IPF. Finally, our analysis was limited to OptumLabs and Medicare Fee for Service data and may not be generalizable to patients that are uninsured or included under Medicaid based coverage. Our analysis was therefore limited to the United States and may not be generalizable to other parts of the world as access to healthcare and medication costs differ greatly across different geographic settings.Despite increasing knowledge regarding the risk factors and available treatments for IPF, delays in the diagnosis of this disorder and the initiation of therapy persist. Furthermore, prognosis remains poor following diagnosis. In addition, anti-fibrotic therapy is underutilized in our cohort. We hope to reduce the delay to IPF diagnosis and treatment initiation by increasing the awareness of care of IPF patients and by enhancing education for providers on the appropriate diagnostic approach.Below is the link to the electronic supplementary material.Supplementary Material 1