The Education and Training Centre for Public Healthcare of the National Medical Centre plays a key role in providing continuing professional development (CPD) to 221 public health and medical institutions in South Korea. To assess the realization of the Centre’s core value and the intended changes, program evaluations are required. The context, input, process, and product (CIPP) model is particularly suitable for evaluating CPD in the public sector, as it allows for recognizing the dynamic nature of the program environment.This research applied the CIPP model to the evaluation of CPD programs, particularly abdominal and thoracic ultrasound programs implemented in 2017 and 2018. Data were collected from 2017 to 2019. The program and its feedback were reviewed in the context evaluation. Based on this, a subsequent program strategy was established for the input evaluation. Observing the program in real time and recording its progress was followed in process evaluation. Finally, the outcomes and impacts of the program were reviewed and compared with baseline data in the product evaluation.In context evaluation, the educational needs of the Centre’s CPD program recipients, impediments that inhibit participation in education, and resources that the Centre can utilize were identified through an online survey, focus group interviews and expert consultation. Through input evaluation, we identified the best alternative that satisfied all pre-selected criteria, which were responsiveness to priority system needs, potential effectiveness, fit with existing services, affordability, and administrative feasibility. Observing the program in real time and recording its progress were conducted in process evaluation, demonstrating that the augmented program went as planned, and even had to be expanded due to increased demand. The impact of the program was measured, interpreted, and assessed in the product evaluation. The review committee decided that the intended change had been occurred, thus the Centre decided to maintain the program.A thorough evaluation is necessary to determine the potential benefits of CPD. The CIPP methodology is valuable for executing formative and summative evaluations. The CIPP model is particularly useful for securing accountability data for large-scale nationwide educational programs supplied by public funds.The online version contains supplementary material available at 10.1186/s12909-022-03271-4. Continuing professional development (CPD) for physicians is important to achieve safe, effective, and improved clinical care . HoweverThe Education and Training Centre for Public Healthcare at the National Medical Centre (NMC), hereafter referred to as the Centre, plays a key role in providing CPD to 221 public health and medical institutions (PHMIs) in South Korea. PHMIs are established and operated by the state, local governments, and public institutions . The CenThe Centre’s core values are to focus on enhancing publicness in healthcare and improving the quality of medical services . EnhanciProgram evaluations should assess how well the Centre’s core values are realized and whether the intended changes have occurred. Educational programs are fundamentally focused on change; therefore, program evaluations should assess whether the desired changes occur as a result of educational initiatives . The CPDAmong the diverse methodologies –11, the Informed by the complexity theory , 15, theWe applied the CIPP model to evaluate CPD programs implemented by the Centre in 2017 and 2018. The Centre offered 17 programs in 2017 and 10 programs in 2018. Among these programs, we evaluated abdominal and thoracic ultrasound programs because major changes were made to these programs, in terms of strategy, educational goals, content, and methodology in 2018, following a needs assessment in 2017, which showed the strongest demand for change among other programs. Educational evaluation is useful for examining the effectiveness of a program after changes are made and for determining whether to continue the changes.To answer the evaluation questions in each phase of the CIPP model, data were collected from learners, instructors, operators, stakeholders, and external experts between May 2017 and January 2019  from December 5, 2017, to March 8, 2018, to achieve in-depth interpretations of the survey results. A total of 106 valid responses were received from the 111 participants.The FGIs targeting learners followed a democratic view of educational needs assessment approaches . This apIn the input evaluation, the most cost-effective and feasible strategy was selected to best satisfy the needs identified through context evaluation. Our online survey responses and FGI-based needs assessment indicated that the ultrasound program was most in need of improvement. Therefore, we performed an input evaluation for the ultrasound program. A literature review, expert consultation, and stakeholder workshops were conducted to identify the strategy selection criteria . Upon coProcess evaluation was conducted by providing an ongoing assessment of the plan implementation, followed by documentation of the process . Acting The goal of product evaluation is to measure, interpret, and assess the outcomes and impacts of a program . Upon thIn the context evaluation, the educational needs of the recipients of the Centre’s CPD program, impediments inhibiting participation in education, and resources that the Centre can utilize were identified. Our online survey results were analyzed for educational needs assessment Table . Due to Similar results were derived using FGIs, which were conducted to achieve an in-depth interpretation of the online survey results . Many reThe ultrasound educational program was the course that respondents most wished to take because it was the most applicable in primary medical care. However, the ultrasound program was cancelled several times in 2017 because of a lack of enrolment. This indicates that the content and delivery of education can prevent learners from enrolling in a program, despite high awareness of its utility, which is problematic.Whether it is applicable in the practical field is also a key factor prominent in FGIs. The groups, who responded to the overall curriculum in 2017 were good but not well-applied, pointed out that they could not use their learning in the practical field due to differences between the environment of a hospital in Seoul and a remote local hospital. The environment, human resources, facilities, and even patient groups are vastly different from that of a university hospital in Seoul and a local hospital in remote areas. In addition, interest and practical applications are prominent factors in promoting learners’ participation. Therefore, to recruit physicians as potential learners and draw good responses, educational content that can be applied in the environment where they work must be developed.In consultation with experts, expanding accessibility and providing educational content tailored to the roles played by public health institutions in the community were highlighted. In particular, significant gaps were observed in the actual clinical environment and patient groups between the learners and university hospital where the educational program was outsourced.In the literature review, two goals for PHMIs were revealed as immediate challenges: ameliorating the financial independence of hospitals by securing sufficient profits and pursuing public interest in improving the health of the local community –21. The In addition, labor productivity was found to greatly influence public hospital profits. Among them, the labor productivity of physicians accounted for a significant part of PHMI’s labor productivity . Due to The abdominal and thoracic ultrasonography program was determined to require the most changes in needs assessment due to the largest gap between willingness to attend the program and actual participation rate. Moreover, the Centre can potentially be held accountable for cancelled programs if poorly planned during inspections by the government administration. Therefore, we reviewed records from the 2017 educational program. Initially, the Centre planned to deliver three ultrasonography courses for a total of 10 classes. However, only six classes were completed, which leads to a potential accountability issue. Financial support from the Ministry of Health and Education and administrative support from the Centre were secured for the program.en bloc to a single contractor. While the Centre provided administrative support, it selected one academic society in a related field that best fulfilled the criteria.Through input evaluation, we identified the best alternative that satisfied all preselected criteria. After consultation with external experts and workshops with stakeholders, responsiveness to priority system needs, potential effectiveness, fit with existing services, affordability, and administrative feasibility were selected as criteria. The abdominal and thoracic ultrasound program was maintained in 2018 because it fulfilled the priority system needs, thereby enabling the quick screening of emergency patients. However, it was clear that the program required improvement. We selected educational materials and providers through open competitive bidding by using a targeted outsourcing strategy for the abdominal and thoracic ultrasound course only. This strategy was determined to be superior to the outsourcing strategy used in 2017 when a total of 17 CPD programs were outsourced One academic society fulfilled the criteria of responsiveness to priority system needs and fit with existing services by setting the educational goal as differentiation of emergency diseases and achievement of skills required for quick referral in primary care. It also fulfills the criteria for potential effectiveness by establishing a practice-centric educational method. By shortening the educational period from four nights and 5 days in 2017 to one night and 2 days, we satisfied the criteria of potential effectiveness and fit with existing services. The educational program described in the initial proposal was affordable and administratively feasible. For these reasons, the academic society was selected as the provider of educational content. In addition, considering the demand for improved geographical accessibility, we decided to provide education not only in Seoul, but also in other areas.After adopting these changes, the revised abdominal and thoracic ultrasound program was introduced in 2018. This program proceeded as planned and even had to be expanded due to increased demand. The initial goal was to educate 24 clinicians in two sessions with 12 students each. However, due to the increased accessibility of the program, compared with the 2017 program in terms of time and location, more students signed up than expected. More sessions were scheduled, thus bringing the program operation rate to 150% and the education recipient rate to 187%, with both being higher than those in 2017 Table .Table 3PTwo of the authors observed the sessions in real-time and recorded progress. The learners, instructors, and operators were requested to complete the free-answer questionnaires. The data obtained were then analyzed during a meeting at the Centre, where concerns were raised that the necessary capabilities may not have been sufficiently trained due to the shortened education period. However, the responses from learners during the sessions indicated that they were not only satisfied but also quite confident about their learning. The instructors believed that the learners had demonstrated the required level of competence. Freely-answered questionnaire responses were collected from all instructors, learners, and operators. The instructors said that the program was highly effective, citing intense concentration and enthusiasm among the learners. The learners were satisfied with the learning content that could be applied in daily clinical practice and the diverse case studies they were provided. Many learners left comments thanking the instructors and operators. The operators described the vibrant interactions between the instructors and learners.Goal attainment was assessed by examining post-education improvements in academic achievement. When comparing pre- and post-education academic achievements, the scores for abdominal ultrasound course and thoracic ultrasound course increased from 3.04 to 7.75 and from 3.07 to 8.06, respectively. Academic achievement, measured based on self-efficacy, increased from an average of 3.06 out of 10 to an average of 7.9 post-education. The educational satisfaction levels for the 2017 and 2018 programs , as well as higher levels of participant satisfaction and cost-effectiveness.This study has limitations because it analyzed only one program provided by the Centre  over a short span of time. Educational programs are constantly affected by factors within and outside the program. Therefore, it is clear that numerous programs provided by the Centre must have mutually affected each other. Needs assessment was carried out through surveys and FGIs, with the subject being the entire educational program provided by the Centre.The scope of this study was limited to an ultrasound educational program that was identified as a high priority through needs assessment. Selecting one course for the program cannot expand its positive effects to the entire program (10 courses in 2018). In addition, this narrow scope may have made it impossible to analyze the complexity and interrelations between programs. Moreover, the FGIs conducted in 2017 were based on voluntary participation, thus only 10% learners participated in the FGIs. Therefore, we cannot dismiss the possibility of bias toward the inclusion of enthusiastic learners. Long-term outcomes, such as the impact on patient treatment, were not evaluated. The study was based on data collected May 2017 to January 2019, when the 2017 program was initiated to when the applications of practical skills were surveyed for the 2018 program. Therefore, the long-term outcomes or impacts of the program have not been fully examined. Nonetheless, one obstetrician-gynecologist reported improvements in the diagnosis and survival rates of emergency patients after receiving ultrasound education.Despite the limitations mentioned above, the study has strengths in being comprehensive, having conducted evaluations at all phases beginning with the previous educational program, improvement and implementation of the revised program, and evaluation of the revised program after implementation. In addition, ongoing program evaluation is carried out based on a shared understanding, thus determining whether year-long education is being provided as planned can be reviewed later. We assessed not only whether the intended outcomes were achieved, but also how and why they were achieved, thus the process can be reproduced in the future, and accountability data can be secured in the future as well.The CIPP approach is suitable for the evaluation and improvement of CPD in public health. It enables the analysis of interactions between internal and external factors affecting educational programs, as well as the context in which the program is provided. Therefore, the CIPP approach helps to produce desired changes in educational programs and evaluate and improve programs funded by taxpayer money because accountability data are generated while simultaneously executing formative and summative evaluations. CPD programs administered by the Centre have the potential to improve quality of care provided by physicians in Korean PHMIs, and eventually to enhance public health. The educational program we examined exhibited notable improvement after the application of the CIPP approach, thus we were successful in realizing our goals.Additional file 1.


BCR-ABL1, that encodes the tyrosine kinase oncoprotein BCR-ABL1. This led to the development of tyrosine kinase inhibitors (TKI), with Imatinib being the first TKI approved. Although the vast majority of CML patients respond to Imatinib, resistance to this targeted therapy contributes to therapeutic failure and relapse. Here we review the molecular mechanisms and other factors  involved in TKI resistance, the methodologies to access these mechanisms, and the possible therapeutic approaches to circumvent TKI resistance in CML.Chronic myeloid leukemia (CML) is a myeloproliferative neoplasia associated with a molecular alteration, the fusion gene BCR-ABL1 fusion gene, encoding the cytoplasmic chimeric BCR-ABL1 protein that displays an abnormally high tyrosine kinase activity. Although the vast majority of patients with CML respond to Imatinib, a tyrosine kinase inhibitor (TKI), resistance might occur either de novo or during treatment. In CML, the TKI resistance mechanisms are usually subdivided into BCR-ABL1-dependent and independent mechanisms. Furthermore, patients’ compliance/adherence to therapy is critical to CML management. Techniques with enhanced sensitivity like NGS and dPCR, the use of artificial intelligence (AI) techniques, and the development of mathematical modeling and computational prediction methods could reveal the underlying mechanisms of drug resistance and facilitate the design of more effective treatment strategies for improving drug efficacy in CML patients. Here we review the molecular mechanisms and other factors involved in resistance to TKIs in CML and the new methodologies to access these mechanisms, and the therapeutic approaches to circumvent TKI resistance.Resistance to targeted therapies is a complex and multifactorial process that culminates in the selection of a cancer clone with the ability to evade treatment. Chronic myeloid leukemia (CML) was the first malignancy recognized to be associated with a genetic alteration, the t. This translocation originates the  Chronic myeloid leukemia (CML) is a clonal hematopoietic stem cell (HSC) neoplasia characterized by an increase in myeloid lineage cells at all differentiation stages . This myBCR-ABL1. This fusion gene emerges from the juxtaposition of the proto-oncogene 1 non-receptor tyrosine kinase, ABL1 gene, at chromosome 9 with the activator of RhoGEF and GTPase, BCR gene, on chromosome 22 [With the discovery of the Philadelphia (Ph) chromosome in 1960 , CML wasosome 22 . This onosome 22 . This acThe development of a small molecule with the ability to block the BCR-ABL1 activity dramatically changed the disease course, and CML gradually became a chronic disease ,10. ThisCurrently, five TKIs are approved for CML treatment. These TKIs are Imatinib, Dasatinib, Nilotinib, Bosutinib, and Ponatinib. Imatinib, Dasatinib, Nilotinib, and Bosutinib are the current first-line treatments approved by the FDA and EMA for the treatment of CML . The evoResistance to target therapy is a complex and multifactorial process that culminates in the selection of a cancer clone with the ability to evade treatment . In CML,BCR-ABL1 transcripts and the detection of BCR-ABL1 kinase domain (KD) mutations enable timely therapy switches and selection of the most appropriate treatments [The quantification of eatments . Althougeatments  and the eatments .BCR-ABL1 mutations and overexpression, abnormal activity of drug transporters, activation of alternative signaling pathways, DNA repair, and genomic instability, epigenetic dysfunction, leukemia stem cell (LSC) persistence, and dysfunction of the immune system , a mutation in the TKI binding site. This is the most frequently detected mutation among resistant patients (frequency between 4 and 20%) [BCR-ABL1 point mutations can be classified into five categories: (I) mutations in the P-loop (ATP-binding site); (II) mutations that directly affect the binding of TKI (drug contact site); (III) mutation in the catalytic domain (C-loop); (IV) mutations on the activation (A)-loop; (V) mutations in myristate pocket [The first described and 20%) ,30. T315e pocket ,32. Mutae pocket . These me pocket .BCR-ABL1. However, even with these new-generation TKIs, some mutations, such as T315I, remain resistant to multiple TKIs [BCR-ABL1 mutants are resistant to at least one single-agent TKI, but Eide et al. (2019) proposed that combination treatment in particular with Ponatinib and Asciminib can be a strategy to overcome this type of resistance [BCR-ABL1 mutational analysis is recommended for patients who fail Imatinib or second-generation TKIs and those who progress to accelerated phase (AP) or BC [The knowledge of this resistant mechanism has justified the development of second and third-generation TKIs, able to overcome and inhibit mutated ple TKIs . The majsistance . AccordiP) or BC . HoweverP) or BC . In thisBCR-ABL1 is not exclusive, as the occurrence of other mutations in the same sequence is possible [Nevertheless, the presence of one mutation on possible . In somepossible . Althougpossible ,41. One BCR-ABL1. These rare splicing mutations are associated with nucleotides insertion, namely the retention of 35 intronic nucleotides on exon 8 to 9 of ABL1 [The occurrence of mutations outside the kinase domain is less frequent but could impact on TKIs resistance. Mutations on the SH domain could affect the BCR-ABL1 conformation and consequently compromise the TKI efficacy ,43. In a of ABL1 ,45. Furt of ABL1 ,37,46.BCR-ABL1 expression due to gene amplification or upregulation at the transcriptional level is another resistance mechanism that is only observed in a small proportion of patients [BCR-ABL1 leads to resistance by increasing the oncoprotein concentration needed to be inhibited with TKI. Besides being more probably to occur, amplification is less frequent than point mutations [BCR-ABL1 transcription [BCR-ABL1 precedes the emergence of point mutations in the kinase domain [patients ,47. The utations , and in cription . Some aue domain .The DNA damage response (DDR) deregulation that leads to DNA damage and genomic instability has been implicated in the CML evolution, leading to TKI resistance and disease transition from CP-CML to more malignant stages ,49,50. TThe BCR-ABL1 oncoprotein is responsible for the genomic instability observed in CML since its activity generates reactive oxygen species (ROS), disrupts the DDR pathways activating error-prone DNA repair, induces replication stress and centrosomal dysfunction, and inhibits apoptosis resulting from different DNA damage-induced lesions ,55,56. AMBD4 and NTHL1 [RAD51L1, FANCA, and ERCC5 [Some studies performed in cell lines demonstrated the involvement of DDR in TKI resistance. The upregulation of the alternative NHEJ factors PARP1, WRN, and DNA ligase IIIa along with the downregulation of the canonical NHEJ proteins Artemis and DNA ligase IV reflect the role of the inefficient and error-prone alternative NHEJ pathway in TKI resistance ,70. Furtnd NTHL1 ,72. Simind ERCC5 ,74,75. Tnd ERCC5 .Treatment efficacy is highly dependent on the access of the drug to its molecular target. For targeting BCR-ABL1 (a non-receptor tyrosine kinase), it is critical for TKIs to reach the inside of CML cells at adequate pharmacological concentrations to achieve therapeutic clinical outcomes. The movement of drugs across cell membranes is largely mediated by drug transporters proteins . The balSLC22A1 gene, and higher mRNA levels were associated with major molecular responses [Most drugs have a low ability to diffuse freely across cell membranes, as their movement is not dependent on ATP and is mediated by solute carrier (SLC) transporters, such as OCT1 . OCT1 isesponses ,80. On tesponses ,82. Someesponses . Howeveresponses . Other tesponses ,85,86. Hesponses . Lower lesponses ,89.ABCB1 expression levels (the gene that encodes P-gp) are associated with poor long-term outcomes and advanced-phase disease [ABCG2 gene [ABCC6), which share some substrates with P-gp. The MRP6 may be especially relevant in resistance to second-generation TKI [Extrusion of metabolites, xenobiotics, and chemotherapeutic agents is mediated by ATP-binding cassette (ABC) transporters ,90. Over disease ,93,94. A disease . AnotherCG2 gene . This prCG2 gene ,98. As dCG2 gene ,100. Addtion TKI .ABCG2, rs2231142 results in loss of function mainly by altering protein folding, and a significant reduction of the transporter has been linked to the presence of the A allele [SLC22A1 has been previously associated with lower major molecular responses (MMR) and high risk of resistance due to low expression of OCT1 and consequent lower TKI uptake [BCR-ABL1 mutations [Genetic variants highly influence the function, expression, and localization of drug transporters. The presence of polymorphic variants in genes actively involved in TKI transport may influence its pharmacokinetics and, consequently, drug efficacy ,102. ForA allele ,104,105.A allele ,107,108.I uptake ,110. Thuutations .To overcome the inhibition of BCR-ABL1, CML cells may activate alternative signaling pathways to compensate the loss of BCR-ABL1 kinase activity. Consequently, cells will be able to proliferate and survive despite effective BCR-ABL1 inhibition. RAS/MAPK, SRC, JAK/STAT, and PI3K/AKT are some of the pathways that contribute to TKI resistance .+ cells increased the sensitivity to Imatinib [GAB2 is one member of the GAB family docking proteins that exerts a critical role in CML by amplifying BCR-ABL signaling . DysreguImatinib .Overexpression of the SRC family kinase protein, such as LYN and HCK, are related to CML resistance cases ,116. TheAfter activation of PI3K, AKT is subsequently phosphorylated, influencing multiples downstream proteins. BAD is one of AKT targets that reduces the apoptotic signal. After being phosphorylated, BAD becomes inactive and consequently did not inhibit anti-apoptotic proteins like BCL-2 and BCL-XL . AnotherT315I cells (a CML resistant cell line) as well as primary BC-CML cells obtained from TKI-resistant patients (with and without BCR-ABL1 kinase mutations) by decreasing the expression of CD44, MYC, p-CRKL, p-STAT5, and survivin [Furthermore, CML patients resistant to TKIs and Imatinib-resistant cell lines show higher survivin levels than the sensitive ones . Survivisurvivin .IKZF1, RUNX1, ASXL1, BCORL1, and IDH1/IDH2; (II) fusions: MLL, MSI2, and MECOM; (III) deletions: PAX5/CDKN2A, HBS1L-MYB, and del(17p); (IV) amplifications: chromosome 8, 19, and 17q [Additionally, all signaling pathways mentioned along with the BCR-ABL1 activity itself lead to an accumulation of ROS in CML cells ,131. Som and 17q ,135. The and 17q . HoweverThe impairment of multiple signaling pathways induced by BCR-ABL1 or by independent mechanisms results in a favorable cancer environment, contributing to poor TKI efficacy and consequently to resistance. The knowledge of this network supports the rationale to new therapeutic schemes with other inhibitors or in combination with TKIs.2 tension (hypoxia) that characterize the bone marrow microenvironment (BMM) niches stabilizes hypoxia-inducible factor 1α (HIF-1α) which is a crucial regulator of maintenance, survival, and proliferation of LSCs [The LSCs display high resistance to TKI showing heterogenous adaptations including a modified transcriptome, genome, and epigenome . Given t of LSCs .The BCR-ABL1-mediated activation of the nutrient-sensitive pathways leads to GSK3-β suppression along with the cytosolic retention and degradation of FOXOs, a number of pleiotropic transcription factors that (among others) are activators of autophagy . Upon trMetabolomics studies have shown that LSCs accumulate high levels of various dipeptides consisting of a range of amino acids . Given tILK (integrin-linked kinase) in LSCs of non-responding patients [Sirt1 gene downregulated OxPHOS-related mRNAs in LCSs and delayed CML onset and progression in a mouse CML model [Hif-1a knockout in a mouse model suggested that Hif-1a enhanced glycolysis and possibly contributes to CML stem-cell survival [Regarding regulation of the tricarboxylic acid (TCA) cycle and OxPHOS, a comparative gene expression analysis in CML stem-progenitor cells isolated from TKI-responding and non-responding patients revealed upregulation of patients . It was patients . The mulpatients . The uprpatients . SupportML model . MechaniML model . FurtherML model . MoreoveML model . Hif-1a survival .Lipids and fatty acid metabolism are also involved in LSCs survival, since arachidonate 5-lipoxygenase  was found to be upregulated in LSCs and modulate β-catenin levels in a BCR-ABL1-independent manner ,156. IntDNMT3A [TET2 [EZH2 [ASXL1 [There is now ample evidence that epigenetic dysregulation contributes to leukemic stem cell generation, maintenance, and progression in CML. Several studies have demonstrated that mutations in epigenetic regulating genes, such as DNMT3A , TET2 [13A [TET2 , EZH2 [1T2 [EZH2 , and ASX2 [ASXL1 , are rel2 [ASXL1  but the 2 [ASXL1 ,162.ABL-1 hypermethylation has been well documented, its role in the pathophysiology of disease progression is not clear. It is, however, evident that increased methylation of genes such as p15 [RASSF1A [TFAP2A [EBF2 [Epigenetic modifications consist in the addition or removal of small molecules, such as methyl or acetyl groups, onto DNA or DNA-related proteins, such as histones, resulting in the remodeling of chromatin and providing sites for the recruitment of other transcription factors . In addih as p15 , RASSF1A[RASSF1A , TFAP2A  [TFAP2A , and EBF2A [EBF2 , among o2A [EBF2 ,170. Thi2A [EBF2 .In parallel to DNA hypermethylation, malignant tissues also acquire histone modifications. The most frequent of these are acetylation, methylation, and phosphorylation. Their main effect is the modulation of chromatin condensation which subsequently alters expression of cell cycle, apoptotic, and tumor suppressor genes . SpecifiMYC in the case of miR-17 [CCN3 in the case of miR-130 [MYB in the case of miR-150 [Recent research has uncovered that, in addition to epigenetic regulation at a nuclear level, post-translational processes play a critical role in epigenetic regulation of protein synthesis. MicroRNAs are the main mediators of these processes, exerting their effect by blocking protein synthesis and promoting mRNA degradation . CML patf miR-17 , CCN3 in miR-130 , and MYB miR-150 . Some mi miR-150 , which hBMM is considered a safe haven for HSCs ,187 and + cells from patients with CML produce ten times more cytokines than their normal counterpart, highlighting the importance of these signaling molecules to this disease [BLC-2, BLC-XL, and MCL-1 [BCR-ABL-positive cells as a feedback loop [The bidirectional interaction between CML cells and the BMM niche is vital to support leukemic development and counteract the TKI effects ,190. Alt disease . TKI tre disease ,199. Thind MCL-1 . CXCL12 nd MCL-1 ,200. Depnd MCL-1 . Similarack loop . Furtherack loop ,199.The immune system is an essential player within the BMM, and expression of specific immune cells might dictate successful TKI responses ,204. At CML treatment optimization has been achieved by the implementation and standardization of molecular monitoring strategies such as real-time quantitative reverse transcription polymerase chain reaction (RT-qPCR). The different methodologies used for CML diagnosis and treatment monitoring in the clinic have undeniably improved the effectivity of patients’ management, improved the detection of the BCR-ABL1 KD mutations, and refined the selection of CML patients with higher probability to achieve treatment-free remission (TFR) after TKI cessation.BCR-ABL1 gene is, at the same time, both a therapeutic target and a robust and precise biomarker of minimal residual disease (MRD). Over the years, CML treatment has been optimized through the implementation and standardization of molecular monitoring. According to the 2020 European Leukemia-Net (ELN) recommendations, the molecular testing of CML should include different molecular methodologies [BCR-ABL1 fusion gene type. During treatment, the regular quantification of BCR-ABL1 transcripts by quantitative real-time PCR is performed to monitor transcript levels. After the achievement of a complete cytogenetic response, cytogenetic tests are warranted only in cases with ACA in the Ph+ clone [BCR-ABL1 transcripts is needed [BCR-ABL1 transcripts, a minimum of 4–5 years of TKI therapy, and a DMR of MR4 or better during more than 2 years [The dologies . At diagh+ clone . These rh+ clone . In TFR,s needed . The 202 2 years . The suc 2 years ,216. How 2 years ,217,218.BCR-ABL1 transcripts [BCR-ABL1 transcript, the need for a standard curve, and the sensitivity to inhibitors. However, several studies have tried to overcome these limitations by improving its performance [BCR-ABL1 transcripts [® qPCR BCR-ABL IS Kit , with a sensitivity of 0.002% IS (MR4.7) for CML patients expressing e13a2 or e14a2 fusion transcripts, the Xpert® BCR-ABL Ultra , which detects the most common BCR-ABL transcripts below MR4.5 or 0.0032% with a short turnaround time, and the digital PCR (dPCR) kit QXDx BCR-ABL %IS  [r = 0.996) in the range between 20% to 0.002% [4.5, 81.2% at MR4.7 (0.002% BCR-ABL1/ABL1 level), and 81% at MR5.0, with a low interlaboratory variation and high assay linearity [BCR-ABL1 levels and those potentially eligible for TKI discontinuation. However, to successfully implement dPCR in clinical settings, this methodology should be optimized and standardized to CML MRD monitoring.In clinical settings, some molecular tests have been robustly validated for detecting and monitoring nscripts . The “clformance . Three rnscripts . These tCA, USA) ,220,221.o 0.002% . These ro 0.002% . Moreoveinearity ,223,224.BCR-ABL1 point mutations that impair TKI binding [BCR-ABL1 KD mutation screening is Sanger sequencing, but this technique has relatively poor sensitivity (10–20%). The clonal configuration of BCR-ABL1 mutations is very important since compound mutations are extremely resistant to first, second, and even third-generation TKIs in some cases [BCR-ABL1 point mutations, next-generation sequencing (NGS) and dPCR are being investigated as alternative methodologies to Sanger sequencing. The NGS approach has better sensitivity (1%) than Sanger and can distinguish between compound and polyclonal BCR-ABL1 mutations when multiple substitutions fall on the same sequence reads. However, this methodology has a high error rate, particularly when sequencing mRNA, due to the use of the error-prone reverse transcriptase [BCR-ABL1 KD mutations resulted in its inclusion in position papers and the 2020 ELN recommendations [BCR-ABL1 KD mutations. Soverini and collaborators (2019) compared Sanger sequencing, NGS, and dPCR in CML patients with failure or warning responses to TKI therapy [Additionally, molecular testing is crucial in the setting of TKI therapy failure since these patients may have acquired  binding . The golme cases ,214,215.criptase . Recentlcriptase ,225. Thendations ,226,227. therapy . In this therapy .omics data from cancer patients and for analyzing gene expression profiles in response to different drugs [GPR34 and ADRBK2 genes (encoding two G protein-coupled receptor kinases) was correlated with a lower activity of Sunitinib .The use and application of machine learning methods for the diagnosis of common types and subtypes of leukemia has been very successful over the past decades, as reviewed by Sarah et al. . One of nt drugs . In thisnt drugs . Conversnt drugs , who ideA recent study that also used ML models was able to predict future diagnosis of CML based on the analysis of data from retrospective electronic health records . In partDuring the treatment of a patient with CML, changes in therapeutic protocol may be due to several reasons: resistance, intolerance, or suboptimal response rates (warning criteria), causing a mandatory TKI switch in case of resistance. The selection of the best second-line therapy needs to be adjusted to patients’ characteristics, comorbidities, and toxicity of first-line TKI, among other factors . DiffereAll TKIs approved for CML treatment are orally administrated and competitively inhibit BCR-ABL1 TK by binding at the ATP-binding site. Dasatinib, Nilotinib, Bosutinib, and Radotinib are second-generation TKIs, where Ponatinib is a third-generation TKI.According to ELN guidelines, in case of resistance, the BCR-ABL1 KD mutation profile needs to be investigated to guide selecting the second line of treatment. Each TKI presents a different sensitivity profile to the different mutations identified , where tBCR-ABL1 point mutations. Nilotinib is resistant to Y253H/F, E255K/V, and T315I [50 of 34 nM [BCR-ABL1 point mutations, namely Y235H, E255V, T315I, and T315M [The resistance pattern to Nilotinib is very similar to that observed for Imatinib regarding drug transporters, but this second-generation TKI presented a different resistance profile to nd T315I ,34,35,93of 34 nM ,239. Nevnd T315M ,241. In nd T315M ,34,35  inhibitor that has granted breakthrough therapy designation by FDA in 2021. This approval was based on ASCEMBL trial results and intend to adult Ph-positive CML in CP previously treated with two or more TKIs or patients harboring the T315I mutation. In opposition to the previously described TKI, Asciminib targets the myristoyl site of ABL1 kinase with an in vitro ICn trials ,243,244.n trials ,246. Duen trials ,247.New TKIs have been designed to overcome ABL1 gatekeeper mutations (mainly T315I) and at present are only preclinically validated inhibitors, such as Bafetinib, Rebastinib, Tozasertib, and Danusertib . Other nDespite the considerable success of BCR-ABL1 inhibitors in CML, even with second and third-generation TKI in the clinical setting, this therapeutic protocol is not a curative approach. The increased knowledge of CML biology, especially with recognition of dormant LSCs, highlights the necessity to explore non-BCR-ABL1 targets ,253. OveAs mentioned before, a critical protein in the transduction of BCR-ABL1 oncogenic signal is GRB2, which is able to activate RAS/MAPK, JAK/STAT, and other signaling pathways resulting in cell proliferation stimuli . BP1001 The RAS/MEK/ERK pathway can be activated independently of fusion oncoprotein in response to growth factors . An initRuxolitinib, a JAK2 inhibitor currently approved for myelofibrosis and polycythemia vera, demonstrated promising results in combination with TKI in reducing LSC viability in CML . Its mecThe recognition of the PI3K/AKT/mTOR pathway as an important drug target in oncology led to the approval of PI3K inhibitors  and mTOR inhibitors (as an example of Sirolimus) for different neoplasias . In precBCL-2 family members became druggable targets in hematopoietic cancers to overcome the anti-apoptotic signal in tumor cells . VenetocIn cancer cells, the repression of tumor suppressor proteins, like P53, leads to uncontrolled proliferation, resistance to apoptosis, and survival. The degradation of P53 is mediated by MDM2, an E3 ligase that targets the protein to the proteasome. Inhibition of MDM2 may constitute a critical approach to restore P53 function and control tumor cell fate indirectly . DiffereTargeting signaling pathways essential to maintain and regulate key features of LSCs is a very promising strategy in different cancers . InhibitMany other molecules and pathway inhibitors were or are currently in preclinical studies trying to overcome resistance and may constitute future therapeutic options in CML. The exploitation of other signaling pathways either alone or in combination with BCR-ABL1 drugs will, sooner or later, become a reality in CML treatment to improve response, avoid resistance, and enhance treatment discontinuation probability.The fact that epigenetic modifications can be manipulated pharmacologically has led to their successful use in clinical practice in both myeloid and lymphoid malignancies.The two most commonly used DNA hypomethylating agents are Azacitidine and Decitabine and both have been particularly successful in the treatment of MDS ,285,286 Histone modifications can also be manipulated pharmacologically, especially with HDAC inhibitors. These include aliphatic acids (phenylbutyrate), cyclic peptides (romidepsin), benzamides (entinostat), electrophilic ketones, and hydroxamates (vorinostat) and may restore normal acetylation of histone proteins and transcription factors . Some HDDuring the past decade, considerable progress has been made in the immunology understanding of CML, raising hopes that this disease may be curable by improving the currently targeted chemotherapy with immunotherapeutic approaches . Immune Several cooperative groups and trials showed significantly higher complete cytogenetic remissions and major molecular remission rates for patients treated with Imatinib in combination with IFN-α as compared with patients treated with Imatinib alone . However, the interferon discontinuation rate in all studies was high (83%), mainly due to toxicity. None of those studies mentioned above found a statistically significant reduction in progression to advanced disease phase or prevention of CML-related death .In CML patients, particularly in those classified as high risk by the Sokal score, the expression of the immune checkpoint proteins PD-L1 and PD-1 had been observed. Thus, the upregulation of PD-L1 is considered an immunological escape mechanism for CML cells ,249. TheMore advanced therapeutic strategies, e.g., vaccines or engineered T cells, are in study to treat CML patients aiming to induce an immune response against the leukemic cells. In general, tumor-associated HLA-presented peptides on malignant cells are relevant targets. However, the role of these neoantigens in cancers with a low mutational burden, such as CML, remains unclear. Several small studies have investigated the potential to induce an anti-leukemic vaccination response in CML patients. One strategy uses ex-vivo generated autologous dendritic cells and other leukemia-associated antigens as the injection of BCR-ABL1 derived peptides (NCT02543749). This strategy has shown clinical efficacy by inducing a T-cell response in most patients and was safe and feasible. However, these studies are single arm and, therefore, prospective randomized trials are needed .RUNX1 mutations in BC-CML patient, where the anti-CD19 CAR-T showed an additive effect when combined with Imatinib. These studies suggest that CART-cells may confer therapeutic benefit, particularly in young or advanced-phase CML patients who are resistant/intolerant to TKI, highlighting the critical importance of the immune system in optimizing treatment responses in CML [Chimeric antigen receptor (CAR)-T cells are an emerging immunotherapy already approved for B-cell malignancies and are being evaluated for myeloid malignancies. The IL1 receptor accessory protein (IL1RAP), a co-receptor for the IL1 and IL33 receptors, is a cell-surface marker expressed by CML cells but not by normal HSCs. In vitro and in vivo studies suggest that CART-cells targeting IL1RAP specifically induce cell death of quiescent CML stem cells and have a favorable side effect profile without off-target toxicity or tumor lysis syndrome, some of the adverse events more commonly associated with CAR-T therapy. A combined CAR-T and TKI approach has also been used to eliminate CML stem cells in limited clinical series. An anti-CD19 CAR-T therapy combined with Dasatinib induced complete molecular remission and the return to CP in CML patients in lymphoid BC harboring T315I mutation. This CART cleared the T315I mutation and re-sensitized cells to Dasatinib. The same effect was observed against s in CML .In TKI pre-era allogenic stem cell transplantation (Allo-SCT) represented a therapeutic option for patients at CP-CML with a compatible donor and fit for the procedure . HoweverThe appropriated time for perform Allo-SCT has not established yet, and the decision must be based on patient’s individual benefit–risk assessment. The multiple transplant-associated risks, such as non-relapse mortality and graft versus host disease, and the presence of high-risk ACA are some of the factors to be considered in patient evaluation . In caseIn CML treatment, the introduction of TKIs had been considered a revolutionary and successful therapeutic, leading to a normal life expectancy. However, a substantial proportion of CML patients fail treatment with drug interruption and discontinuation, rapidly leading to disease resurgence due to minimal residual disease re-emergence  and probIn this context, adherence to therapy and compliance with clinical instructions are critical in the management of CML. Adherence to long-term therapy is defined by the World Health Organization (WHO) as “the extent to which patients’ behavior taking medication corresponds with agreed recommendations from a healthcare provider” . The adhSeveral studies have shown that lack of adherence to Imatinib is frequent and may significantly impact patient outcomes. A Belgian study found that only 14% of CML patients took all of their prescribed Imatinib. This lack of compliance led to increased suboptimal responses . Others The factors that seem to facilitate adherence are fitting the Imatinib into the daily routine, using prompts to remember to take the tablets and finding ways of coping with side effects. Another intervention that may help reduce intentional non-adherence is making a patient phone call and discussing how they are getting on with their medication. The interventions to improve adherence need to consider both intentional and unintentional reasons for not taking Imatinib and should target the specific personal reasons for not correctly receiving TKI treatment .Chronic myeloid leukemia was a pioneer in terms of targeted treatment approaches . The devBCR-ABL1 independent mechanisms seems to be important to understand and identify the role of other proteins in treatment response [The molecular mechanisms behind TKI resistance are multiple, ranging from changes in the molecular drug target itself, to mechanisms that alter drug concentration or modify leukemic cells signaling network . In addiresponse . Identifresponse . Additioresponse .In this context, blood stem cell transplant therapy is the only proven cure for these patients, but this therapy has higher toxicity and is limited by donor availability ,312. ThiThe emerging landscape of immune dysfunction and immunosurveillance in CML highlights the critical importance of the immune system in optimizing treatment responses in CML. Curiously, IFN-α, a previous standard of care therapy, exhibits non-specific, untargeted effects on the immune system, leading to repurpose it in order to enhance TFR in the TKI era. However, newer precision oncology approaches targeting immune system, such as vaccines, ICIs, and CAR-T-cell therapies constitute a great promise in CML therapy, especially in cases of TKI resistance and/or intolerance. Together, specific and non-specific immunological effectors can make the concept of "operational cure" a reality for the vast majority of patients in the next decade .


This study aims to investigate the admission criteria used by physician assistant postgraduate education programs in selecting licensed PA applicants for postgraduate training in the United States. To our knowledge, there have been no previously published reports on selection criteria and/or other factors influencing postgraduate PA admission decisions.A non-experimental, descriptive research study was designed to obtain information from members of the Association of Postgraduate Physician Assistant Programs (APPAP).extremely/very important included board certification/eligibility, letters of recommendation, advanced degree, and personal essay. Survey data suggest that publications, undergraduate transcripts, and class rankings are not considered to be of high importance in applicant selection.Twenty-three out of 73 postgraduate programs (31.5%) responded to the survey. The study reported that applicant PAs and nurse practitioners (NPs) are largely selected on the basis of several factors. The most heavily weighted factor is the interview itself; other selection criteria perceived to be The number of PA applicants applying to each postgraduate training program averages around 26 and total number of enrollees is about 3.6 per program. Additionally, some programs reported furloughing of trainees  during the pandemic, whereas the vast majority of postgraduate PA programs remained operational and some even experienced an increase in application volume. The total cost of training a PA resident or fellow in postgraduate programs is currently $93,000 whereas the average cost of training a categorical physician resident is estimated at $150,000 per year when considering both salary and benefits.This novel study examined criteria and other factors used by postgraduate PA programs in selecting candidates for admission. Results can be used by postgraduate programs to improve or modify current selection criteria to enhance the quality of trainee selection. Further research is needed to examine correlations between applicant attributes, selection criteria, and trainee success in completing postgraduate training.The online version contains supplementary material available at 10.1186/s12909-021-03059-y. The development of postgraduate programs for physician assistants (PAs) began over four decades ago , 2. CurrWhen it comes to PA postgraduate selection admission criteria, there is no published data on which factors used by postgraduate programs most affect the admission decision. Therefore, the purpose of this study is to determine the selective admission criteria that are important to postgraduate programs in selecting applicants for medical or surgical specialty training.A non-experimental, descriptive research study was designed to obtain information from members of the Association of Postgraduate Physician Assistant Programs (APPAP). The list of programs was drawn from an updated APPAP membership page (APPAP.org). After review of the literature and based on the substantive experience of the authors in postgraduate PA education, an online survey was developed. The survey was comprised of 22 items and after a successful pilot, the survey was finalized. The three respondents who took part in the pilot test did not participate in the overall survey to limit bias. Individual postgraduate programs were sent an email invitation and a link to a voluntary, anonymous online survey. The email introduction to the survey contained all the necessary elements of written consent, and submission of the survey indicated the respondents’ consent to participate. The study period was from 8/17/2021 through 9/17/2021. Five reminders were sent to postgraduate programs during the 30-day study period. The participants completed the survey through the online tool SurveyMonkey. Confidentiality was maintained throughout the study and no identifying information was recorded. Survey responses were aggregated, and descriptive statistical analyses were utilized through a statistical package embedded within the survey software. The survey was reviewed with the University of California Irvine (UCI) IRB team and the study is exempt under the UCI Exempt Self-Determination Tool. As part of using the Exempt Self-Determination Tool, Lead Researchers and Faculty Sponsors (as applicable) provided their assurance that they followed relevant Human Research Protection Program (HRPP) policies and procedures, among other criteria.Seventy-three postgraduate programs were invited to participate, and 23 programs completed the entire survey. The overall response rate was 31.5%. Response rates varied by specialty: emergency medicine (34%), general surgery (13%), critical care medicine (8%), orthopaedic surgery (8%), neonatology (8%), cardiothoracic surgery (8%), psychiatry (4%), obstetrics and gynecology (4%), internal medicine (4%), and trauma and surgical critical care (4%). Of those that completed the survey, 87% were program directors, 9% associate program directors, and 4% were NPs affiliated with the program. Additionally, (34%) of respondents enroll both PAs and NPs, whereas (65%) only enroll PAs.extremely important by postgraduate programs included personal interview (96%), board certification/eligibility (83%), letters of recommendation (61%), graduate degree (52%), and personal essay (44%). Perceived as very important: clinical rotation grades (44%), awards/achievements (44%), transcripts from PA or NP entry level program (44%), and overall GPA from PA or NP school (39%). Perceived as important: community service and class ranking (48%). Perceived as somewhat/not important: publications (78%), membership/position in a local PA or NP association (52%), and transcripts from undergraduate study (52%)  or rolling admission process (17%); however, application and selection deadlines vary among postgraduate programs as there is no centralized application processing service available to PA and NP postgraduate applicants. A majority of respondents do not charge an application fee (60%). Respondents reported that participants involved in the screening process include the program director (91%), associate program director (47%), medical director (26%), and human resources business partner (13%).The program personnel involved in interviewing applicants comprises a diverse interview panel and includes the program director (100%), medical director (69%), associate program director (52%), staff PAs/NPs (52%), physicians (34%), and to a lesser extent, resident physicians and human resource recruiters (17%). Moreover, postgraduate program interviews come in many different forms such as structured interview (60%), panel interview (39%), and multiple mini interview (34%). During the interview, most programs included a facility tour (95%) and introduction to program staff (73%), while other programs reported that candidates participated in a group activity (39%). Nearly (20%) evaluated candidates with a pre-admission assessment.The data from our study suggest that postgraduate programs include multiple stakeholders in selecting candidates for admission. Respondents reported program personnel responsible for making the final admission decision included the program director 100%), medical director (73%), associate program director (43%), and staff PAs/NPs (21%)  of postgraduate programs were operational during the evolving COVID-19 pandemic, however, 22% of programs reported that their trainees experienced furlough; the length of the furlough was not assessed in our study. Approximately 4% of programs experienced closure. Additionally, 70% of programs reported an increase in the number of applicants without a corresponding increase in enrollees. Lastly, 26% of postgraduate programs experienced a decrease in program funding.The goal of this research study was to investigate admission criteria used by PA postgraduate programs in selecting licensed PA applicants for postgraduate training in the United States. There are no published data detailing factors used by postgraduate programs to drive admission decisions.This study revealed the influence and weight of particular selection criteria as well as the stakeholders evaluating that criteria. Criteria driving the selection process include personal interview, board certification/eligibility, letters of recommendation, achievement of a graduate degree, and personal essay. Least important to key stakeholders, namely program directors, but also medical directors, associate program directors, and staff PAs/NPs was participation in research/publication, membership in local PA/NP associations, and undergraduate transcripts.Aside from board certification/eligibility and achievement of a graduate degree, the factors deemed most important of the selection criteria for admission to a postgraduate PA program are subjective in nature. The weight placed on the personal interview, personal essay, and letters of recommendation supersedes that of graduate GPA, class rank, and clinical rotation grades, suggesting that applicant characteristics of personability, interpersonal skills, communication skills, emotional intelligence, and adaptability are as important to demonstrate as clinical acumen via board certification. This is not too dissimilar from how physician residency programs evaluate certain characteristics in the selection of candidates for postgraduate training . ImportaAccording to published research, 36% of physician residency programs cited “demonstrated involvement and interest in research” as a factor when granting applicants an interview ; howeverBarriers to research contribution have previously been considered and include lack of experience with research, lack of time, lack of interest, lack of mentors and lack of funding ; data frAlso, of little importance to postgraduate admissions was membership in a PA/NP association. Encouraging participation in local and national organizations should be a priority at both the graduate and postgraduate level to increase advocacy of the PA profession from career inception. Preparing future advocates to advance the profession will serve to fortify the PA presence as key stakeholders in health care advancement at local, national, and international levels.Additionally, this study identified program personnel involved in the admission process, from screening applications to conducting interviews and making applicant selections, indicating that the program director primarily, along with associate program directors and medical directors had the greatest influence on the interview process and selection of candidates for admission. It should be noted that the graduate medical education (GME) office was not found to be involved in the screening, interviewing, or selection of candidates for postgraduate PA training. This distinction suggests that, while housed in the same institution, PA postgraduate education processes and GME education processes are two separate entities with different funding, goals, and resources.Throughout the COVID-19 pandemic, the vast majority of postgraduate PA programs remained operational, while many saw an increase in application volume. The uptick in applications may have been attributed to a perceived skills gap among recent PA and NP entry level graduates who experienced limited face-to-face clinical education  during their training or encountered difficulty in securing employment subsequent to graduation as a consequence of the COVID-19 pandemic. Additionally, a quarter of postgraduate programs experienced a decrease in program funding. This may be largely due to financial losses exacerbated by COVID-19 in which US hospitals now face a liquidity crisis; some institutions may have elected to cut back on funding for training programs that they deem a drain on already limited resources [Lastly, the estimated cost  of training PA and NP postgraduate fellows is around $93,000. This is the first study to provide a reasonable estimate of typical costs associated with PA and NP postgraduate training. Awareness of the costs of training will help shed light on the financial contribution of the sponsoring institution to develop or sustain new and existing postgraduate PA programs.Our study is not without limitations. First, this is survey data and only members of APPAP were surveyed. Post-graduate programs are heterogeneous in nature so this sample may not represent the wide diversity of programs. Second, we did not evaluate the impact of racial, geographical, or gender differences among applicants during program selection. Third, this study includes a low response rate of (31.5%), which may have contributed to response bias. Fourth, overrepresentation of emergency medicine programs may also have led to some bias and the following specialties in which PA postgraduate training is available but underrepresented in this survey includes transplant surgery, cardiology, geriatrics, hospice and palliative care, structural heart, hepatobiliary surgery, plastic surgery, vascular surgery, urology, urgent care, pediatric medical specialties, pulmonology, primary care including rural medicine. Lastly, when programs were asked to report on the total cost  of training a PA resident or fellow, five programs reported only salary costs. Therefore, we multiplied the salary cost by the 32% benefit rate (Bureau of Labor Statistics) as a percentage of salary (2018) to determine total postgraduate cost of training of a PA and NP in those 5 programs. . DespiteThis study presents the admissions practices of PA postgraduate education programs in the US. The data suggest that these postgraduate programs use a multifactorial application process in accepting candidates for specialty training. The data from this study can help guide and improve selection practices across postgraduate education programs as well as inform future applicant and policy decisions.Future research is needed to examine correlations between applicant attributes, selection criteria, and trainee success in completing postgraduate training. Another area of exploration is whether postgraduate education programs closely align with employment and/or academic institutional priorities. Future investigation of postgraduate training should also include evaluation of other fixed costs not highlighted in this study, such as program personnel and technology costs, faculty development expenses, and accreditation fees. Lastly, research is needed to explore the main factors  influencing the choice of graduate PAs in pursuing a specialty postgraduate training program.Additional file 1.Additional file 2.


Modern public transit systems are often run with automated fare collection (AFC) systems in combination with smart cards. These systems passively collect massive amounts of detailed spatio-temporal trip data, thus opening up new possibilities for public transit planning and management as well as providing new insights for urban planners. We use smart card trip data from Taipei, Taiwan, to perform an in-depth analysis of spatio-temporal station-to-station metro trip patterns for a whole week divided into several time slices. Based on simple linear regression and line graphs, days of the week and times of the day with similar temporal passenger flow patterns are identified. We visualize magnitudes of passenger flow based on actual geography. By comparing flows for January to March 2019 and for January to March 2020, we look at changes in metro trips under the impact of the coronavirus pandemic (COVID-19) that caused a state of emergency around the globe in 2020. Our results show that metro usage under the impact of COVID-19 has not declined uniformly, but instead is both spatially and temporally highly heterogeneous. Metro systems are among the most important transportation systems around the world. In big cities thousands or even millions of trips are conducted by metro every day. Compared to other public transit modes, metro systems are characterized by higher speeds and larger capacities, thus attracting high passenger demand . To the best of our knowledge, this study is the first to comprehensively map spatio-temporal metro trip patterns and spatio-temporal changes in metro usage at this level of detail, and among the first to do so under the impact of COVID-19.COVID-19 has caused a state of emergency around the globe. At the end of December 2019 the World Health Organization picked up information about a ‘viral pneumonia of unknown cause’ in China  links are not investigated  interactions  and Taoyuan Airport MRT  are not considered. Taipei Metro, which has been operating since 1996 and has since been gradually extended, encompasses an operating network length of 146.2 km with 6 lines  by the Department of Rapid Transit Systems, Taipei City Government  which coBoth the origin-destination passenger count matrices and the metro stations data were translated from Chinese into English, following the English names in the official Taipei Metro route map. All data used in this study is freely available as open data.1A subset of the origin-destination passenger count data was created by first removing the weeks before the first COVID-19 case was confirmed in Taiwan on January 21st 2020 , walking or private bicycle, or Taiwan’s popular bikesharing system YouBike. But this seems to rarely have been the case, as the drops in counts are minor, thus indicating that people either lacked other feasible transportation options, or that they simply accepted the risk.2 of the simple linear regression as shown in Fig. 2 ≥ 0.96). The weekend stands out as distinctly different (R2 ≤ 0.48). Saturday and Sunday can predict each other very well (R2 ≥ 0.93), although not quite as well as the weekdays. The observations are true for all cases, compared within the year 2019, within the year 2020 and between the years 2019 and 2020.Another indicator of the similarity of temporal passenger flows between days is the RThe results indicate that Monday to Friday could be grouped together, as Fridays differ only to a minor degree from other weekdays. However, we analyze Saturday and Sunday separately, as they have distinct characteristics  with an exceptional decrease in passengers in comparison to other areas. On weekdays at noon, in the evening and at night Xinyi CBD, Ximending CBD and R15 Jiantan station form a more or less pronounced triangle of metro trips with especially strong declines in passengers. One reason for R03 Taipei 101/World Trade Center station and BL23/BR24 Taipei Nangang Exhibition Center station experiencing such strong declines in passengers is probably that there are large exhibition centers near these stations where big fairs take place. Many fairs in the period analyzed were cancelled due to COVID-19. Furthermore, R28 Tamsui station and BR01 Taipei Zoo station often stand out: they are the last stations on their respective lines, thus relatively remote, but with high declines in passenger volumes. Despite the fact that patterns on weekends are highly diffuse, Xinyi CBD stands out in almost all time slices with an extremely high decline in passengers. In some time slices on weekends, Ximending CBD can also be identified as having many links that lost many more passengers than other areas. The overall patterns of Saturday and Sunday are similar, although there are differences in details.In a temporal perspective, it is important to re-iterate a key result from Sect. This study presented a detailed network-wide spatio-temporal analysis of Taipei Metro trip patterns using passively collected AFC smart card trip data. This included metro usage in normal conditions (January–March 2019) and changes in metro usage under the impact of COVID-19 (January–March 2020). Based on simple linear regression and line graphs, days of the week and times of the day with similar temporal passenger flow patterns were identified. Then, we created radial flow maps to visually examine station-to-station mobility patterns based on weekdays and daytimes. The results provide deep insights into the spatio-temporal characteristics of metro usage, which help public transit operators, transport planners and urban planners to better understand urban mobility as well as the impacts of COVID-19 on metro usage.To the best of our knowledge, this study is the first to comprehensively map network-wide spatio-temporal metro usage at station-to-station level for a whole week divided into several time slices, while taking actual geography into account and clearly indicating magnitudes of passenger flow. Furthermore, this study is the first to comprehensively investigate spatio-temporal changes in metro usage under the impact of COVID-19 at the same level of detail. We showed that metro usage under normal conditions and the decline in metro usage under the impact of COVID-19 are both spatially and temporally highly heterogeneous. The applied mapping approach allows the direct visualization and analysis of these various heterogenous patterns.Despite Taiwan's success in dealing with the COVID-19 pandemic, metro usage decreased strongly and seemed to stay at a lower level even after COVID-19 was fully controlled. This development poses a serious threat for public transit operators due to financial losses, and represents a risk for the transition to more sustainable and public transit-based cities. However, the data used do not permit to investigate whether the decline in passengers happened because people are going out less often, or because people changed to other modes of transportation—and if so, to which modes. If the former is the case then it is likely that public transit usage may recover and return to the pre-COVID-19 levels. Less traffic in general is not bad news for urban planners. Mode switches, on the other hand, especially to individual motorized modes, may have more lasting effects. This would be the ‘bad case’ scenario for public transit operators, urban planners and the environment, which could pose many challenges in the future. From the existing literature one may take that both factors are likely to be at play. On the one hand, working from home has become a 'new normal' to many in various countries, and this has reduced commuting (Follmer et al. Since Taiwan has so far fully controlled the COVID-19 outbreak, public transit usage should continue to be closely monitored. Recent data for 2021 are available now and can be explored in future research. What happens in Taiwan—or other places that are in a similar situation—could indicate what public transit in cities around the world might face in the future after having more or less controlled COVID-19. The most important lesson to be learnt from COVID-19 is that such an outbreak can happen any time, and there is need to be prepared in the future. For public transport this may include to improve hygienic conditions in vehicles and at stations, more physical distancing between customers and, thus, more capacities, but also to develop new business models that may be needed if increased working from home relaxes the commitment to public transit in terms of owning season tickets. In addition, further studies should be carried out to investigate changes in mode choice and the level of mobility, and whether these changes are likely to be short-, medium-or long term.


International evidence supports the effect of intensive family preservation and reunification services in preventing children’s placement in out-of-home care (OOHC). Evidence within Australia is scarce. This protocol paper describes a hybrid effectiveness-implementation evaluation of the Victorian Family Preservation and Reunification (FPR) Response implemented by MacKillop Family Services. Participants include families engaged in the program and staff involved in program delivery. A pre-post study design will be used to assess the effectiveness of the FPR in improving family outcomes from intake to closure, including: (i) parenting knowledge, skills, and capability; (ii) family safety and home environment; (iii) child development, adolescent behaviour, education attendance and attachment; (iv) connection to services; and (v) prevention of children from entering or re-entering OOHC. Interviews and focus groups will be conducted with staff to evaluate the program’s fidelity, reach, feasibility, acceptability, and enablers and barriers to implementation. Quantitative data will be analysed using descriptive statistics and a series of paired-samples t-tests and F tests to examine changes in outcomes over time; thematic analysis will be used for qualitative data. If the FPR can yield significant improvements in families’ outcomes, this would provide strong support for its scale-up across Australia, to better support vulnerable families. Intensive family preservation and reunification services are designed to keep children safe in the care of their parents and prevent child removal or subsequent placement into out-of-home care . They aThe Children Youth and Families Act 2005 gives a legislative basis for a system of services targeting vulnerable families . In Victhow the program is being delivered), significant brokerage  and a new role within child protection called a Navigator. The Navigator is a government position that has been established within child protection to ensure that coordinated, targeted, appropriate, and culturally safe referrals are made into the FPR program. Rather than being prescriptive in program design, the Victorian government allowed all community service organisations to propose their own version of the FPR Response (within some broad parameters), with the view that consistency will be embedded through the government’s delivery of training and coaching to common practice elements  [In June 2020, a tender was put out requesting submissions from suitable community service organisations across Victoria who deliver family services to vulnerable families. Seventeen tenders were released to deliver the program across sites in both urban and regional settings. The goal of the Response was stated as being “strong families-with children who are safe, healthy, resilient and thriving; and parents who are supported to create a safe and nurturing home environment” . The proelivery) . MacKillMacKillop Family Services is a community service organisation that is committed to providing early intervention and family support services to disadvantaged families. Specifically, MacKillop strives to empower communities and families so that children can enjoy their childhood in a safe and loving home, can be healthy, thrive, and can develop to their potential. Key areas of intervention include operating foster care, residential care homes, homelessness support services, a suite of family service programs, specialist education services, and disability support and coordination services. In doing so, Mackillop supports children and young people to return to family, transition to independent living, heal from trauma, and to reconnect with learning and education. The FPR program aligns with MacKillop’s aim to provide early intervention for our most disadvantaged and vulnerable children, young people, and families. It addresses a key gap in the community—the lack of intensive family preservation and reunification services for high-risk and complex families. Therefore, the FPR program presents a significant opportunity to permanently change the trajectory of vulnerable youth out of the Child Protection system and OOHC, and capacity to breakdown the intergenerational cycles of maltreatment, OOHC placement, and disadvantage . This prThe MacKillop FPR Response has eight core components in addition to the government’s key program guidelines. These include: (1) partnerships with other agencies to build expertise, local knowledge and networks; (2) embedding an independent evaluator; (3) utilising existing programs to incorporate into the Response; (4) extensive onboarding and training for staff; (5) being data-driven through assessment use to drive best practice; (6) a communities of practice for practitioners and team leaders; (7) organisational governance meetings; (8) suitable human resources to deliver the program and additional resources that enable program delivery with fidelity . More information about these components can be found in Families enter the program via the CP navigator. The navigator supports child protection practitioners to identify, engage, and connect the most appropriate families into the FPR program. Known as a ‘connection’ , the child protection practitioner, the Navigator, and MacKillop team leader discuss the family members, their circumstances, and past-history, prior to MacKillop accepting the family into the program. In alignment with collaborative practice and the two mandated Information sharing schemes (Child Information Sharing Scheme and Family Violence Information Sharing Schemes) ,19, CP sThe MacKillop key worker/practitioner engages with the family intensively, defined as a home visit for at least one hour, three times a week, or less visits and regular phone contact plus other activities as required. The timing of these visits is flexible to accommodate the needs of families. Funding is provided to allow for 200 hours of work with an additional 40 hours of step-down towards closure. It is expected that families will engage with the program for approximately six months, but this could last for up to a year. The most practical caseload per full time practitioner is four families, however workers may hold up to six families, albeit at a decreased intensity and with less fidelity to the model. In general, workers focus on reducing stressors and barriers to health and family functioning, such as housing, and offer referrals to specialist services if needed. Workers also focus on the interactions within the family and strengthen core life skills.For families who are subject to an unborn report (reported to CP while pregnant) or with children aged 0–9, workers use evidence informed methods that focus on parenting skills , parent–child attachment, and capacity to effectively respond during a crisis, such as those from the newborn behavioural observation method  or circlThe program is also designed to support families to link into the community and its services to build independence, self-sufficiency, and a social network. Multi-disciplinary teams forged through partnerships with other agencies enhances the direct engagement of families with experts in parenting, drug and alcohol services, as well as regional care. Once families are better placed to meet the challenges they face and preferably with achieved goals, MacKillop will close the connection, but not before they provide the families with referrals to other programs or services if needed. For some families with young children, and where agreed to by the CP Navigator, an additional episode of care (240 h) will be provided to ensure that the family’s capacity is sufficient to keep children safe and thrive.Families in metropolitan areas with young people aged 10–17 years engage with the evidence based Multi-Systemic Therapy Psychiatric (MST-Psych) program ,23 whereDue to the lack of Australian evidence regarding the impact of intensive family preservation and reunification programs, a comprehensive mixed-methods evaluation of the MacKillop Family Services FPR program is being undertaken. In order to determine the effectiveness of the program, outcomes including parental knowledge, parenting skills, and capability; child development, adolescent behaviour and education attendance; connection to and use of family services; and protection from child abuse and neglect, were chosen in consultation with key stakeholders. These outcomes are associated with healthy, resilient and thriving children, and the work of family preservation and reunification programs ,5. FamilThe overall aim of this paper is to outline MacKillop Family Services’ delivery of the FPR across nine sites and provide the research protocol for the three-year study period (October 2020–October 2023). This evaluation focuses on both parent/caregiver (hereafter parents) and child outcomes in relation to the effectiveness of FPR, and also seeks to evaluate the barriers and enablers, feasibility, fidelity and reach of the FPR program during the delivery and implementation phase.To use reliable and valid measures to assess the effectiveness of the FPR through changes in the following parent and child outcomes: (i) parenting knowledge, skills, and capability over time; (ii) family safety and home environment; (iii) child development, adolescent behaviour, education attendance and parent/child attachment; (iv) connection of parents to services; and (v) prevention of children from entering or re-entering the OOHC system over a long-term period (12 months).To examine the implementation of the FPR by investigating the following outcomes: (i) the acceptability, appropriateness and feasibility of the program from the perspective of parents and staff; (ii) determine if the model can be implemented with fidelity ; (iii) evaluate the readiness of families to engage with the program, including reach; and (iv) examine the process, barriers and enablers of program delivery.Research objectives:In addition to these research objectives, parental history of adverse childhood experiences and OOHC participation are being examined as risk factors contributing to current child abuse and neglect in response to recent gaps in the literature .This paper was written using the Standard Protocol Items: Recommendations for Intervention Trials (SPRIRT) framework (2013) for writing protocol papers see . The proApproval was obtained from the Monash University Human Research Ethics Committee to undertake the research study. MacKillop Family Services also reviewed and approved this research.This study aims to investigate the effectiveness and implementation of the FPR concurrently. To do so, this study adopts a mixed-methods study design involving a longitudinal single group pre-post study design combined with qualitative components. Given that participants are at high risk of child removal, it is not possible to include a control group, and a waitlist or similar group are also not viable. Other alternative study types are also not suited  who are vulnerable to removal from the home due to a child protection assessment of risk, have children already in OOHC, or are pregnant and have had ‘unborn’ reports to child protection. These parents may have multiple factors that are contributing to their risk profile including recent family violence, drug or alcohol misuse, fragile mental health, court orders and justice involvement among others. Parents can vary in age and may be adolescent parents or those who are currently living in a care arrangement . Sample size calculations have been conducted using G*Power based on the NCFAS as this measure goes across most efficacy outcomes. For an analysis of variance test with the alpha set at 0.005, correlations between repeated measures set at 0.50 and power of 0.95, a total sample of 98 families will be required. To allow for a 25% rate of non-completion, at least 123 families will be recruited. During the study period, it is expected that up to 1100 families will engage in MacKillop’s FPR program. Based on similar research, we expect a high number of families to consent to participate in the study [MacKillop Family Services Staff: MacKillop directors, managers, team leaders and practitioners involved in the delivery of the FPR program are eligible to participate in interviews or focus groups about their experiences of implementing the program. Only staff who consent will take part in the research evaluation.Parents: All parents participating in the program are invited to participate in the current study through their key worker, however refusal to be in the study does not preclude them from receiving the FPR program. Due to the complexity of families and the need to have a trusting relationship established with the key worker, consent is likely to be obtained between the second and fourth months of program participation based on worker discretion. A flyer, written at a grade 5 literacy level and covering the most salient points of the study, is provided to all parents as they consider participating; declining participation has no effect on service delivery. After reading the flyer, an explanatory statement and consent form is provided (available on request), that requests approval for the receipt of the parent’s assessments and limited non-identifiable demographic data to be shared with the evaluation research team. For parents with children who are not adults, their legal guardian provides consent and the parent also needs to give assent. A separate explanatory statement and consent form have been developed for any parents who are interested in being interviewed during or post program participation.n = 50 at minimum) as focus groups/interviews are being conducted during scheduled team meetings. Nonetheless, ongoing recruitment of staff will occur concurrently with data analysis so that saturation of themes can be assessed.MacKillop Family Services Staff: An explanatory statement and consent form is provided to all staff prior to participating in research activities like a focus group or interview. Choosing not to participate does not impact their current or future employment with MacKillop Family Services or their relationship with Monash University. Each of the nine teams comprises a team leader and approximately five key workers/practitioners. It is expected that the majority of staff will participate  :a = 0.79–0.91) when implemented with families involved with Child Protection [The NCFAS-GR assesses change over time on ten family functioning domains that cover topics like safety, wellbeing, environment, and social life. The scale is completed by the FPR worker who accumulates knowledge about the family and then completes it in one sitting. Domain scores are measured using a six-point Likert scale ranging from “Serious Problem/Weakness = −3” to “Clear Strength = 2”. There is the option to provide a short comment to the score. The scores strongly guide the planning and practice, while also informing the child and family action plans and family goals. The NCFAS-GR is completed at the start of the program, after approximately 3 months or the middle, and upon closure. The NCFAS-GR has high internal consistency (otection .As per Parent Efficacy and Empowerment Measure (PEEM) :a = 0.92) [The PEEM is comprised of 20 positively framed statements about parenting that are rated by the parent on a scale from one to ten. The measure examines a parent’s feelings of control and confidence in parenting, especially when challenges present. Therefore, it is being used to determine the effectiveness of the FPR in improving parental confidence. The measure has been developed with an understanding of social work practice and information is available to support workers to use it appropriately, and it has high internal consistency ( = 0.92) . Parentsa = 0.86) [Ages and Stages Questionnaire : This sc = 0.86) . As sucha = 0.53–0.86) [Strengths and Difficulties Questionnaire (SDQ) : The SDQ53–0.86) . The SDQChild Behaviour Check List (CBCL) : The MSTBrief Symptom Inventory (BSI) : The MSTParents will be given the opportunity to provide feedback about their experience of the FPR program in a semi-structured interview lasting between 30 and 45 minutes. Questions about skills and confidence, parent/child relationships, connection to services, wellbeing, safety and home environment will be asked only to those who choose to participate. The researcher will rely on the expertise of practitioners to engage parents who can speak to their progress. We acknowledge that this approach may lead to confirmation bias, however we feel this is ethically appropriate and is in agreement with MacKillop Family Services staff views. The key worker will provide an explanatory statement and consent form to be completed prior to the interview. It is not a prerequisite that they have signed the consent form for sharing their quantitative measures to complete the interview. The interviews will take place during one of their home visits and the key worker will be there to support the parent pre- and post-interview. Due to COVID-19 pandemic restrictions, geographical distance, and privacy/confidentiality, the interviews will occur over the key worker’s phone or via Zoom, depending on the preference of the parent.To understand if the MacKillop Family Services FPR has been successful at reducing family involvement with child protection and preventing children from entering/re-entering OOHC, three types of linked data from the government are being sourced, subject to approval: (1) reports made to child first, or other child protection organisations who use the Client Relationship Information System (CRIS) system, about FPR children and parents in the six months before participating in the FPR, the six months during and then six months after program closure; (2) number of times children have been in OOHC in the 12 months post FPR participation; (3) time to reunification after removal; or (4) time to removal/re-removal.Extensive administrative data are being used to support the effectiveness evaluation of the FPR. The connection form  and its attachments  provide detailed information to support our understanding of parental context, connection to services and culture, and existing safety concerns. The closure checklist, which is completed by the key worker, also provides information about the service and education connection and participation.Acceptability, appropriateness and feasibility is being assessed using qualitative methods at both the parent and staff levels. While validated measures for these factors exist, including the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM) and Feasibility of Intervention Measure (FIM) , the conImplementation fidelity is being measured, relating to the connection, assessments, information sharing, adherence and quality. Parent readiness and the program reach is also being examined. Much of the data are sourced through case notes, presence of certain documents in the case file and routine reporting.Readiness is assessed through the connection form , case notes and the first PEEM completed by parents at baseline. The PEEM offers a snapshot of parent reflexivity and understanding of their parenting practices. For example, if parents score themselves 9 or 10 for most questions, this suggests that they may not be ready to undertake work on their parenting skills. Program reach is determined by the number of clients who participated fully in comparison to the number who refused service.The Consolidated Framework for Implementation Research  has fiveParental trauma history will be examined in two ways: (1) examination of connection documentation for personal OOHC experience; and (2) through the Adverse Childhood Experiences (ACEs) survey . The conThe primary quantitative outcome of this study is to examine the degree to which there are improvements in psychosocial outcomes among families who engage in the program. All completed questionnaires will be de-identified by a specific data aggregation worker who will assign a unique ID code  to each family so that their data remain anonymous to the research team. Descriptive statistics will be calculated based on administrative and demographic data sourced from the referral form. Main caregiver complexity will be determined using the ACEs survey and the number of ACEs will be calculated. A series of paired sample t-tests (repeated measures) will be used to compare participants’ mean differences on the measures at the beginning and end of the program. Analysis of Variance will be employed to review the number of reports before, during and after the program to determine any statistical differences. The number of care episodes and time to reunification will be analysed descriptively. All analyses will be performed using the Statistical Package for the Social Sciences (SPSS) software.All qualitative data will be analysed using thematic analysis . Data coAll project materials, including data, are stored on a secure password-protected online database. Only the chief investigator from Monash University has access to the data set. All data will be deleted six years following the conclusion of the project. Any adverse events that occur as a result of this project will be reported to the Monash University Human Ethics Committee and managed accordingly. An independent audit will occur annually to ensure appropriate trial conduct is upheld and that the data collected and stored is accurate and confidential. Given this, a data monitoring committee is not necessary.The findings will be disseminated to MacKillop Family Services via regular meetings and reports to the organisation throughout the three-year evaluation period. A summary of findings will be shared with participants. It is also intended that the findings of this hybrid effectiveness-implementation evaluation will be disseminated with the public via a series of publications in peer-reviewed journals. The research team from Monash University and key stakeholders involved in the delivery of the program from MacKillop Family Services will be included as authors on such publications, where relevant.This paper describes the protocol of a hybrid style, implementation and outcomes study using a pre-post design with selected follow-up data and qualitative data. The study will serve two key purposes. First, it will seek to demonstrate the extent to which the Response is effective in improving outcomes for families and children following the program. Whilst this is not a randomised controlled trial, the pre-post design will allow some conclusions to be made about the potential impact of the program on engaged families. In turn, this will develop the Response’s evidence base and provide new knowledge about the implementation of intensive family services more broadly. Second, the findings of this study, particularly the implementation findings, will help inform future iterations of the program. It is intended that key program components, processes, or operational issues may be adapted, refined, or remedied to improve implementation and uptake of the Response. Therefore, the evaluation will help ensure that the program is trialled, tested, and adapted accordingly to effectively meet the complex and unique needs of some of the most vulnerable families in the Victorian community.The current study is necessary and critical for generating Australian evidence regarding the effectiveness of an intensive family service program that aims to preserve or reunify families through the provision of assertive outreach support. Importantly, if the program is found to be well-implemented and yields significant improvements in families’ outcomes from baseline to closure, then this would provide strong support for the scale-up and dissemination of the Response across Australia. In turn, this could begin to foster the intergenerational outcomes that many of the target families currently experience.