Meta-analyses of telemonitoring for patients with heart failure conclude that it can lower the utilization of health services and improve health outcomes compared with the standard of care. A smartphone-based telemonitoring program is being implemented as part of the standard of care at a specialty care clinic for patients with heart failure in Toronto, Canada.The objectives of this study are to (1) evaluate the impact of the telemonitoring program on health service utilization, patient health outcomes, and their ability to self-care; (2) identify the contextual barriers and facilitators of implementation at the physician, clinic, and institutional level; (3) describe patient usage patterns to determine adherence and other behaviors in the telemonitoring program; and (4) evaluate the costs associated with implementation of the telemonitoring program from the perspective of the health care system , hospital, and patient.The evaluation will use a mixed-methods approach. The quantitative component will include a pragmatic pre- and posttest study design for the impact and cost analyses, which will make use of clinical data and questionnaires administered to at least 108 patients at baseline and 6 months. Furthermore, outcome data will be collected at 1, 12, and 24 months to explore the longitudinal impact of the program. In addition, quantitative data related to implementation outcomes and patient usage patterns of the telemonitoring system will be reported. The qualitative component involves an embedded single case study design to identify the contextual factors that influenced the implementation. The implementation evaluation will be completed using semistructured interviews with clinicians, and other program staff at baseline, 4 months, and 12 months after the program start date. Interviews conducted with patients will be triangulated with usage data to explain usage patterns and adherence to the system.The telemonitoring program was launched in August 2016 and patient enrollment is ongoing.The methods described provide an example for conducting comprehensive evaluations of telemonitoring programs. The combination of impact, implementation, and cost evaluations will inform the quality improvement of the existing program and will yield insights into the sustainability of smartphone-based telemonitoring programs for patients with heart failure within a specialty care setting. Currently, one of the greatest challenges for health care systems worldwide is the growing fiscal and social burden of preventing and managing chronic diseases . Heart fThere is mounting evidence that embedding self-care within existing health care services provides an effective model to meet these needs for a number of chronic diseases, including HF -7. AlthoIn HF, meta-analyses conclude that TM lowers the utilization of health services, improves HF health outcomes, and improves health-related quality of life (QoL) -20. HoweAlthough the ubiquity of mobile phones is believed to have the potential to make TM interventions more accessible and cost-effective ,28, addibring your own device model is planned, which would enable patients with smartphones to use their own devices by downloading the Medly app.An algorithm-based smartphone-based TM program for patients with HF, called the Medly Program, is being implemented as part of the standard of care at the University Health Network’s Ted Rogers Center of Excellence for Heart Function in Toronto, Canada (hereafter referred to as the HF clinic). The objective of the Medly Program is similar to other HF TM programs: improve patient self-management and decrease health care utilization . PatientIf there are signs of deteriorating health of a patient, the Medly algorithm generates an alert to a HF clinician who is part of the patient’s care team. The alerts are made available to clinicians in 2 formats. The first is through an automated email containing the latest weight, blood pressure, and symptoms along with the patient’s target ranges. In addition, the email contains the patients’ latest medication list and HF–related laboratory results, and contact information. Second, clinicians can choose to view the alerts by accessing a secure Web portal that presents a list of all the alerts triggered. Here, clinicians can review details of the latest alert and graphs showing historical weight, blood pressure, symptoms, and HF–specific laboratory results, which are visually contextualized according to the patient’s target ranges. The clinician will follow-up with the patient depending on the clinical need, documenting all actions and decisions taken in response to the alert in the hospital electronic medical record (EMR).The Medly Program is intended to be delivered as part of the standard of care; as such, the goal is for it to be seamlessly integrated within the existing workflows of the HF clinic. Before deployment, a service blueprint was created through ethnographic methods including observation and informal interviews with clinicians and support staff in the HF clinic. This helped identify areas where the processes of the Medly Program could be incorporated. In addition, an RCT of 100 patients with an embedded qualitative component was previously conducted in the HF clinic with an earlier version of the Medly system. This study concluded that patients receiving the intervention experienced improvements in QoL and self-care maintenance compared with a control group . LessonsTraining as to how to use the Medly app and associated devices is provided to patients at the time of enrollment into the Medly Program by a telehealth analyst. Clinicians monitoring patients through the Medly system participated in a formal training session approximately 1 month before program deployment. Both patients and clinicians are provided with a user manual to supplement the in-person training along with contact information of the telehealth analyst who offers technical support during normal business hours.The first objective is to evaluate the impact of the Medly Program on health service utilization, patient health outcomes, and their ability to self-care. Alerts sent to patients and clinicians will help identify periods of symptom exacerbation and volume overload. The impact of this is expected to permit earlier intervention for worsening conditions, thus avoiding trips to the emergency department (ED) and hospitalizations. For patients who are enrolled into the Medly Program on hospital discharge, the program is expected to reduce 30-day readmission rates. In addition, participation in the Medly Program is expected to improve patients’ ability to self-care, leading to improved clinical outcomes and QoL.The second objective is to evaluate the degree to which the Medly Program was implemented as intended and to identify the contextual barriers and facilitators of implementation. This objective will answer the following questions:To what extent did the HF clinic implement the Medly Program as intended?What contextual factors influence the implementation of the Medly Program?What adaptations were needed to implement and sustain the program within existing clinical workflows?The third objective is to describe patient usage patterns to determine adherence and other behaviors in the Medly Program. This objective will answer the following questions:To what degree do patients adhere to the Medly Program and how do adherence patterns change over time?What factors influence patient adherence?The fourth objective is to evaluate the costs associated with the implementation of the TM program from the perspective of the health care system , the hospital, and patients.Data for the 4 objectives will be collected using mixed-methods. This approach will include a multiple pre-and posttest design for the evaluation of patient-level impacts, patient adherence, and cost. Quantitative data will include data collected as part of the standard of care  and usage data from the TM system. Additional patient-level data will be collected using questionnaires at baseline, 1 month, 6 months, 12 months, and 24 months. The qualitative component will take the form of an embedded single case study . The 2 eRepresentatives from all stakeholder groups involved in the implementation of the Medly Program will be recruited for participation.Patients can be enrolled into the Medly Program provided they (1) are 18 years or older, (2) have been diagnosed with HF and are followed by a cardiologist at the HF clinic, (3) can speak and read English  to adequately understand the text prompts in the Medly app, and (4) are able to comply with using Medly . As the Medly program is being implemented as part of the standard of care, there is no explicit exclusion criteria for participating in the program and its evaluation. The duration of program participation will be decided by patients and their treating cardiologist on an individual basis.Upon enrollment into the Medly Program, patients will be presented with the option of answering questionnaires and participating in interviews related to the program evaluation. Patients will be asked to sign a written consent form before participating in the evaluation.All members of the Medly Program staff will be asked to participate in semistructured interviews. These include clinicians providing care for patients with HF in the HF clinic (n=7), telehealth analyst (n=1), project manager (n=1), and members of the implementation team (n=2). These individuals will be asked to sign a consent form before their first interview.The primary outcome for evaluating the impact of the Medly Program is the number of hospitalizations because of HF in the 6 months before versus the 6 months after enrollment. Secondary impact outcomes comparing 6-month to baseline values are described below.The number of hospitalizations , 30-day readmission rate, days in hospital , number of ED visits , visits to family doctor , number of HF–related outpatient visits, and changes to medication will be recorded.The following HF–specific clinical parameters will be collected: left ventricular ejection fraction (LVEF), brain natriuretic peptide (BNP), creatinine, sodium, potassium, hemoglobin, and uric acid levels.Patient mortality will be tracked. In addition, the Seattle Heart Failure Model (SHFM) will be calculated at program entry. Projected SHFM survival versus actual survival will be compared . The calPatients will be asked to describe their level of breathlessness using a visual analogue scale for dyspnea on a scale ranging from 0 (no shortness of breath) to 10 (shortness of breath is the worst it can be).The EQ-5D-5L is a measure of generic health status and will be administered to all patients as a measure of QoL ,35. ThisThe self-care of HF index asks respondents to respond to 22 items on a 4-point Likert scale to assess their ability to self-care across 3 subscales . The tool has undergone validity and reliability testing involving patients with HF .Demographic information will also be collected using a questionnaire, which includes questions on the following: age, sex, income, native language, living arrangements , whether or not they have a caregiver , and type of living areas . In addition, patient comorbidities will be tracked. Finally, questions will be asked to assess the patient’s experience and comfort level with technology and smartphones, including frequency of use.Data collection will occur at baseline, 1 month, 6 months, 12 months, and 24 months, or until the patient exits the program. Data collected in addition to the primary evaluation period (baseline to 6 months) will provide an opportunity for post-hoc analyses aimed at quality improvement, including recommended duration of use. For example, the 1-month time point was included to determine if the bulk of changes to self-care and QoL occur immediately after enrollment. Similarly, longitudinal data will help determine whether patient-level impacts are sustained or change when using the system long term.Health service utilization, laboratory results, mortality, prediction of survival, and select demographic information will be obtained from the hospital EMR. In addition, health service utilization information will be verified by patient participants through self-reports via a questionnaire.Baseline and follow-up questionnaires containing the validated survey tools listed in t tests and Wilcoxon signed rank tests comparing baseline and 6-month values for all patient-level outcomes. For patients enrolled in the Medly Program on hospital discharge, the 30-day readmission rate will be compared with the readmission rate before the launch of the Medly Program, as determined using hospital administrative data. Secondary analyses aimed at determining the longitudinal impact of the Medly Program , and the correlation of independent variables  with outcomes will be analyzed using general linear mixed model procedures. In addition, descriptive statistics will be produced for all variables collected, which may inform necessary subgroup analyses. All statistical analyses will be performed using the statistical software application SPSS  [The primary analyses will be paired Student on, USA) .d=0.3) in the number of hospitalizations because of HF within the first 6 months of enrollment with 80% power and an alpha of .05 (two-sided) [We will aim to recruit at least 108 patients into the Medly Program at the HF clinic before analyses of patient-level outcomes are undertaken. This number is based on being able to detect a small effect size (Cohen o-sided) . This nuThe mixed-methods implementation evaluation will be guided by the framework by Proctor et al, which describes outcomes that can serve as indicators of implementation success , and theImplementation outcomes are defined by Proctor et al as “the effects of deliberate and purposive actions to implement new treatments, practice, and services” . A totalSeparate semistructured interview guides based on the constructs in the CFIR will be formulated for each category of program staff previously outlined. Interviews will be conducted at baseline, 4 months, and 12 months to align with the different phases of Stetler et al’s typology of formative evaluations . The intDescriptive statistics will be produced for the indicators of the implementation outcome to provide an objective measure of implementation success.Furthermore, 2 independent researchers will analyze interview transcripts and documents using the Framework Method of qualitative analysis . An initQuantitative and qualitative data will be triangulated such that the interview data will help explain success or failure of the implementation.A sequential explanatory mixed-methods approach  will be Adherence will be assessed by analyzing patient usage rates of the Medly system—specifically, the proportion of days for which the patient took a complete reading  over the previous 30 days. Usage data will be obtained and exported on a regular basis through Google Analytics .A sample of patients enrolled in the Medly Program will participate in semistructured interviews aimed at understanding reasons for adherence or nonadherence and their general experiences with the Medly Program. Interview guides will be formulated based on the constructs of the Unified Theory of Acceptance and Use of Technology 2 . For exaMonthly adherence rates will be examined using descriptive statistics to identify any patterns in the increase or decrease of patient adherence to Medly over time. In addition, general linear mixed model procedures will be performed to determine if any baseline patient characteristics  or time since program enrollment predicts patient adherence to the Medly system. Semistructured interviews will be analyzed using the Framework Method , as prevThe costs associated with implementing the Medly Program will be determined from the perspectives of the public payer, hospital, and patient. In reporting these results, costs will be interpreted in relation to the patient-level impacts determined in objective 1.Costs will be calculated using a 6-month time frame. Specifically, we will compare costs before the implementation of the Medly Program (assessed at baseline) versus the costs after enrollment of patients into the program (measured at 6 months). This time frame was chosen because it represents the time horizon over which most of the health effects and costs of using the Medly Program are expected. Most of the cost variables will be self-reported by patients and triangulated using administrative data whenever possible . Therefore, questions related to the cost will be added to the patient outcome questionnaires (objective 1) and will be administered at baseline and 6 months.Costs to the public payer will be determined by looking at health care utilization of patients enrolled in the Medly Program before versus after their enrollment. These will include hospitalizations, ED visits, HF clinic visits, family physician visits, and use of home care services. In addition, costs for inpatient medications will be considered.Costs from the hospital perspective will be valued based on time spent by human resources involved in the Medly Program. This time will be converted to costs based on those individuals’ respective salaries. This will include time the clinicians spent reviewing and responding to Medly alerts as well as time the clinicians spend in training sessions, learning to use the system, and seeking technical support. In addition, costs for the hospital perspective will include equipment costs  and the salary for employing a telehealth analyst responsible for recruiting, training, onboarding, managing inventory support, and providing training.Costs for patients will primarily be determined by the time they spend accessing care for HF. This will involve determining their employment status and annual income as well as time they spend traveling to and from appointments, time spent at appointments, and how much work time was missed because of their HF condition (vacation or unpaid). In addition, this will include the time patients spent learning to use the Medly system, time spent using the system, and time getting technical support. Additional costs considered include travel, parking, and all other out-of-pocket costs related to accessing care or using the intervention. Moreover, time of informal caregivers  will be valued as part of the patient perspective. Costs for informal caregivers will be based on the average hourly rate of personal support workers.The Medly Program was launched in August 2016. As of April 4, 2018, 166 patients have been enrolled. The primary impact analysis is expected to be conducted by January 2019.This study aims to evaluate the implementation and impact of a smartphone-based TM program being implemented as part of the standard of care in a specialty care setting in a large Canadian city.Unlike TM systems evaluated in the context of academic research, the lack of strict patient inclusion and exclusion criteria for the Medly Program has the potential to lead to heterogeneity among evaluation participants, which will make it difficult to generalize the results to other health care settings. Another important limitation is that the nature of this evaluation and the availability of data do not allow for a distinct comparator group. Without discounting these limitations, we believe that one of the strengths of this evaluation is its pragmatic nature and that these threats to internal and external validity will be mitigated through a detailed description of the context and participants when results are reported. The interpretation of the Medly Program evaluation results will include comparisons with previous TM RCTs conducted within the HF clinic  and otheUnlike other TM studies that focus primarily on quantitative outcomes, this evaluation will also examine the context and mechanisms that lead to them. Therefore, this pragmatic mixed-methods study will allow for an interpretation of results using realist evaluation principles . The inf


BCR-ABL1 tests standardized to the International Scale (IS) is key to proper disease management, especially when treatment cessation is considered. Most laboratories currently use a time-consuming sample exchange process with reference laboratories for IS calibration. A World Health Organization (WHO) BCR-ABL1 reference panel was developed (MR1–MR4), but access to the material is limited. In this study, we describe the development of the first cell-based secondary reference panel that is traceable to and faithfully replicates the WHO panel, with an additional MR4.5 level. The secondary panel was calibrated to IS using digital PCR with ABL1, BCR and GUSB as reference genes and evaluated by 44 laboratories worldwide. Interestingly, we found that >40% of BCR-ABL1 assays showed signs of inadequate optimization such as poor linearity and suboptimal PCR efficiency. Nonetheless, when optimized sample inputs were used, >60% demonstrated satisfactory IS accuracy, precision and/or MR4.5 sensitivity, and 58% obtained IS conversion factors from the secondary reference concordant with their current values. Correlation analysis indicated no significant alterations in %BCR-ABL1 results caused by different assay configurations. More assays achieved good precision and/or sensitivity than IS accuracy, indicating the need for better IS calibration mechanisms.Molecular monitoring of chronic myeloid leukemia patients using robust  BCR-ABL1 tyrosine kinase inhibitors, from the first-generation imatinib to newer agents such as nilotinib and dasatinib, has enabled progressively deeper molecular responses in chronic myeloid leukemia (CML) patients undergoing tyrosine kinase inhibitor therapy.2 Deeper molecular responses are defined as BCR-ABL1 levels of ⩽0.01% (MR4) and ⩽0.0032% (MR4.5) on the international reporting scale ) and are important milestones for patients considering treatment cessation.3 Other landmarks on the IS also represent different treatment decision thresholds and prognostic outcomes.4 For example, patients who reach 10% IS or below at 3 months after treatment have significantly higher rates of MR4.5 by 5 years,5 and reaching 0.1% IS (major molecular response) by 12 months of treatment is predictive of subsequently achieving undetectable BCR-ABL1 levels.6 Thus, regular molecular monitoring using real-time reverse-transcription quantitative PCR (RT-qPCR) is recommended for optimal disease management, and treatment decisions rely on achieving milestone molecular responses in the first year of therapy and beyond.7The development of BCR-ABL1 assays across the entire measurement range is crucial for patient management, especially in patients with deep molecular responses when considering possible treatment cessation. It is well known that high variability exists between RT-qPCR methods used in different laboratories.9 The first international standardization attempt occurred in 2003, when different BCR-ABL1 assays used in the International Randomised Study of Interferon versus STI571 (IRIS) trial established IS based on 30 CML patient samples.10 Subsequently, a process for establishing a test-specific IS conversion factor (CF) by exchanging 20–30 CML patient samples with a reference laboratory was developed.11 Although this process works well for laboratories with tests that show good stability over time, it is time consuming, expensive and difficult to access for smaller laboratories.13As treatment decisions are directly impacted by test results, accuracy and precision of BCR-ABL mRNA' was developed as a primary standard for BCR-ABL1 assay IS calibration and accredited by the World Health Organization (WHO).14 The WHO panel is made of lyophilized K562 and HL-60 cell line mixtures, which allows the inclusion of cellular RNA extraction in the IS calibration against the two major BCR-ABL1 breakpoints (e13a2 and e14a2) and carries three sets of nominal %BCR-ABL1 values using ABL1, BCR and GUSB as reference genes. Owing to restricted access, the WHO panel is currently only available to manufacturers of BCR-ABL1 test kits and secondary standards.13 The commercial secondary standards available to date are made of RNA;16 thus, RNA extraction is not included in the IS calibration process, except when the standards are artificially spiked into the cell samples. Furthermore, none of these are calibrated to the WHO panel against all three reference genes.In 2010, the ‘first International Genetic Reference Panel for quantitation of BCR-ABL1 secondary reference panel that is traceable to and faithfully replicates the WHO panel in both raw materials (lyophilized K562 and HL-60 cell mixes) and manufacturing process, with the addition of a MR4.5 level. Nominal %BCR-ABL1 IS values were assigned to the secondary panel using reverse-transcription droplet digital PCR (RT-ddPCR) against ABL1, BCR and GUSB. The secondary panel was successfully evaluated by 45 different BCR-ABL1 assays in a subsequent international multi-center evaluation study.In this study, we describe the successful development of the first cell-based et al.14 with minor modifications . Calibration using BCR and GUSB was conducted in a second study using another 10 sets of WHO panels. On each day of 10 non-consecutive days, 1 WHO panel and 2–3 secondary panels were tested using RT-ddPCR in 4 replicates for the MR1 (10% BCR-ABLIS) to MR4 (0.01% BCR-ABLIS) samples and in 8 replicates for the MR4.5 (0.0032% BCR-ABLIS) sample, to enhance assay precision. Data analysis was performed using the statistical methods described by White et al.14K562 (ATCC CCL-243) and HL-60 cells (ATCC CCL-240)  were cultured, mixed and lyophilized following methods described by White ications . CalibraBCR-ABL1 and reference genes were run as singleplex reactions in separate wells. To achieve optimal assay precision and avoid signal saturation, cDNA input for BCR-ABL1 per 20 μl reaction was 80 ng for the MR1 sample, 400 ng for the MR2 sample and 675 ng for samples ⩽MR3. cDNA input for reference genes was 10 ng per 20 μl reaction for all three reference genes. For each RT-ddPCR run, wells with >9025 accepted droplets were considered valid as per the manufacturer's recommendations.RNA extraction from the secondary panel was performed using RNeasy mini kits . Reverse transcription was performed using ABI High Capacity cDNA reverse-transcription kit  and ddPCR was performed using 2X ddPCR Supermix  on the QX-100 or QX-200 ddPCR system (Bio-Rad). All primer and probe sequences are listed in BCR-ABL1 lyophilized cell reference panel, using the same K562 and HL-60 cell lines and following similar manufacturing procedures as the primary WHO panel  and Vial C (MR3) of the panel using 50, 100, 200 and 400 ng of RNA (for one-step assays) or cDNA (for two-step assays) input per PCR reaction; three replicates were run per sample at each input level , even when data from either the highest or lowest sample input were allowed to be removed based on auxiliary-pick-regression analysis (1), 29% (12 of 42) showed decreasing %BCR-ABL1 with increasing sample input (3), 22% (10 of 45) showed decreasing %BCR-ABL1 with increasing sample input  was considered the optimal cutoff value for determining differences in measurements in this study. Selecting 1 s.d. as the cutoff value took into consideration the fact that a <1-s.d. cutoff value would require a substantially larger sample size to be considered statistically robust, whereas >1 s.d. would increase the number of misclassifications.23 Thus, in this study, a mean difference of ⩾0.2log in %BCR-ABL1 value at different sample inputs by the same test was considered as beyond the inherent variability of an assay. We found that among the assays that showed changing %BCR-ABL1 against different sample inputs, 55% (12 of 22) at MR1 and 78% (14 of 18) at MR3 obtained results with ⩾0.2log difference. Overall, these results showed that some BCR-ABL1 tests were nonlinear and might therefore yield statistically different %BCR-ABL1 results against different sample inputs. To mitigate the risk of inaccurate %BCR-ABL1 measurements from using an inappropriate sample input, it is highly recommended that laboratories standardize CML patient sample inputs by quantifying the extracted RNA before performing RT-qPCR.In this study, we observed that the mean standard deviation (s.d.) in %BCR-ABL1 measurements from all 45 assays was 0.2log, which was mathematically equivalent to a 1.6-fold difference in the linear scale. Based on recommendations by Thiers BCR-ABL1 and reference gene assay using the formula ‘Efficiency=−1+10(−1/slope)'  (n=20) (n=16) ' .24 A welnalysis) . Assays ) (n=20) , whereas) (n=16) . This in results . OverallBCR-ABL1 tests. Interestingly, among the 31 laboratories that routinely quantified their CML patient sample inputs, the reported patient sample input was on average 2.4-fold higher than the calculated optimal input of the secondary panel, after two extreme outlier values of 11.6- and 48.3-fold were identified using robust regression analysis and excluded from the calculation , 750 for BCR (n=6) and 420 for GUSB (n=5) (data not shown). These results indicated that the optimal sample input of the secondary panel was approximately half of the patient sample input typically used by the laboratory in terms of ng RNA or cDNA. Thus, when using the secondary panel, laboratories might consider using twofold less cDNA input per PCR reaction compared with CML patient samples, to achieve similar copy numbers and avoid exceeding the linear dynamic range of their assay. Nonetheless, it is recommended that laboratories perform a similar standard curve experiment to identify the optimal sample input specific for their assay before using the secondary panel for the first time. The optimized sample input should maximize copy number detection but minimize potential PCR inhibition caused by carryover from the reverse-transcription reactions.As different assays had different linear dynamic ranges, we observed a >30-fold range of optimal sample input for the secondary panel calculated for the different culation . This waand 1245 , the meaBCR-ABL1 test should demonstrate good IS accuracy, precision and sensitivity within statistical limits, especially at the lower disease levels. In Study 2, laboratories were asked to use the optimal sample input determined in Study 1 to test three sets of the secondary panel on six different days, following the design previously used by White et al.  for GUSB, which differed by 1.7-fold/0.24log. For results from individual assays, 60% (27 of 45) obtained mean %BCR-ABL1 within twofold of the panel's assigned values at all levels (26 and 18% (8 of 45) used commercial BCR-ABL1 kits  successfully achieved an s.d. of ⩽0.25 log from MR1 to MR4. For BCR-ABL1 tests that obtained >0.25log s.d., precision may be improved by performing further assay optimization and increasing the number of replicates per patient sample.30Most clinical samples are typically run in only one or two replicates, which requires a high degree of assay precision to ensure accuracy of each of the final BCR-ABL1 assay sensitivity and precision are key performance characteristics. In Study 2, we found that 93% (42 of 45) assays successfully detected all replicates at MR4 and 76% (34 of 45) detected all replicates at MR4.5. To further understand how sample input affected assay sensitivity and precision, we analyzed results from the 32 ABL1 assays, as this provided the largest sample size. Among the ABL1 assays, 56% (18 of 32) detected all replicates at MR4.5 and achieved ⩽0.25log s.d., 19% (6 of 32) detected all replicates but had >0.25log s.d. and 25% (8 of 32) assays had at least 1 undetected replicate. Logistic regression analysis showed a strong positive correlation between increased sample input and increased detection rate (P=0.001). In addition, the median ABL1 copy number per PCR reaction was 98 202 for laboratories that achieved good detection rate and precision, 74 923 for those that achieved good detection rate but suboptimal precision and 29 717 for those that had undetected replicates at MR4.5, further illustrating that an increased sample input could improve the sensitivity and precision of MR4.5 detection.For monitoring deep molecular response, good BCR-ABL1 testing (4.5 sensitivity (no undetected replicate), 82% (14 of 17) showed stable %BCR-ABL1 measurements against sample inputs at MR3 in Study 1, compared with 46% (13 of 28) among assays with less optimal assay performance. This illustrated that although BCR-ABL1 assays of different designs can perform equally well, proper PCR optimization is required to ensure good clinical performance. Interestingly, the number of assays that achieved good precision (84%) and MR4.5 sensitivity (76%) exceeded the number that achieved good IS accuracy (60%), indicating that there remains an unmet need for a simple and broadly available IS calibration mechanism.The laboratories that participated in this study used diverse assay configurations for  testing . To deteBCR-ABL1 IS values of the secondary panel, it was possible to calculate an IS CF traceable to the WHO panel for each test. Before CF calculation, Bland–Altman analysis was performed for each assay as described by White et al.,14 to assess whether bias was linear across the %BCR-ABL1 range. Of all the assays, 69% (31 of 45) showed no statistically significant trend in the Bland–Altman analysis, indicating that a valid CF could be calculated for these assays and compared with the laboratory's current CF, most of which were obtained from sample exchange. Mathematically, when the ratio between the assay's current CF and the CF calculated from the secondary panel is <0.63 or >1.58, the resulting %BCR-ABL1 values will have a >0.2log difference  of assays achieved mean %BCR-ABL1 values within twofold of the panel's assigned values, 84% (38 of 45) achieved good precision (⩽ 0.25log s.d.) from MR1 to MR4 and 76% (34 of 45) achieved 100% detection rate down to MR4.5. Three factors probably contributed to these excellent results: usage of a validated optimized sample input specific to the assay, the fact that 78% (35 of 45) assays used either the EAC primer design or a commercial kit and that 71% (32 of 45) assays were IS calibrated via sample exchange with one of the two major international reference centers. This indicated that using published assay designs and a harmonized IS calibration approach may lead to BCR-ABL1 test standardization. Nonetheless, we noted that the number of assays that achieved good precision and sensitivity exceeded the number that achieved good IS accuracy, indicating that there remains an unmet need for a simple and broadly available calibration mechanism, such as this secondary panel, to ensure IS accuracy is maintained in laboratories over time.Many of the 44 laboratories that participated in the secondary panel evaluation currently act as a national reference laboratory for their country. The panel was successfully processed by all laboratories, indicating that it is compatible with many different 4.5. In addition to being a reference sample for IS calibration,14 the secondary reference material and its derivatives could also be used in assay analytical validation and optimization. For example, it can be used as standardized samples in External Quality Assessment programs for proficiency testing,31 especially as nominal values  are available for each member of the panel. Second, it can become a source of positive control samples to be run alongside patient samples for quality assurance. This can become especially powerful when multiple laboratories participate in a peer group quality-control monitoring program, in which results from such positive control samples are compared and monitored regularly. Lastly, the MR4.5 sample in the reference panel can be used to validate the sensitivity and MR4.5 detection capability of an assay.We also showed that different assay designs including different reference genes, RNA extraction methods and usage of commercial kits versus laboratory-developed tests did not affect assay performance. Nonetheless, better PCR optimization correlated with better assay performance and increased sample input improved detection rate and precision at MR4.5 level can provide easier access to IS calibration, as well as act as a tool for assay optimization, validation and quality assurance.In conclusion, a secondary reference panel traceable to the WHO panel with an additional MR


Program Science is an iterative, multi-phase research and program framework where programs drive the scientific inquiry, and both program and science are aligned towards a collective goal of improving population health.To achieve this, Program Science involves the systematic application of theoretical and empirical knowledge to optimize the scale, quality and impact of public health programs. Program Science tools and approaches developed for strategic planning, program implementation, and program management and evaluation have been incorporated into HIV and sexually transmitted infection prevention programs in Kenya, Nigeria, India, and the United States.In this paper, we highlight key scientific contributions that emerged from the growing application of Program Science in the field of HIV and STI prevention, and conclude by proposing future directions for Program Science. Progpractice , whereasProgram Science was conceptualized in response to challenges encountered at the interface of research and programs in HIV/STI prevention, where there remained a disconnect in the perspectives and priorities of scientists, program implementers and policy makers , 8, 9. PThe three spheres of a program cycle include: (1) strategic planning; 2) program implementation; and (3) program management and evaluation  at higher risk of HIV/STI acquisition as well as characterizing population-level chains of transmission.Programmatic Mapping involves a systematic approach to generating key information about the size and distribution of KPs within a defined geographic area [Innovations in hic area . Other mhic area –30 and thic area . David Whic area .spatial distribution of the epidemic at the province/state/district level as highlighted in work led by Tanser et al. and Abu-Raddad et al. [While programmatic mapping provides data on micro-level geographic concentration of risk, there has also been work to understand the macro-level d et al. , 16. Tand et al.  to reducd et al. , 35.HIV transmission at a population-level rather than focusing on HIV acquisition at an individual level. For example, condomless sex acts in the context of sex work may lead to a small number of HIV infections in the short-term, but contribute to a large number of HIV infections over time through onward transmission [Additional innovations have included approaches for characterizing HIV epidemics by understanding the causal pathway of smission –38. Disesmission .The uptake of some of these innovations into policy for resource allocation can be seen with an example from the Centers for Disease Control and Prevention (CDC). In 2013, the National STD Prevention Program in the United States was revised to incorporate a strategic planning component to its state funding allocation and provides a useful example of the application of Program Science in a northern hemisphere country context. The Division of STD Prevention at the Centers for Disease Control and Prevention in Atlanta is responsible for all of STI prevention in the United States. The funding requirements use a Program Science framework for resource allocation , using Swhere, when, and for whom interventions should be prioritized. Avahan is also a very nice example of using Program Science to determine what intervention mix is required and how to deliver these interventions in their programs [The Avahan India AIDS Initiative of the Bill and Melinda Gates Foundation was a large scale focused HIV and STI Prevention Program in South India for KPs. Avahan used programmatic mapping for strategic planning and specifically to determine programs , 39, 40.what” and the “how” also evolved. The program began to incorporate structural interventions aimed at reducing violence and improving community mobilization [Avahan clearly demonstrated the need to combine behavioural, biomedical and structural interventions to achieve the maximum impact in reducing HIV and STI rates. Biological and behavioural surveys conducted among female sex workers (FSWs) revealed a decline in HIV, syphilis, chlamydia and gonorrhea prevalence in most sex work sub-groups and most locations as a result of combination prevention interventions which included STI prevention and treatment –44. As tlization . The inclization , 46. Thelization . The useAn important dimension of Avahan’s effectiveness is the integration of community knowledge in informing its intervention mix. Ashodaya Samithi , 48, 49,Improving program efficiency requires an approach to identify and define existing opportunity gaps. The Program Science Initiative in Kenya, through a Technical Support Unit (TSU) to the National AIDS and STI Control Programme (NASCOP), developed innovative field level tools to capture data on HIV/STI prevention program indicators. HIV prevention programs in Kenya follow a combination prevention approach with a focus on biomedical, behavioural and structural interventions. The tools developed and used by these programs were developed to collect data on all aspects of the program covering all three of these intervention focus areas. Kenya, as many other countries do, has several funders of KP programs. As such, implementers were using many different reporting formats used by the many different funders. The TSU, with support from NASCOP, worked with all funders and implementing partners through the National Key Population Technical Working Group to develop standard data collection tools. A basic 15 indicator reporting tool was developed and all implementing partners were mandated to report to NASCOP on a quarterly basis on all 15 indicators , 51. ThiImportant next steps within Strategic Planning include enhanced geographic mapping along with micro level appraisals. For example, a particular challenge noted by program staff in several countries was the provision of services to young and new FSWs with high rates of HIV acquisition prior to program engagement , 53. TarAdditional future directions for strategic planning involve an adaptive design by the phase of the epidemic  and in the context of baseline and co-existing interventions , 56.Next steps for implementation include resolving tensions between agentic, individual and structural interventions with a focus on optimizing synergies across delivery platforms . ConsideThe scientific arms of Program Science comprise a range of methods and disciplines—and most importantly—a multidisciplinary scientific approach. Empiric evidence covers multiple ‘layers’, from the molecular to environmental Fig. , while cFuture expansions of the Program Science toolbox include the development of new mathematical models with novel applications; effective data visualization tools for program monitoring to reflect complex interactions; analytic frameworks to integrate multiple layers of biological (host and pathogen) and behavioural data to disentangle causal pathways to population-level transmission; resource allocation tools that incorporate explicit trade-offs within programs, health-systems, and communities.Finally, expansion of Program Science includes the development of a Community-Based Program Science framework which draws on scaling up the principles of participatory engagement.Program Science is an emerging field in public and population health. Through the country examples, this paper highlights some of the important scientific contributions that have developed over the past 5 years. Program Science as a framework is unique among other research strategies because it systematically combines the program cycle with the research strategy by embedding research within programs and having programs set and drive the research agenda. This approach requires partnership between policy makers, program leaders, service providers, researchers and communities. This combined effort results in a focus on ensuring maximum population level benefit of a program through detailed understanding of the local needs and context. This strategy has the potential to close the gap between evidence, action and policy and may be applicable to many important public health areas globally.


Nine focus groups comprising 23 program implementers recruited from nine schools were conducted to evaluate the Tier 1 Program (Secondary 2 Program) of the Project P.A.T.H.S. . Qualitative findings showed that a majority of the program implementers regarded the program as beneficial to the program participants in different psychosocial domains. The program implementers also described the program positively and positive metaphors were used to represent the program. In conjunction with the previous research findings, the present study provides further support for the effectiveness of the Tier 1 Program of Project P.A.T.H.S. in promoting holistic development among Chinese adolescents in Hong Kong.


N-methyl-2-pyrrolidone (NMP) has been shown to be the most effective solvent for liquid phase exfoliation and dispersion of a range of 2D materials including graphene, molybdenum disulphide (MoS2) and black phosphorus. However, NMP is also known to be susceptible to sonochemical degradation during exfoliation. We report that this degradation gives rise to strong visible photoluminescence of NMP. Sonochemical modification is shown to influence exfoliation of layered materials in NMP and the optical absorbance of the solvent in the dispersion. The emerging optical properties of the degraded solvent present challenges for spectroscopy of nanomaterial dispersions; most notably the possibility of observing solvent photoluminescence in the spectra of 2D materials such as MoS2, highlighting the need for stable solvents and exfoliation processes to minimise the influence of solvent degradation on the properties of liquid-exfoliated 2D materials. N-methyl-2-pyrrolidone (NMP) is the prototypical solvent for dispersion of a range of carbon nanomaterials including fullerenes1, conjugated polymers2, nanotubes3 and graphene4. This general applicability is attributed to the matching of surface tension and Hansen solubility parameters with these materials7. This results in low enthalpy of mixing and allows individualised fullerenes, nanotubes and nanosheets to be dispersed at high concentration. Consequently, there is renewed interest in NMP for 2D materials beyond graphene such as molybdenum disulphide (MoS2) and black phosphorus. These layered materials require solvents which enable exfoliation from the bulk powder and stabilisation against restacking. While the surface tensions and Hansen parameters of these materials are difficult to measure directly and inherently dependent on the technique, MoS2, black phosphorus and many other layered materials can be easily exfoliated into NMP9.10 and degradation11 under the standard processing techniques for exfoliation of 2D materials. In addition, the yield and degree of exfoliation are also sensitive to the concentration of dissolved oxygen and water in the solvent13. Such sensitivities indicate that liquid phase exfoliation in NMP is not simply described by solubility parameter theory. The process involves chemical modification of the solvent, generating degradation and polymerisation products which influence the exfoliation process and results. While this may be desirable, the nature of these sonochemical processes can result in a wide variety of products which are difficult to characterise, and may be present as residual contaminants which influence the properties of the exfoliated nanosheets. The influence of degradation during exfoliation may lead to considerable variation of the performance of NMP as a solvent for a given nanomaterial, and potentially skewing of the measured surface tension and solubility parameters of the nanomaterial towards those of the pristine NMP7.While clearly an effective solvent, NMP is also known to be very susceptible to sonochemical polymerisation10. In addition, exfoliation yield and optical absorbance of NMP have also been acknowledged to vary with the age of the solvent11. Optical spectroscopy of NMP dispersions is further complicated, albeit rarely acknowledged in the literature on nanomaterials, by weak photoluminescence (PL) of the solvent15. We have observed the enhancement of this PL with degradation, with an increase in the overall intensity and the influence of a redshifted component in both aged, untreated samples and sonicated samples. The absorbance and photoluminescence of the degraded NMP present challenges for spectroscopy of nanomaterial dispersions, with the features still observed in as-received NMP and their influence exacerbated by the sonochemical degradation processes during exfoliation of the nanomaterial.Furthermore, it has been noted that the optical properties of NMP are also influenced by its degradation, with yellowing of the solvent usually attributed to increased scattering of sonication products11. Figure The initial motivation for this study was the observation of visible yellowing of NMP samples with age as previously acknowledged by others14, the samples were illuminated with a 405 nm laser. Under this excitation, all samples were observed to exhibit strong blue-green photoluminescence with PL intensity clearly increasing with both ageing and sonication of the NMP, as shown in the inset photographs in Fig. Given that NMP has been reported to be weakly photoluminescentRaman spectroscopy was performed with laser excitation at 532 nm to compare the vibrational modes of the different samples and identify any new species present. These spectra show that NMP and NMP(s) have indistinguishable vibrational modes with fixed peak positions and intensities Fig.  which ar1H nuclear magnetic resonance (NMR) spectroscopy was performed to identify any covalently-modified species which are present at trace levels and/or too similar in their vibrational modes to be detected by Raman spectroscopy. In addition to the expected proton peaks for the NMP, the NMR spectra show a number of peaks present at around 0.1% by number. The peaks with chemical shifts of around 5 ppm are attributed to alkenyl protons, suggesting the formation of RC=CH2 species, while the peaks around 0.9 ppm correspond to protons in a methyl group. These characteristic features are also observed in a previous study of sonochemical degradation of NMP which proposes a mechanism for formation of an enamine species (–N–C=C–) which undergoes polymerisation to form oligomeric nanoparticles10. The observation of similar features in the NMR spectra suggest that this mechanism could describe degradation due to both sonication and ageing. Furthermore, the presence of these features in the NMR spectra of all samples suggests that any degradation product is also present, at lower concentration, in the as-received NMP. Notably, the alkenyl and methyl protons associated with the previously-proposed mechanism of enamine formation and polymerization, are present and show downfield shifts relative to the as-received NMP. This is consistent with the downfield shifting of peaks expected due to deshielding by a nearby π-system, which could be a conjugated species which gives rise to the absorption and PL in both aged and sonicated NMP.Subsequently, To determine whether the emergent absorption and PL are likely to be due to the pristine NMP or from a modified species, time-dependent density functional theory calculations were performed (see supplementary information). The unmodified NMP molecule was found to have optical transitions in the UV range of the spectrum only, with the primary absorption and emission features around 180 and 250 nm respectively, which strongly suggests that emerging properties in the visible range are due to a modified species. Using the previously-proposed enamine species as a starting point for degradation and polymerisation, the primary optical absorption redshifts by more than 40 nm. In addition, calculations show the emergence of a strong PL transition in the visible at 470 nm. While the modified species is likely to be more complex than the NMP derivative used, such calculations demonstrate that simple structural modifications can result in significant shifts and emergent transitions in the optical spectra of NMP.In order to ascertain the extent of the emission from the new species, PL spectroscopy was performed to obtain photoluminescence excitation (PLE) maps containing all excitation and emission spectra Fig. . These Pem = 400 nm, λexc = 325 nm) of NMP and NMP(s) are shown in Fig. −1 in Fig. The PL spectra for peak emission  highlights the potential for the solvent degradation under typical exfoliation processes for 2D materials. In order to understand the influence of sonochemical degradation on liquid phase exfoliation of 2D materials, samples of graphene and MoS2 were exfoliated by sonication into as-received NMP and pre-sonicated NMP(s). UV-visible absorption spectra for the dispersions are shown in Fig. 10, the concentration of graphene dispersions is increased for exfoliation into NMP(s), as illustrated by the 10% increase in extinction. It is possible that this is due to solvation of the graphene nanosheets by a polymeric degradation product, similar to the polymer wrapping suggested for carbon nanotubes. While covalent modification of the nanomaterial has also been suggested as a potential mechanism, we note that liquid phase exfoliation in NMP has been shown to produce defect-free graphene4 and therefore anticipate that polymer wrapping is the more likely mechanism. By contrast, the concentration of MoS2 dispersions is decreased for exfoliation into NMP(s), suggesting that the production of degradation products and their interaction with the 2D materials may be sensitive to the chemical composition of the system. While the properties of these liquid-exfoliated dispersions are also likely to be influenced by the process itself, the differences observed demonstrate the sensitivity of the 2D materials to degradation of the solvent which will occur dynamically during exfoliation and is therefore inevitable for sonication-based exfoliation processes.It has been shown that sonication of NMP results in modification of both the chemical composition and the optical properties of the solvent. Sonochemical degradation has previously been shown to influence the behavior of NMP as a solvent for exfoliation and dispersion of nanomaterials11. As such, it is not appropriate to use an untreated NMP sample as a reference for absorption spectroscopy as this will be chemically different from the sonicated NMP in the dispersion. However, if the reference sample is also sonicated, the ultrasonic cavitation field and thereby the sonochemical processes will differ significantly between the dispersion and the NMP reference sample. Importantly, this indicates that it is not possible to produce a reference sample of NMP for optical spectroscopy which is identical to the NMP in the nanomaterial dispersion. As such, the reference samples used for the spectra in Fig. 16 and highlights the need for alternative solvents and exfoliation processes.In addition, the influence of NMP degradation on the optical properties of the solvent present previously-unacknowledged challenges for spectroscopy of dispersions and solutions in NMP. Any dispersion produced by ultrasonication will be subject to degradation of the NMP, which further increases scattering in the solvent and the dispersion17, carbon nanotubes18, transition metal dichalcogenides20 and black phosphorus21. For most of these materials, the photoluminescence is not in the same wavelength range as the NMP but the excitation wavelengths used could result in broad background luminescence from the solvent, although there is likely to be some quenching by the nanomaterial. This presents the possibility of the NMP emitting into the absorption band of the nanomaterial and providing an undesired longer wavelength excitation. For example, excitation of a dispersion of MoS2 or black phosphorus in NMP at 450 nm will result in PL emission from the NMP in the range 500–600 nm and further excitation of the dispersed nanosheets, potentially resulting in additional spectral features or influencing spectroscopic measurements of such materials. We note that this is particularly problematic for quantitative spectroscopic measurements such as that of monolayer fraction from PL and Raman spectroscopy22, which could be invalidated by secondary excitation of the MoS2 by PL emission from the NMP.Finally, the most significant influence of NMP degradation is in its use as a solvent for photoluminescence spectroscopy of nanomaterials, despite the solvent itself exhibiting strong and broad photoluminescence. Such nanomaterials include conjugated polymers2 exfoliated into NMP for excitation at 450 nm is shown in Fig. 2 dispersions typically show weak PL since emission is only from monolayer nanosheets and self-absorbance reduces the outgoing signal. As such, the PL spectrum shows features at low intensity corresponding to A and B exciton PL at ~680 nm and ~625 nm respectively. The intense broad feature at ~530 nm is not due to the MoS2 but is instead due to the NMP, observed at longer wavelength under excitation at 450 nm. We note that this feature is likely to have been incorrectly attributed to MoS2 in previous studies20. To confirm this, the MoS2 dispersion was centrifuged at high speed to sediment the nanosheets out of the dispersion. The NMP was then discarded and the material was redispersed into aqueous surfactant solution in a manner similar to liquid cascade centrifugation22. Figure 2 in surfactant and water. The broad background due to the feature at ~530 nm is no longer present, confirming that it is indeed PL from the degraded NMP, and the spectrum shows only the PL from the MoS2, albeit dominated by the B exciton due to self-absorbance by multilayers in the range of the A exciton. Together these spectra demonstrate the influence of NMP degradation on the PL spectra of liquid-exfoliated 2D materials and present a potential route, through sedimentation and redispersion, to transfer nanomaterials into fresh or different solvents to minimise the influence of solvent degradation on spectroscopy of 2D materials.To illustrate the influence of NMP degradation, the PL spectrum for MoS2 in NMP shown to be dominated by the degraded solvent. These observations indicate that the use of NMP requires careful consideration for liquid phase exfoliation and optical spectroscopy of 2D materials.In conclusion, NMP, the most widely used solvent for 2D materials, has been shown to present significant challenges for liquid phase exfoliation due to its susceptibility to sonochemical degradation. It has been shown that NMP is even susceptible to such degradation with ageing under ambient conditions. Both ambient and sonochemical degradation result in chemically-similar products which give rise to strong visible photoluminescence of the NMP. In addition, this solvent degradation has been shown to influence the performance of NMP as a solvent for liquid phase exfoliation of 2D materials and perhaps even complicate the understanding of solubility parameters for solvent selection. Measurements of such dispersions are complicated by the increasing optical absorbance with solvent degradation and the nature of ultrasonic processing suggests that it is not possible to produce a sample of NMP as a reference sample for extinction spectroscopy, which is identical to the solvent in the dispersion. Furthermore, the emergence of photoluminescence of the degraded NMP influences spectroscopic measurements of photoluminescent materials in NMP, such as transitional metal dichalcogenides, with the PL spectrum of MoSNMP for the as-received and sonicated samples was purchased from Sigma Aldrich, product number 270458.NMP for the aged samples was purchased from Sigma Aldrich and Fisher Scientific with the dates of receipt and product numbers as follows: NMP-9, 06/07/2007, Sigma 328634; NMP-4, 24/10/2012, Fisher M/5120/08; NMP-1, 01/10/2015, Sigma C112402. Molybdenum disulphide powder was purchased from Sigma Aldrich. Graphite powder was supplied by Zenyatta Ventures Ltd.−1. Time-resolved photoluminescence measurements were performed with a Horiba DeltaFlex TCSPC system with excitation at 336 nm, 349 nm and 409 nm using a 6 nm bandpass. 1H NMR spectroscopy was performed on a Varian VNMRS 600 spectrometer operating at a 1H frequency of 599.7 MHz.UV-visible spectroscopy measurements were performed using a Shimadzu UV-2501PC spectrophotometer and Shimadzu UV-3600 Plus spectrophotometer using quartz cuvettes. Photoluminescence spectroscopy was performed using Cary Eclipse spectrophotometer and a Shimadzu RF-6000 spectrofluorometer. Raman spectra were acquired using an NT-MDT NTEGRA Spectra system with 473 nm laser excitation and Renishaw inVia system with 532 nm laser excitation. The photoluminescence background was subtracted using spine interpolation and the spectra were then normalized to the Raman mode at ~2900 cm2 samples, MoS2 powder was added to 20 mL of NMP at an initial concentration of 25 mg/mL. The supernatant was then discarded and the sediment was redispersed into another 20 mL of NMP. This was then sonicated for 1 hour at 50% amplitude with a pulse of 6 s on and 2 s off. The sample was then centrifuged for 1 hour at 2000 g, the sediment was discarded and the supernatant was collected. After the final centrifugation, all samples were left to stand overnight before characterisation. For the non-degraded samples, as-received NMP was used throughout the process. For the degraded samples, NMP(s) was used throughout the process.Sonochemical degradation of NMP was performed with a Sonics Vibra-Cell VCX130 ultrasonic probe. 20 mL of NMP was sonicated for 1 hour at 75% amplitude (~30 W power output), to produce the sample designated NMP(s). For the liquid-exfoliated graphene samples, graphite powder was added to 20 mL of NMP at an initial concentration of 25 mg/mL and sonicated for 1 hour at 75% amplitude. The sample was then centrifuged for 1 hour at 4000 g using a Thermo Scientific Sorvall Legend X1. For the liquid-exfoliated MoSSupplementary Information


We have several changes in the masthead of the JACMP for this summer issue. We welcome as new Associate Editors: Jefferson Fairbanks, William Salter and Albert Zacarias in Radiation Oncology Physics, and James Rodgers in Radiation Protection and Regulations. We also recognize the dedicated contributions of outgoing Associate Editors: Michael Herman and John Gibbons (Radiation Oncology Physics), James Deye (Radiation Protection & Regulations), and Richard Stark (Associate Editor at Large). Our heartfelt thanks go to the both of you for your years of dedicated service to the JACMP.This quarter, it is my pleasure to welcome a guest editorial from Paul J. Early, DABSNM, DABR, DABNM, who is Corporate RSO with Digirad Corp. The issue he describes, validation of authorized users, will be of interest to every medical physicist that practices in the United States. In particular, if the medical physicist practices in a state that comes under the direct jurisdiction of the Nuclear Regulatory Commission, this issue may be of special concern. The neglect of information contained in this editorial could result in administrative penalties for the Licensee or the Authorized User.Michael D. Mills, PhDPaul J. Early, DABSNM, DABR, DABNM, Corporate RSO, Digirad Corp.A recent NRC proceeding has established the NRC's official interpretation of certain regulations concerning duties the NRC believes are incumbent upon a licensee in verifying the accuracy of information provided by prospective Authorized Users (AUs) the licensee seeks to add to its license. These duties apply regardless of whether the prospective AUs are employees of the licensee, are independent contractors or are consulting organizations providing services.In the summer of 2004, the NRC Office of Investigations (OI) initiated an investigation to determine whether a physician (the “Physician”) listed on a licensee's (the “Licensee”) NRC radioactive material license (“RML”) had submitted false information to the Licensee to become an AU on the Licensee's existing NRC RML. Based on the evidence developed during its investigations, OI substantiated that the Physician had submitted false and/or inaccurate information to the Licensee for the purpose of being named as an AU on the Licensee's existing RML. Specifically, the NRC determined that the Physician had provided the Licensee with (a) a preceptor letter, signed by another AU, attesting that the Physician had received the required minimum level of supervised clinical and work experience to be an AU, and (b) a statement that the Physician was already an AU on another existing NRC license. Neither document was accurate.Even though the NRC also determined that the Licensee had been unaware of the falseness or inaccuracy of the information, the NRC took the position that the Licensee's act of submitting that information to the NRC constituted a potential violation by the Licensee itself, and began enforcement proceedings against the Licensee. The Licensee and the NRC then agreed to participate in an alternative dispute resolution session (“ADR”) to resolve this apparent violation and pending enforcement action. ADR is a process in which a neutral mediator, with no decision‐making authority, assists the parties in resolving any disagreements on whether a violation occurred, agreeing on the appropriate enforcement action (if any), and stipulating to the appropriate corrective measures. That ADR session was held in late 2005.The NRC took the position that, in submitting the inaccurate information, the Licensee acted in “careless disregard of NRC requirements”. The contention of the Office of Enforcement was that, even though the Licensee was unaware of the inaccuracy of the information submitted, a violation had occurred because licensees are responsible for the acts and omissions of their agents. The Licensee disagreed with this interpretation of the regulations, and contended that, because it had not been aware of any inaccuracy, it had not committed a violation.Subsequent to becoming aware of the NRC investigation and its results, and of the apparent violation, the Licensee took several actions to assure that these events would not recur. These actions included:Immediately removing the Physician from its licenseCanceling an existing business relationship with the PhysicianAttaching to physician and preceptor statements the following notice:“Notice to Physician and Preceptor: 10 CFR 30.9(a) and 30.10(a) require that all information provided to the NRC by a licensee or its agents shall be complete and accurate in all material respects. The submission of false information constitutes a serious violation of applicable regulations and may cause you or us to be fined, to lose licensing privileges, or to suffer other significant penalties.”Requiring any physician added to its license to sign and date a document containing a statement equivalent to the following:“In connection with my application to be named as an Authorized User on [Licensee's] radioactive material license, I am aware that the submission of information that is not complete and accurate in all material respects is a violation of 10 CFR Sections 30.9(a) and 30.10(a). I hereby represent and warrant that, to the best of my knowledge, the information I have submitted to [Licensee] in connection with my application to be named as an Authorized User is complete and accurate in all material respects.”An ADR session was held between the NRC and the Licensee in the NRC Region I headquarters in King of Prussia, Pa in late 2005. As described above, the session was mediated by a professional mediator. As a result of that session, as well as subsequent discussions, a settlement was reached. The elements of the settlement agreement include the following:The NRC and the Licensee agreed to disagree on the interpretation of the regulations as to whether the violation represented careless disregard of NRC requirements. The Licensee continued to maintain that its submission of inaccurate information was not in careless disregard of NRC requirements since it had no knowledge of the inaccuracies in the information provided to it by the AU.The Licensee agreed that the Physician, listed as an AU on its RML, had provided inaccurate information to the Licensee to become an AU on its license.the NRC maintained that a violation in careless disregard of NRC requirements occurred because a licensee is responsible for the acts and omissions of its agents.The NRC agreed that the Licensee did not knowingly submit the inaccurate information to the NRC, but nonetheless, The Licensee agreed that it must submit complete and accurate information to the NRC in accordance with 10 CFR 30.9(a).In addition to the corrective actions taken by the Licensee before the ADR session, the Licensee took the following additional corrective actions:The Licensee agreed, for all future NRC AU applicants and on a yearly basis, to audit the training and experience credentials of the first 10 AU applicants and 25% of any applications received after the first 10. The audit will include an attempt to locate and call preceptors as well as CME providers to verify the information given by the AU applicants. The Licensee also agreed to submit the results of this audit to the NRC at the end of a two year period, as well as to notify the NRC immediately after identification of any discrepancies as a result of the audit. The parties agreed that, if no falsifications are uncovered during the two year period, the Licensee will discontinue the audit procedure.An officer of the Licensee agreed to prepare and submit a commentary on this event to a variety of scientific journals addressing Nuclear Medicine and/or Medical Physics, as well as to include such commentary in future lectures, to provide an opportunity for other licensees in the industry to learn from this incident.In light of the corrective actions the Licensee had already taken prior to the ADR and the further corrective actions agreed upon, the NRC agreed to issue a Severity Level III Notice of Violation, but not to impose a civil penalty.An issue not covered in the ADR was the extent to which the Licensee could reasonably rely on information provided by third parties. This has particular significance when the evidence for satisfying AU credentials consists of (1) the prospective AU already being named as an AU on another NRC/Agreement State RML, or (2) the prospective AU having been certified by the Certification Board of Nuclear Cardiology (“CBNC”) or any other professional board recognized by the NRC. Can a Licensee be held responsible for any lack of due diligence on the part of these credentialing entities that supported an application in either of these situations? The Licensee thus directed a letter to the Office of General Counsel (“OGC”) of the NRC, posing the following question: “Can [the Licensee] rely, without the need for independent verification, on the fact that the physician seeking to be named on our license has already been named as an AU on another license (NRC and/or Agreement State) or has already been certified by one of the approved certification boards … as proof of the requisite training and experience?” The OGC responded that the answer to this question is, “yes”, provided a licensee:Verifies that the board certifying the physician is one that is currently recognized by the NRC;Verifies that the physician is listed as an AU on an NRC or Agreement State license; andhttp://www.nrc.gov/what‐we‐do/regulatory/enforcement.html.Verifies that no escalated enforcement action, which could jeopardize the standing of the applicant as a qualified individual, has been issued to the physician by either the NRC or an Agreement State by accessing: Since licensees are responsible for the acts as well as the omissions of its employees and agents, the NRC believes that the above steps of verification are necessary as applicable. This is true whether the prospective AU is applying to be named as an AU for the first time or has been named on another RML or is certified by a NRC‐recognized board.


Anti-pneumococcal capsular polysaccharide (PCP) IgM, IgG and IgA ELISAs have been developed to aid assessment of the adaptive immune system. The relationship between the concentrations of PCP IgM, IgG, and IgA was investigated.The concentrations of PCP IgM, IgG, and IgA were measured in sera obtained from 231 adult blood donors.Concentrations of each isotype were not normally distributed. The median concentration for PCP IgM was 54 U/mL (range 37–75 U/mL), IgG 40 mg/L (range 26–79 mg/L) and IgA 21 U/mL (range 13–44 U/mL). The median PCP IgM titres decreased with age and were significantly lower in patients aged 81–90 years compared to those aged 18–80 years. By contrast, there was a significantly higher median serum PCP IgG titre in the 61–90 years group compared to those aged 18–60 years and a significantly higher median serum PCP IgA titre in the 51–90 years group compared to those aged 18–50 years. The correlation between PCP IgG and IgA was more significant than between IgM and IgA and between IgM and IgG. Correlation of PCP IgA and IgM concentrations identified four phenotypes: high PCP IgM and IgA; high PCP IgM only; high PCP IgA only; and low PCP IgM and IgA. A significant number of individuals with a PCP IgG concentration >50 mg/L had low PCP IgA and IgM concentrations.The additional measurement of PCP IgA and PCP IgM, alongside PCP IgG, in individuals investigated for a compromised immune system may provide a more detailed antibody profile. We hypothesise that the simultaneous measurement IgA and IgM in addition to PCP IgG may give the clinician a more detailed antibody profile for the assessment of immunocompetence.22.1Serum samples were obtained from 231 blood donors  aged 18–90 years. Only subjects who were free of recurrent infections or inflammation at the time of donation (assessed by questionnaire) and whose C-reactive protein concentrations were <10 mg/L were included in the analysis. Samples were stored at −80 °C. The samples were collected in donor centres by Biomex Solutions  and purchased from Quest Biomedical . Sample collection was approved by the Institution Ethics Review Board (#05142), with all donors providing written informed consent.2.2Commercially available test kits  were used to measure PCP IgM, IgG and IgA according to the manufacturer’s instructions. All three ELISAs employ the use of capsular polysaccharide (CPS) absorption to improve the specificity of pneumococcal antibody detection. The data obtained was stratified by age  and medi2.3U tests were performed using Graphpad Prism software . p<0.05 was considered statistically significant.Shapiro–Wilk, Chi squared, Fisher’s exact and Mann Witney 3p<0.0001). The median adult concentrations (18–90 years) and IQR ranges (25–75%) were as follows: PCP IgM: median 54 U/ml, range 37–75 U/ml (n=231); PCP IgG: median 40 mg/L, range 26–79 mg/L (n=228); and PCP IgA: median 21 U/ml, range 13–44 U/ml (n=230).The Shapiro–Wilk test for Gaussian distribution demonstrated that PCP IgM, IgG and IgA concentrations from adult blood donors were not normally distributed (p=0.0017). 50% (114/226) of samples had a PCP IgG concentration greater than the median (40 mg/L). There was a significantly higher median PCP IgG titre in the 61–70 years , 71–80 years , and 81–90 years  groups compared to those aged 18–60 years (median 35 mg/L). For PCP IgA, 50% (113/226) of samples had a concentration greater than the median (21 U/mL). There was a significantly higher median PCP IgA titre in the 51–60 years , 61–70 years , 71–80 years , and 81–90 years  groups compared to those aged 18–50 years (median 20 U/mL). The age-specific concentration ranges for PCP IgM, IgG and IgA are shown in r=0.45, p<0.0001) than the correlation between IgM and IgA , or between IgM and IgG  (p<0.0001). When a concentration cut-off of 50 mg/L was used for PCP IgG, PCP IgG and IgA were again highly correlated (p<0.0001) (The correlation between PCP IgG and IgA was more significant ( p=0.86) . The num<0.0001) . When puFour phenotypes could be differentiated using median concentrations of PCP IgA and IgM : (i) higPCP IgA and IgM concentrations were measured in samples with a PCP IgG concentration>50 mg/L . 10–94% 4Measurement of PCP antibody concentrations currently relies on the IgG isotype only whereas assessment should be multi-isotypic to enable complete identification of a deficient antibody response. The purpose of this study was to determine the isotype-specific antibody titres in age defined populations to provide a baseline for subsequent studies.The production of PCP IgM is pivotal for the protection against pneumococcal disease The median concentration of PCP IgM antibodies decreased with age. The opposite was true of PCP IgG and IgA concentrations which were both stable up until approximately 60 years and then increased significantly from 61 to 90 years. A significant proportion of this older population are likely to have received a pneumococcal vaccination. Pneumovax™, which was approved by the US Food and Drug Administration (FDA) in 1983, is recommended for US individuals aged 50 years. Prevnar 7 and Prevnar 13 have been approved by the FDA more recently . Prior vaccination or exposure to pneumococcal pathogens in this older population may be responsible for the elevated IgA and IgG titres.By applying median concentrations of PCP IgM and PCP IgA as cut-off values, we differentiated 4 phenotypes, with approximately equal distribution of individuals across each group . ImportaThe measurement of PCP IgG is used to aid the diagnosis of an antibody deficiency and can be used to support the selection of therapy or treatment in those individuals. In our study, a large percentage of adult blood donors with PCP IgG>50 mg/L possessed either low PCP IgM (7–53%) or both low PCP IgM and low PCP IgA (7–15%). A PCP IgG concentration>50 mg/L In conclusion, we report reference ranges in adult blood donors for PCP IgM, IgG and IgA concentrations, and offer an explanation for the trends observed. A substantial number of individuals in our population did not have PCP IgM, IgG or IgA concentrations suggestive of protection from infection


Background: The Tuberculosis Operational Research Group (TORG) implemented a capacity-building model involving academics and practitioners (i.e. clinicians or program staff) in an operational research (OR) team in Indonesia.Objective: This study explored academics’ and practitioners’ perspectives regarding the benefits of participating in a tuberculosis (TB) OR capacity-building program in Indonesia.Methods: We conducted a qualitative study involving in-depth interviews with 36 academics and 23 practitioners undertaking the TORG capacity-building program. We asked open-ended questions about their experience of the program. Data were analyzed via content analysis.Results: The findings demonstrated the social multiplier effects of the OR capacity-building program. Both academics and practitioners reported perceived improvements in research knowledge, skills, and experience, and described additional individual- and institutional-level benefits. The individual-level benefits level included improvements in understanding of the TB program, motivation for research and self-satisfaction, the development/enhancement of individual networking, receipt of recognition, and new opportunities. The additional benefits reported at an institutional level included improvement in research curricula, in-house training, and program management and the development/enhancement of institutional partnerships.Conclusions: The program improved not only individuals’ capacity for conducting OR but also the quality of the TB program management and public health education. OR should be included in research methodology curricula for postgraduate public health/disease control programs. The capacity-building model, in which academics and program staff collaborated within an OR team, should be promoted. Operational research (OR) has recently received increased attention because of its contribution to efforts to control tuberculosis (TB) epidemic . OR is eThere are several OR capacity-building programs, including the Union/Médecins Sans Frontières Structured Operational Research and Training Initiative program  and the Naidoo et al.  suggesteSince 2004, the TORG program has been conducted as an OR capacity-building program for provincial OR teams. The provincial OR team consists of two academics and two–three clinicians or disease control program officers (practitioners). The OR capacity-building program was conducted as a series of workshops over 1.5 years and included the following: a proposal development workshop, project implementation mentoring, a data management and analysis workshop, and a report-writing workshop. The workshops applied experiential learning methods   who participated in the TORG workshops. We received a list of academics and practitioners from 33 provincial OR groups that participated in TORG workshops from TB Sub-directorate, Ministry of Health Republic of Indonesia. We conducted a purposive sampling to recruit the principal investigator and another co-investigator from each team, who were undertaking the series of TORG capacity-building workshops completely. The rationale of this sampling method was selecting the informants with rich information about their experiences from participating TORG capacity building. Using the list we contacted the potential study participants. Following the qualitative study method, we determined the number of study participants when data was saturated .Data were collected via in-depth interviews. To minimize the bias to the position of the authors as TORG members, a trained research assistant conducted the interviews under the supervision of AP. We developed an interview guide, which consisted of the following questions:How would you describe the experience you have gained from your participation in the TORG capacity-building program?How did the experience influence your career?How did the experience influence your research interests?How did the interactions with OR peers influence communication between your relative institutions?All interviews, which included the provision of informed consent, were audio recorded. The research assistant wrote verbatim transcripts within 48 hours of each interview. AP checked the accuracy of the verbatim transcripts. AP and BW conducted individual coding of the data. The results of the individual coding were discussed in the data analysis and interpretation with all researchers.An inductive qualitative content analysis ,14 was pTo increase the trustworthiness of the qualitative study , we perfWe sent emails to participants, to inform them about the study and assure them of their autonomy, prior to initiating the interviews. All of the participants provided verbal consent, which was audio recorded. We maintained participants’ anonymity throughout the interview transcription and data analysis and presentation. Ethical approval for the study was granted by the ethics committee, Faculty of Medicine Universitas Sebelas Maret in Indonesia. The study was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2008.We approached 63 participants via email, short messages, and/or phone calls, and four practitioners from four OR groups did not respond. Therefore, we ultimately analyzed 59 verbatim transcripts of interviews with 36 academics and 23 practitioners.Our data analysis showed two levels of effect: individual and institutional. Individual effects were categorized into six categories: strengthening of research knowledge, skills, and experience; improvements in understanding of the TB program; increases in motivation for research and self-satisfaction; development/enhancement of individual networking; receipt of recognition; and new opportunities. Institutional effects were categorized into three categories: improvements in research curricula and learning processes, improvements in program management, and development/enhancement of institutional partnerships. Examples of the data analysis processes that consisted of questions, meaning units, codes, categories, and subthemes are shown in Almost all of the participants (56 of 59) reported improvements in research knowledge, skills, and experience. They described attainment of knowledge and skills, particularly on OR, qualitative research, and data analysis and presentation. Besides this, they reported improvements in research dissemination skills including those involving presentation and writing policy briefs and scientific articles. One of the participants described this as follows:By participating in the TORG program, I obtained a lot of experience in how to develop a good research proposal and conduct proper data analyses. Another participant stated,I was introduced to thinking about what has happened and why it happened, then analyze that and write small articles, which I then send to the city government magazine. The program also provided experience in conducting applied research and working in multidisciplinary teams.The program improved participants’ knowledge, particularly academics’ knowledge, regarding the TB program. This perception was reported by one-third of the academics. One participant stated,I came to understand that TB is really a community problem. A few practitioners reported attainment of deeper knowledge regarding the TB program. One practitioner stated,The field study I did automatically add to my understanding of staff in real situations in the TB program. It was an incredible experience. This could help me, as a programmer, to record the real situation in the community. (Participant 59)The experience of participating in the TORG program motivated almost one-third of the participants to conduct further OR. A practitioner from West Java province conducted further OR in a hospital setting. In addition, academics in Central Java were motivated to conduct OR in other districts and for other programs. Remarkably, a practitioner in Papua accepted a research position at the National Institute of Research and Development because the TORG program inspired him about a job as a researcher. In addition, several participants continued their education, with TB as a specific research topic.Participation in the TORG program improved participants’ motivation for conducting research. The participants were pleased that they had been provided with the opportunity to implement the knowledge regarding research methods that they had obtained from schools. One academic stated,I feel like my battery has been recharged. (Participant 24)For practitioners, in particular, the experience of program participation provided new insight regarding research, as expressed in the following participant’s statement:I feel like I have been in a world other than my own. (Participant 7)A few participants reported that program participation had induced self-satisfaction because of their contribution to the improvement of the program. One participant stated,In OR, the study results have the potential to be utilized in the program. As the researcher, I feel proud when that happens. Almost half of the participants reported that the TORG program had triggered new individual network connections or intensified existing connections between academic and program staff. Individual networking facilitated academic or TB program-related activities, data collection, and information sharing. One participant described this as follows:Before I participated in the training, I had no connection to program staff. By participating in the training, I could have discussions, share information, and consult with program staff. Another participant stated,I got to know some university staff. I helped them and their students to conduct research in my district. Participants described an increase in recognition following their participation in the TORG program and OR. One practitioner (Participant 17) stated that fellow program staff members had recognized her as a TB or research expert, while other practitioners reported appreciation from the Ministry of Health as another form of recognition. From academics’ perspectives, examples of recognition included appreciation from other researchers and invitations to speak at scientific seminars, serve as internal reviewers for research grant selection at universities, and review manuscripts for scientific journals.Networking between academics and practitioners provided additional opportunities. Academics reported receiving invitations to speak at seminars arranged by program staff. In addition, a few participants described receiving fellowship opportunities for TB research from an international donor and invitations to review manuscripts for scientific journals. Moreover, participation in the TB research fellowship provided the opportunity to receive a grant from the international donor for OR concerning tobacco control. In addition, one academic stated that the experience of conducting OR had inspired him to develop a community service project in collaboration with students and the provincial health office.This category was revealed from the interviews with academics. Two-thirds of the academics acknowledged improvements in their institutions’ research curricula and learning processes as benefits of OR capacity building. Most academics reported using the field experience in the TORG program as research methodology learning materials for their students. One academic stated,I frequently use my experience from OR to illustrate good data collection processes. (Participant 22)Other academics used materials and experiences obtained via the TORG program to develop in-house training for other faculty members at their institutions. In addition, some academics replicated the teaching techniques used in the TORG workshops and field mentoring. Academics acknowledged that their program participation had enriched their supervision of students’ research.Both academics and practitioners reported that OR improved program management. Practitioners improved their problem analysis skills via their program participation; therefore, it enhanced their ability to analyze and solve problems in program management. One participant stated,I knew the problem with the TB control program, but I learned how to analyze it in the TORG. In addition, the results of OR broadened program staff members’ perspectives regarding innovation in program management. One participant stated,Our study results opened up the perspective of the provincial health office regarding health, allowing us to develop program innovations that we only knew about from books. The interactions between academic and practitioner established new links between institutions, such as those between academic institutions and provincial health offices or nongovernmental organizations. They also improved existing institutional partnerships between academic institutions and provincial health offices. An academic in Bali province described intensive joint activities between universities and provincial health offices. One academic from West Nusa Tenggara province stated,Yes, this program strengthened the link [between the provincial health office and university]. Besides strengthening individual links, it also improved our memorandum of understanding with the municipality health office. (Participant 39)The findings also demonstrated a relationship between individual and institutional effects . PerceivWe named the relationship between individual and institutional effects as a ‘social multiplier effect’ as a theme represented the perceived benefit of the TORG capacity-building program from academics’ and practitioners’ perspectives. The theme demonstrated that the program exerted not only individual effects on research knowledge, skills, and experience of the study participants but also other individuals related to institutional effects such as curriculum development and networking.The study demonstrated that the OR capacity-building program exerted social multiplier effects, indicating that it exerted multiple effects from both academics’ and practitioners’ perspectives. The social multiplier effects theme in our study resulted from our reflection on a connotative meaning of an economic concept of positive social multiplier. The positive social multiplier effect is a spillover of individual behavior to other individuals due to social interactions among them . We meanThe findings showed that the capacity-building program exerted individual effects on the motivation for conducting further OR and improved partnerships between individuals, which is similar to the results of previous evaluations ,17. The The results demonstrated three types of institutional effect: improvements in research curricula and learning processes, improvements in program management, and development/enhancement of institutional partnerships. The institutional effects were observed in relation to individual effects. Perceived improvements in research curricula and learning processes reflected improvements in practice among academics. Perceived improvement of data analysis in program management reflected improvements in practice among practitioners. Both academics and practitioners expressed the institutional partnership enhancement effect.Individual and institutional effects were induced by the application of four experiential learning methods in the TORG capacity building. The participants did not only learn from concepts from lectures, but they also learned from the field works of OR. They received feedback from supervision and mentoring sessions by TORG members. Duration of the TORG capacity-building program (i.e. over 1.5 years) can contribute to the intensity of interactions between the academics and practitioners; hence it can contribute to the institutional networking enhancement.Boyko et al.  identifiThe perceived effects of individual networking and institutional partnerships required the context of sufficient intensity of communication between academics and practitioners in OR teams. Moreover, our previous research revealed that communication problems existed within OR teams . One strThe role of the authors as the Tuberculosis Operational Research Group members on one hand is instrumental to conduct this evaluation. This role facilitated access to information on the sampling frame of informants and helps to familiarize the context and phenomenon. On the other hand, it could introduce biases in data collection and analysis. To minimize this potential bias, we recruited an experienced anthropologist, who was not a TORG member, to conduct interviews and analysis together with AP.The TORG capacity-building program exerted multiplier effects at both individual and institutional levels. The academics and practitioners perceived that the program improved individual capacity and motivation in research, and enhanced networking between academics and program staff. In addition, the program triggers the development of institutional links and improved the quality of research curricula in the academic institution and disease control program management of health authorities.The integration of academic and program staff in the TORG represents a breakthrough in addressing the challenges arising from program staff members’ limited capacity for conducting OR. On the other hand, it allows academics to understand the operational issues of a public health program. Other concerned about inapplicable research due to lacking collaboration between academics and users of research . TherefoTherefore, we recommend the intensification and promotion of OR and enhancement of OR capacity building for both academic and program staff as also yield by other . In addi