To what extent is economic growth associated with reduced prevalence of childhood malnutrition, what are the most important contributing factors of malnutrition, and how are these contributing factors associated with economic growth?This cross-sectional study of 1 138 568 children aged 0 to 35 months from 58 low- and middle-income countries found an ambiguous association between economic growth and the prevalence of childhood malnutrition. Moreover, while strong associations between several contributing factors and childhood malnutrition were identified, the associations between economic growth and these contributing factors themselves were often ambiguous.These findings suggest that to reduce childhood malnutrition, economic growth should be accompanied by targeted investments that improve contributing factors of malnutrition not necessarily affected by economic growth, such as maternal human capital. This cross-sectional study investigates the association between per-capita gross domestic product and children’s risk of being malnourished in low- and middle-income countries (LMICs) and evaluates contributing factors that provide the broadest scope for economic growth to reduce malnutrition. Economic growth may reduce childhood malnutrition through improvements of several contributing factors, but the empirical evidence is mixed. Identifying the most important factors that contribute to child malnutrition and their associations with economic growth can inform decision-making about targeted investments to improve children’s health.To assess the associations between economic growth and malnutrition, contributing factors and malnutrition, and economic growth and contributing factors of malnutrition in low- and middle-income countries (LMICs).This cross-sectional study used data from 239 Demographic and Health Surveys from January 1, 1990, to December 31, 2021. Observations included 1 138 568 children aged 0 to 35 months with valid anthropometric measures and information on contributing factors of malnutrition from 58 LMICs. Data were analyzed from May 20, 2022, to February 16, 2023.National per-capita gross domestic product (GDP) was used as a proxy for economic growth.z score <−2), underweight (weight-for-age z score <−2), wasting (weight-for-height z score <−2), overweight (weight-for-height z score >2), obesity (weight-for-height z score >3), and dietary diversity failure (consumption of less than 5 of 8 different food groups in the past 24 hours). Eighteen contributing factors of malnutrition were constructed, of which 10 were underlying determinants  and 8 were immediate determinants .Six measures of childhood malnutrition were constructed: stunting  were included in the analysis. Of these, 27.3%  had stunting; 25.7% , underweight; 11.2% , wasting; 3.8% , overweight; 1.1% , obesity; and 79.8% , dietary diversity failure. Per-capita GDP was weakly associated with childhood malnutrition. The odds ratios associated with a 5% increase in per-capita GDP were 0.99  for stunting, 1.01  for wasting, 1.00  for underweight, 0.98  for overweight, 0.98  for obesity, and 1.03  for dietary diversity failure. Although strong associations were found between many contributing factors and most outcomes for malnutrition, associations identified between per-capita GDP and these contributing factors themselves were ambiguous.In this multicountry cross-sectional study, economic growth was weakly associated with childhood malnutrition and several contributing factors. To reduce child malnutrition, economic growth may need to be accompanied by more targeted investments to improve contributing factors that are strongly associated with child malnutrition, such as maternal health and education. Enabling determinants include political, financial, social, cultural, and environmental conditions and influence children’s health through immediate and underlying determinants. Immediate determinants of malnutrition are more directly related to child malnutrition, like diets and care, while underlying determinants of malnutrition refer to family and community characteristics influencing a child’s well-being more indirectly   developed a conceptual framework of malnutrition in 1990 as part of its Strategy for Improved Nutrition of Children and Women in Developing Countries.ucation) .10 while a negative association was found for Egypt.11 Two studies12 used large samples of pooled Demographic and Health Surveys (DHS) and found an inverse association between childhood stunting, underweight, and wasting and economic growth. One study13 found a large, inverse association between childhood stunting and economic growth based on pooled DHS data from 20 African countries. Empirical studies that investigate multiple household-level contributing factors of childhood malnutrition are also scarce. One recent cross-country study14 considered 9 direct and 17 indirect factors but analyzed only stunting, underweight, and wasting . Other studies have focused on a specific country, like India15 and Kenya.16 A recent systematic review17 identified maternal educational level, household income, maternal nutrition, and access to sanitation as the most important contributing factors of childhood malnutrition.Few studies have analyzed the association between aggregate economic growth and children’s risk of being malnourished. No evidence for such an association was found for India,5 but used substantially more surveys and countries and considered overweight and obesity, as well as dietary diversity failure. We analyzed the associations of household-level contributing factors with malnutrition to test which contributing factors may provide the biggest scope for economic growth to reduce malnutrition. Last, we investigated the association between per-capita GDP and these contributing factors.In this study, we contribute to the understanding of the associations among economic growth, childhood malnutrition, and several potential contributing factors in low- and middle-income countries (LMICs). Combining individual-level data from 239 DHS from 58 LMICs, we first investigated the association between countries’ per-capita gross domestic product (GDP) and children’s risk of being malnourished. We built on work by Vollmer et alSTROBE) reporting guideline.This cross-sectional study was a complete case analysis. As DHS are publicly available anonymized data sets, this study was exempt from institutional review board approval and the need for informed consent based on national and institutional policies by ICF International, Inc. We complied with the Strengthening the Reporting of Observational Studies in Epidemiology  in logistic regression models correspond to a 5% increase in per-capita GDP.20 We considered binary indicators of stunting (1 if height-for-age z score <–2), wasting (1 if weight-for-height z score <–2), and underweight (1 if weight-for-age z score <–2). Overnutrition was measured by binary indicators of overweight (1 if weight-for-height z score >2) and obesity (1 if weight-for-height z score >3). For stunting, z scores were calculated as the child’s height minus the median height for that child’s age and sex in the WHO reference population, divided by the SD of this group in the reference population.21 The z scores for the other outcomes were calculated analogously. Biologically implausible values  were excluded.20 We also analyzed dietary diversity failure since anthropometric measures alone do not sufficiently capture malnutrition and should thus be complemented with dietary-based measures.22 This was measured as a binary indicator based on a score ranging from 0 to 8, with 1 point assigned for consuming grains, roots and tubers, legumes and nuts, dairy products, flesh foods, eggs, vitamin A–rich fruits, or vegetables in the past 24 hours before the interview. Scores lower than 5 indicated dietary diversity failure.Our primary outcome was individual-level childhood malnutrition, following the World Health Organization (WHO) 2006 Child Growth Standards.14Our secondary outcomes were potential contributing factors of malnutrition , size, and location . Child and mother control variables included the child’s sex, age, and birth order and the mother’s age at birth. Additionally, we added indicators for the countries to account for all observed and unobserved country-level factors (or above). We also controlled for the survey year to control for period effects and potential differences in measurement across time.Data were analyzed from May 20, 2022, to February 16, 2023. Our main analysis consisted of 3 parts. First, we used logistic regression models to obtain ORs and 95% CIs for the association between per-capita GDP and our outcomes for child malnutrition, including undernutrition, overnutrition, and dietary diversity failure . Second,In all regressions with malnutrition as outcome, household, child, and mother control variables were included. In all regressions with contributing factors as outcomes, household control variables were included. We clustered SEs by primary sampling unit to account for intracluster correlation within primary sampling units, and we controlled for country fixed effects in all models to mitigate omitted variable bias. When using multiple DHS per country, that is, whenever per-capita GDP was the main regressor, we also included survey year fixed effects to control for global trends in GDP growth and our outcomes of interest.23 This method estimated the relative contribution of the different covariates to an outcome’s explained variance.To synthesize the importance of the various factors considered in our analyses, we used a Shorrocks decomposition.We classified associations based on the magnitude of ORs (in terms of distance from 1.00) obtained through the regression analyses. Large ORs indicated strong associations; small ORs, weak associations. We classified associations as ambiguous when ORs were generally close to 1.00 (weak) with some negative and some positive.24 Second, we trimmed the sample to exclude extreme observations with regard to childhood malnutrition . Third, we used instrumental variable regressions with the investment share of GDP 5 years ago as an instrument for log per-capita GDP to address 2 potential statistical problems: measurement error in GDP that could bias the results downward, and endogeneity of GDP, which could bias the findings because of either reverse causality or omitted variable bias.We conducted several sensitivity analyses based on a linear probability model. First, we reweighted the observations with the population size of the country using data from the United Nations population prospects.P < .05 was chosen a priori to represent statistical significance.We used Stata, version 17.0 (StataCorp LLC) for all analyses. All statistical tests were 2 tailed, and A total of 1 138 568 children  were included in the analysis. In the pooled sample, 27.3%  of children had stunting; 25.7% , underweight; 11.2% , wasting; 3.8% , overweight; 1.1% , obesity; and 79.8% , dietary diversity failure.The P < .001) and a 0.7% increase in the odds of wasting . Per-capita GDP was not associated with underweight . In contrast, associations were found for overnutrition. A 5.0% increase in per-capita GDP was associated with a 1.9% decrease in the odds of having overweight  and a 2.4% decrease for having obesity . Last, a 5.0% increase in per-capita GDP was associated with a 2.5% increase in the odds of dietary diversity failure . Except for wasting, the results were robust to using linear probability models, to weighting observations by countries’ population size, and to excluding outliers. Only overweight and obesity were robust to the instrumental variable specification  and incomplete course of vaccination , while for most remaining covariates, ORs were either insignificant or negative and of small to moderate magnitude . A similDietary diversity failure was positively associated with 10 of 15 contributing factors, most strongly with incomplete course of vaccination , followed by unsatisfied family planning needs , no consumption of oral rehydration solution despite diarrhea , no care seeking for suspected pneumonia despite cough , no skilled birth attendant , no maternal education , inadequate antenatal care , no access to improved sanitation facilities , delayed breastfeeding initiation , and low maternal BMI  . These reFigure 3 in We found negative associations between most underlying determinants and per-capita GDP. Apart from unsafe practices for child’s stool disposal , all ORs were very small. No access to safe drinking water and unsatisfied family planning needs were positively associated with per-capita GDP. Unsafe practices for child’s stool disposal and access to improved sanitation were robust to all sensitivity tests, and inadequate antenatal care, low maternal height, and low maternal BMI were robust to all except instrumental variable regressions . Second, children’s nutritional status could be a proxy for overall population health, which affects the economic development of a country.30 Concerns of potential reverse causality and unobserved heterogeneity were mitigated through instrumental variable regressions (eTables 3 and 5 in 5This study has some limitations. The first concerns the measurement of undernutrition, in particular underweight and wasting. Improvements in these outcomes might not necessarily indicate improved nutritional status but could also mirror a change in diet toward more sugar and animal fats.In this cross-sectional study, economic growth was weakly associated with childhood malnutrition and several contributing factors. Economic growth should not be considered a sufficient condition to improve childhood malnutrition but should be accompanied by targeted investments to improve important contributing factors, such as the human capital of mothers, providing them with increased opportunities to reduce malnutrition among their children.


Insects possess a very strong and flexible protein in their appendages and wing joints called resilin. In flying insects, resilin provides wing flexibility that improves flight and movement dynamics. In our study, we investigated how resilin changes with age in honey bees. We discovered that resilin genes are expressed at high levels in worker honey bee pupae, but then levels decline significantly with age. We also determined that the resilin protein is found at higher levels in some joints in the newly emerged honey bees compared to older age classes, further confirming the age-dependent nature of resilin presence in honey bee wings. Specifically, resilin is highly expressed in pupae, deposited at high amounts in some joints in newly emerged honey bees, and then expression and deposition both decline rapidly with age. Resilin must be kept hydrated to stay functional, so these results suggest that although resilin genes may not be expressed significantly after emerging, the resilin protein is likely maintained through fluid support to allow for strong flight dynamics throughout life. Resilin expression and maintenance should be explored for their potential to be used as an indicator of wing development and overall wing health in aging honey bees in the future.Apis mellifera (honey bee) worker age classes and drone honey bees. Resilin gene expression was determined via ddPCR on whole honey bees and resilin autofluorescence was measured in the 1m-cu, 2m-cu, Cu-V, and Cu2-V joints on the forewing and the Cu-V joint of the hindwing. Resilin gene expression varied significantly with age, with resilin activity being highest in the pupae. Autofluorescence of the 1m-cu and the Cu-V joints on the ventral forewing and the Cu-V joint on the ventral hindwing varied significantly between age classes on the left and right sides of the wing, with the newly emerged honey bees having the highest level of resilin autofluorescence compared to all other groups. The results of this study suggest that resilin gene expression and deposition on the wing is age-dependent and may inform us more about the physiology of aging in honey bees.The presence of resilin, an elastomeric protein, in insect vein joints provides the flexible, passive deformations that are crucial to flapping flight. This study investigated the resilin gene expression and autofluorescence dynamics among  Apis mellifera, the European honey bee, is the single most important species of pollinators to human agriculture. Its wide distribution and use of generalist strategies allow A. mellifera to be of importance to global natural habitats and agricultural production [A. mellifera colonies during the brood-rearing months is a temporary physical caste system, where work is based on age and morphological specialization [A. mellifera follows a “push-pull” model in which the nurse class is “pushed” into the guard class by the development of the newly emerged honey bees and the guard honey bees are “pulled” from classes by the population of foragers [oduction . As withlization ,3,4. Thilization ,6. Tempoforagers .A. mellifera has many advantageous flight adaptations—indirect flight mechanisms and asynchronous flight allow for increased flapping flight frequencies; rows of hamuli link the forewing to the hindwing, creating a larger airfoil surface that generates greater lift; and the presence of wing joints embedded with resilin [A. mellifera flapping flight. It increases load-lifting capacity, which the foragers need to carry pollen; increases vertical force production; reduces drag ratios; increases body stability and decreases bodily rotation during turbulent airflow; increases chamber and cupping, which aid in lift; reduces energy expenditures by nearly 25%; and passively stabilizes the insect while hovering [The forager class flies the farthest distance of any biological age class, making wings and flight ability crucial to their role.  resilin ,9,10,11. resilin ,13,14. Whovering ,16,17. TInsect wings are composed of two layers of integument. Each of these layers contains cuticle, a chitinous, extracellular matrix. The wing veins contain six layers of cuticle. Inside the vein, hemolymph, neurons, tissues, and tracheae exist, hydrating the cuticle of the wing membrane and allowing the membrane to retain its elasticity ,19,20. TResilin is the most efficient elastomer known with 97% resilience under high-frequency conditions and a fatigue lifetime of 300 million cycles ,22,23. TEpiophlebia superstes have indicated that resilin distribution on wings varies between species and individuals of the same species [A. mellifera workers experience physiological variations between biological age classes including, but not limited to, expression levels of vitellogenin and adipokinetic hormone receptors, stress resistance, and fat body sizes [A. mellifera biological age classes and be an age-dependent predictive marker of wing development has not been reported. This study explored the hypothesis that resilin varies among biological age classes in honey bees.Previous studies on the dragonfly  species ,31,32. Ody sizes ,34,35,36dy sizes . HoweverHoney bees were collected from Bountiful Blossoms Bee Company, LLC in Glenwood, IA, USA from May–June 2019. Biological age class was estimated on-site using class-specific characteristics and behavioral norms ,38. SpecFor each honey bee, the wings were dissected along the axillary sclerites. Each wing was first swabbed with 70% ethanol and then secondly with a solution of 50% Triton X and 50% distilled water to clean off pollen and other particles on the wing. The fluorescence of resilin is sensitive to pH: in neutral and alkaline media, the excitation/excitation maxima are 320/420 nm, while in an acidic media, they are 285/420 nm . The wasWings were observed using fluorescence microscopy on an Olympus BX61 microscope  equipped with an aniline blue filter . Images were taken with an Olympus SC100 Color Camera that was connected to the microscope. Initial images of resilin were taken with samples and underwent tests to determine the exposure that allowed for the greatest clarity of resilin across multiple joints and focal planes without saturation. All images were taken at 20× magnification and 450 ms exposure. The dorsal, ventral, left, and right sides of all wings were imaged at the first medio-cubital (1m-cu), second medio-cubital (2m-cu), cubito-vannal (Cu-V), and second cubito-vannal (Cu2-V) joints on the forewing and the cubito-vannal (Cu-V) joint on the hindwing .All images were blinded by a third party and uploaded to ImageJ version 15.2 . The scThe autofluorescent resilin was manually traced three times and the average for each measurement was found. Then three selections of the background were taken and the average for each measurement was found. The corrected total cell fluorescence (CTCF) equation, a quantitative fluorescence imaging analysis, was used to calculate the final measurement: CTCF = Mean Integrated Density −  . CTCF caTo prepare RNA for downstream reverse transcription and droplet digital PCR (ddPCR), the RNA from whole honey bees stored in RNAlater was extracted using the Qiagen RNeasy Mini Kit  with some adjustments as per the manufacturer’s instructions. Initially, the samples were homogenized in RNAlater using cold, sterile zirconia beads and a bead beater. A small portion of the homogenate was then mixed with lysis buffer (buffer RLT with dithiothreitol (DTT)). After another round of homogenization in the lysis buffer and subsequent centrifugation, the resulting supernatant was transferred to a separate tube containing 70% ethanol to eliminate any debris that could potentially obstruct or harm the Qiagen RNeasy spin column membrane. The remaining steps for RNA isolation followed the manufacturer’s protocol.For reverse transcription of samples, the Bio-Rad iScript cDNA Synthesis kit  was employed following the manufacturer’s instructions. The following thermocycler settings were utilized: priming for 5 min at 25 °C, reverse transcription for 20 min at 46 °C, and reverse transcription inactivation for 1 min at 95 °C. The samples were subsequently stored at −80 °C until they were required for further analysis.A. mellifera and its similar magnitudes of expression compared to pro-resilin [Droplet digital PCR was completed according to the manufacturer’s protocol using the Bio-Rad QX200 ddPCR System. Pro-resilin gene expression analysis was used to make inferences about resilin deposition timing as it related to worker age classes. Pro-resilin is the precursor protein that gets transported to the subcuticular space before crosslinking to form a functional resilin protein network . The con-resilin ,44,45.A ddPCR was then performed according to Tokach et al. . Briefly4,161 = 18.29; p < 0.001), we used the ‘lm.rrpp’ function from the ‘RRPP’ R package to perform a permutational ANOVA, using age class as the explanatory variable and resilin expression as the response data.As there was unequal variance in resilin expression among age classes (Fp < 0.001), we used the ‘lm.rrpp’ function from the ‘RRPP’ R package to perform a permutational ANOVA, using age class, joint type, and an interaction term as explanatory variables and fluorescence as the response variable. We then performed a pairwise comparison test using the ‘pairwise’ function from the ‘RRPP’ R package to identify statistically significant differences among age classes at each wing joint. R code used in this analysis can be found at https://github.com/andrewckraemer/ResilinAnalysis_2023Apr18_ACK (accessed on 18 April 2023). As resilin fluorescence was not normally distributed . Specifically, the pupae age class expressed significantly more resilin than all other groups . We also found significant differences in the expression of resilin among age classes depending on the joint type, except for all dorsal joints, both ventral and dorsal 2m-cu and Cu2-V joints, and 1m-cu on the ventral hindwing. Among joint types with significant differences, the newly emerged age class expressed the most resilin (p < 0.001). Representative images of each joint with significant differences in autofluorescence are shown in Resilin autofluorescence was best predicted by a model with the main effects of age class and joint type as well as an interaction term  were typically located on the ventral side of the wing and were visually brighter during fluorescent microscopy. Studies on other species such as Manduca sexta also indicate differences between ventral and dorsal wing flexibility [There were no significant differences between the left and right wings at the group level in this study. Although the ventral sides of the wings had significant differences in resilin deposition among groups, there were no significant differences among groups for the dorsal sides of the wings in this study. This confirms the findings of Ma et al. , where Axibility ,14. The xibility .A. mellifera. Research on the dragonfly Zenithoptera lanei generated the first evidence of a tracheal system in a flying wing for a modern adult insect [Schistocerca americana and confirmed the presence of hemolymph flow in all wing veins and determined the types of local flow patterns. These studies and others show that wing veins, containing hemolymph, trachea, and nerves, are not highly sclerotized in adult insect wings but allow for critical circulation.In this study, resilin deposition levels were maintained from the nest to the forager stage of workers in all joints studied. Historical data have indicated that post-ecdysis wings are highly sclerotized, dead cuticle . Howevert insect . Salcedot insect ,53 mappeA. mellifera wings may provide key insights into the biological functions of wings in adult insects and determine where resilin endocuticular deposition occurs and is maintained in adult insects.The circulation of hemolymph may be key in maintaining the levels of resilin deposition. Resilin needs to be hydrated by water because it only exhibits elasticity and deformability while in aqueous media ,54. WhenA. mellifera age class. This is the first study to demonstrate a relationship between resilin and A. mellifera biological age class and caste. The use of resilin as a marker of wing development, structural integrity, and potentially overall wing health should be investigated, especially in colonies affected by pathogens that affect wing development and morphology, such as deformed wing virus (DWV). DWV causes marked atrophy of wings during pupal development, resulting in honey bees that are unable to fly and are removed from the colony shortly after emerging [The expression and deposition of resilin is dependent upon emerging . DWV is emerging . Precociemerging ,57,58. P


Digital just-in-time adaptive interventions can reduce binge-drinking events  in young adults but need to be optimized for timing and content. Delivering just-in-time support messages in the hours prior to BDEs could improve intervention impact.future, that is, same-day BDEs 1 to 6 hours prior BDEs, using smartphone sensor data and to identify the most informative phone sensor features associated with BDEs on weekends and weekdays to determine the key features that explain prediction model performance.We aimed to determine the feasibility of developing a machine learning (ML) model to accurately predict We collected phone sensor data from 75 young adults  with risky drinking behavior who reported their drinking behavior over 14 weeks. The participants in this secondary analysis were enrolled in a clinical trial. We developed ML models testing different algorithms  to predict same-day BDEs (vs low-risk drinking events and non-drinking periods) using smartphone sensor data . We tested various “prediction distance” time windows  from drinking onset. We also tested various analysis time windows , ranging from 1 to 12 hours prior to drinking onset, because this determines the amount of data that needs to be stored on the phone to compute the model. Explainable artificial intelligence was used to explore interactions among the most informative phone sensor features contributing to the prediction of BDEs.F1-score=0.95 and 0.94, respectively). This XGBoost model needed 12 and 9 hours of phone sensor data at 3- and 6-hour prediction distance from the onset of drinking on weekends and weekdays, respectively, prior to predicting same-day BDEs. The most informative phone sensor features for BDE prediction were time  and GPS-derived features, such as the radius of gyration (an indicator of travel). Interactions among key features  contributed to the prediction of same-day BDEs.The XGBoost model performed the best in predicting imminent same-day BDEs, with 95% accuracy on weekends and 94.3% accuracy on weekdays  BDEs in young adults. The prediction model provides “windows of opportunity,” and with the adoption of explainable artificial intelligence, we identified “key contributing features” to trigger just-in-time adaptive intervention prior to the onset of BDEs, which has the potential to reduce the likelihood of BDEs in young adults.ClinicalTrials.gov NCT02918565; https://clinicaltrials.gov/ct2/show/NCT02918565 Binge-drinking events (BDEs), defined as consuming ≥4 drinks on a single occasion for women or ≥5 drinks on a single occasion for men , represent a common, risky drinking pattern, primarily seen among young adults . BDEs are associated with multiple adverse alcohol-related harms, such as traffic fatalities and violent behavior -6. Digitpredict future episodes of alcohol use.Timely intervention delivery depends on the ability to accurately predict the imminent  occurrence of a BDE. To date, most research has focused on the simpler problem of “detecting” an episode of alcohol use that has already started. For example, sensors  worn on the wrist or ankle that use biological sampling can “detect” a BDE ,10 and pSmartphone sensors provide potentially powerful tools for collecting continuous data that can be used to infer personal behaviors or activities associated with alcohol use with low burden and at low cost. The smartphone’s accelerometer has been used to detect or measure alcohol-related intoxication through gait analysis -15. In aTo date, most prediction models of imminent drinking and BDEs are based on self-report. For example, the A-CHESS smartphone app is a JIT adaptive intervention (JITAI), which used weekly check-ins to track alcohol-dependent patients’ recovery progress , with a Bayesian network prediction model of lapse risk  in the coming week that had 75% sensitivity and 88% specificity . Other sbefore they begin theoretically allows for a greater probability that the intervention material would be effective. A study that developed a prediction model of substance use craving using GPS developed a random forest model with 93% accuracy in predicting drug craving 90 minutes into the future, based mostly on predicting the absence of craving (owing to the low craving base rate)  and the University of Washington (STUDY00016480), under the grant number 1R21AA030153-01. A unique device ID was assigned to the participants. The sensor and SMS text message data were labeled with the assigned unique ID. The participants were told that they could drop out of the study at any time during the study. Researchers informed the participants that phone sensors needed to be enabled during the study. However, the participants recognized that the AWARE app and certain sensors are configurable, which means that, for example, GPS can be disabled manually by accessing the setting menu on their smartphones. To keep the sensor data confidential, the study data were deidentified, and personal identifying information  was stored separately from other study data in a secure  location.The participants provided informed consent before study participation, completed baseline questionnaires, and installed our AWARE data collection app  on theirWe developed our mobile data collection app based on the AWARE framework  to colleThe participants were asked to report alcohol consumption using SMS text messages twice weekly, on the 2 days of the week that they reported drinking the most in the baseline assessment . To measTo identify drinking events, we used the responses to SMS text message queries regarding the prior day’s alcohol use . Table 1To develop a model for predicting drinking events, we followed a standard ML pipeline approach: preprocessing data, preparing the data set, extracting and selecting features, and training and testing models.In total, 1168 events were reported by the 75 young adults: 729 non-drinking events , 236 low-risk drinking events , and 203 BDEs . We included the participants in our analysis if they reported at least 1 non-drinking at least 1 low-risk drinking or BDE and if there were at least 3 days with the minimum amount of phone sensor data needed for analysis. We excluded 414 events from the 75 participants (mean 5.52 events per person) because they did not provide GPS-based location data, and more than half of their key features were missing. Missing key phone sensor features might have occurred, for example, because the AWARE app allowed users to disable GPS collection. As such, we excluded days that were missing a few hours of sensor data , resulting in a final data set of 754 events from the 75 participants: BDEs , low-risk drinking events , and non-drinking events . For missing sensor values , we interpolated the average value between 2 adjacent sensor data points.w hours (where we varied w), we predicted whether the participant will have a BDE within d hours , we labeled the windows before and after the drinking event as non-drinking event. For a woman, windows during a drinking event were labeled as BDE if they consumed ≥4 standard drinks, and for a man, windows during a drinking event were labeled as BDE if they consumed ≥5 standard drinks. Otherwise, the windows were coded as low-risk drinking. More than 60%  of the BDEs reported started between 4 PM and 10 PM, as shown in We explored different prediction windows using 1-, 3-, and 6-hour distances prior to drinking onset to see how far in advance predictions could be made about BDEs . SimilarWe split our data into 2 nonoverlapping data sets to represent the randomly selected data set: training (80%) and testing (20%) data sets, which were split by rows. The training set was used to train ML algorithms and for model optimization. The trained model was then tested on the “unseen” data set. We used 10-fold cross-validation to identify the best model with our training data set. We report our final results on the 20% holdout test data set. To balance our data set (consisting of mostly non-drinking events), we oversampled the instances of the underrepresented classes using the synthetic minority over-sampling technique (SMOTE)  in our tBuilding on our previous work in detecting BDEs with smartphone data ,18, we eThe statistical features of physical activity and the magnitude (power) of body movement were extracted from the accelerometer. For location features, we chose features that have been shown to be associated with drinking behaviors : total tWe calculated the following from call logs and contacts: the total number and duration of incoming and outgoing calls, number of SMS text messages, and total number of contacts. We hypothesized that, for example, calls and SMS text messages  could be used to predict BDEs. We chose these features, as young adults are likely to communicate with their friends before drinking events .We used screen lock and unlock events and interaction times to understand how young adults use smartphones during a day. These phone features are a potentially powerful tool to infer whether certain types of mobile phone use associated with social communication relate to near-future alcohol use . For device use, we used app use, screen status, and battery charging and extracted the following details from the mobile app use data: the number of unique apps used, the duration of app use, the number of times the individual switched between apps, and the number of apps running in the foreground. We extracted the following from screen status data: the number of times the screen was turned on or off and the total duration of screen use. We extracted the following details from the data on battery charging: time when the battery was fully charged, the total duration of charging battery, and battery percentages.Finally, we extracted the number of unique Wi-Fi hotspots as an environmental feature.We built our prediction models by parsing the data for each person into 15-minute epochs. Each 15-minute epoch contained 70 sensor features and a drinking label . We then identified all sensor features from all the 15-minute epochs in the 24 hours before the onset of a drinking event as our primary period for predictive modeling. For example, if a person reported starting drinking at 9 PM on a day they self-reported a BDE, then we would include all sensor data from the 8:45 PM–to–9 PM epoch on the previous day to the 8:45 PM–to–9 PM epoch on the current day. For days when there was no reported drinking, we randomly selected a time between 6 PM and midnight and marked this observation as the “start of the non-drinking event.” There were missing observations in the data set, so not all labeled days had ninety-six 15-minute epochs of sensor data.As part of our modeling, we assessed the performance of the predictive models using a range of analysis window sizes  to compare the accuracies of the ML models. To test different analysis window sizes, the “prediction distance” window was held constant at “1 hour” prior to a drinking event. Note that the results indicated that a prediction window of 6 hours on weekdays and 3 hours on weekends prior to a drinking event had the best performance across analysis window sizes; therefore, the prediction windows of 6 hours on weekdays and 3 hours on weekends are reported here  for a 3-way classification task: non-drinking event (N) versus low-risk drinking event (D) versus BDE using Optuna . We alsoF1-score of the model. As part of hyperparameter optimization, we tried different values of XGBoost. Hyperparameter tuning uses a search strategy to identify an optimal set of parameters that maximizes the performance of a model. To evaluate and hyperparameter tune our models, we implemented 10-fold cross-validation and then min-max scaling for feature normalization and used oversampling on the training set with SMOTE to handle class imbalance. We ran 100 iterations of Optuna to find the best hyperparameter by optimizing the We chose the hyperparameter combination that achieved the best performance on the validation set for the final model that was then evaluated on the test set. To try multiple combinations of parameters, cross-validate each, and determine which values result in the best performance, we tested a variety of parameters, such as the model type, distance from the drinking events , the amount of data used , and hyperparameters. The analysis window size was particularly important, as it defined how much data were used to fit the model. For example, an analysis window size of 3 and a prediction distance  of 6 means that the hours ,18,49 beF1-score, κ, precision, and recall. To measure the performance of the models across different classes, we computed precision, recall, and F1-score. In addition, we built a baseline model using only the day of the week to test how much the phone sensor features improved model performance.To evaluate the performance of the models, we computed the following metrics to measure the overall performance: accuracy, F1-score to optimize the prediction model. The F1-score is a balanced measure of precision and recall. Second, we used the κ value if the F1-scores were the same. The κ value is a measure of the similarity between observations and predictions while correcting for agreement that happens by chance [κ value is also used to test performance for imbalanced classes and multiple classes.First, we considered the y chance . The κ vWe used the most represented XAI, SHAP , and PDPResults section, we present the SHAP feature importance bar plot to show the top features for BDE classification, SHAP summary plot to show how features impact BDE prediction based on the values that they take, and the SHAP dependence plots to show some of the features’ interaction with time of day and their impact on the BDE prediction on a more detailed level.First, we selected SHAP  because a test data set) of features and their impact on drinking predictions [Second, we selected PDPs , which ddictions . In thisF1-score, and κ: 93.9%, 0.94, and 0.84, respectively).Overall, we found that the XGBoost models generally outperformed the logistic regression, support vector machine, random forest, and decision tree models for a range of analysis window sizes. In F1-score, κ: 90.2%, 0.90, and 0.83, respectively).For the weekend prediction model , we founF1-scores for window sizes 1 to 12 hours for the weekdays  are shown in F1-score increases according to the increase in window size until it reaches the peak when the window size is 9. When the vertex is reached, the F1-score decreases with increase in window size. WDXGBoost-W9D1 refers to the weekday XGBoost model in which the analysis window size is 9 hours and prediction distance is 1 hour from drinking onset.The F1-scores for window sizes 1 to 12 hours for the weekends  are shown in F1-score increases according to the increase in window size. WEXGBoost-W12D1 refers to the weekend XGBoost model in which the analysis window size is 12 hours and prediction distance is 1 hour from drinking onset.The Next, we present the performance of different prediction distances, holding the analysis window size constant (weekday=9 hours and weekend=12 hours). In this section, we present the performance of the models with a range of prediction distances: 1, 3, and 6 hours from drinking events.F1-score for predicting BDEs  had the highest F1-score increases according to the increase in distance until it reaches the peak when the distance is 6 hours from drinking onset. WDXGBoost-W12D1 refers to the weekday XGBoost model in which the analysis window size is 9 hours and prediction distance is 6 hours from drinking onset.F1-score for predicting BDEs  had the highest F1-score increases according to the increase in distance until it reaches the peak when the distance is 3 hours from drinking onset. “WEXGBoost-W12D3” refers to the weekend XGBoost model in which the analysis window size is 12 hours and prediction distance is 3 hours from drinking onset.F1-score and accuracy for the prediction of drinking events overall on weekdays and weekends, respectively.In summary, the combination of the 9-hour and 12-hour analysis window sizes with a 6-hour and 3-hour prediction distances resulted in population models with the highest F1-score  for BDEs (vs low-risk drinking and non-drinking periods) on the validation set. This same model, which had the highest F1-score, also had the highest accuracy. Specifically, the weekday model had an F1-score (for BDE) of 0.94 on the test data set and a high average precision , recall , and F1-score . By contrast, the weekend model had an F1-score (for BDE) of 0.95 on the test data set and a high average precision , recall , and F1-score  latitude and longitude were the most important features, in addition to time of day contributing to the prediction of BDEs both on weekdays and weekends, and (2) radius of gyration is a relatively more impactful predictor of BDEs on weekends than on weekdays.Overall, From the summary plots on the left, we can see that acceleration features tend to be negatively related to the BDE prediction on both the weekends and weekdays, meaning that higher acceleration values decrease the probability of BDE prediction. This greater activity or movement of the smartphone is associated with a lower likelihood of BDEs. In addition, radius of gyration and number of locations are positively related to the BDE prediction .We present the interaction effect between key features: time of day with GPS-derived travel pattern or location. As shown in The radius of gyration followed similar patterns on both weekends and weekdays for values greater than approximately 1000 m; however, radius of gyration and day of week (weekday or weekend) had opposite patterns for values between 0 and 800 m contributing to BDEs on weekdays. When radius of gyration is between 0 and 800 m, SHAP values for weekdays follow an inverted u–curve peaking at around 500 m, whereas for weekends, the same radius of gyration range follows a u-curve  to localize the meaning of the values. Looking at the average latitude plots, we see that the patterns are inverse of each other for weekdays and weekends for the same latitudinal coordinates, which means that young adults visit certain areas before BDEs on the weekends but not on weekdays. This is highly likely owing to bars and other drinking spaces being in the vicinity of workplaces in the city.PDPs show the marginal effect that 1 or 2 features have on the ML model . In FiguFrom the plots, we can see that young adults are more likely to travel within a larger area (radius of gyration) before BDEs compared with non-drinking or low-risk drinking events on weekdays. By contrast, it is shown that a greater radius of gyration on weekends is negatively related to non-drinking positively related to low-risk drinking events and BDEs. However, specifically the range between 1223 and 10,952 m was positively related to BDEs. Furthermore, on weekends, the probability of not drinking alcohol declines as the radius of gyration increases, whereas on weekdays, it follows a more linear pattern with a slight increase in probability at higher radius of gyration.In This plot can be used as a map to display which visited areas increase the probability of BDEs on weekends versus weekdays. The horizontal axes represent the average latitude values, and the vertical axes represent the average longitude values. To preserve privacy and simplify the plot, rather than mapping out the exact locations of participants and the probability of BDEs at specific locations, we plotted their general locations (zip code level or neighborhood level) to give a bird’s eye view on their locations and calculate the probabilities in general areas rather than specific locations. The color spectrum represents the probability of the person starting a BDE within 6 to 15 hours later that day. From dark to bright colors, the probability of BDEs increases.From the weekend plot, we found that in certain geographic locations (represented in yellow in the plot), the probability of BDEs was 24%, higher than the expected probability of 21.4% (relative increase of 11% and absolute increase of 2.6%).As for the weekday plot, in the areas colored in yellow, the average probability of binge drinking behavior was 17%, higher than the expected probability of BDE of 11.9% in the entire data set, which indicates a 1.43× increase in BDE prevalence . Thus, it can be concluded that just by looking at the 15-minute average locations of young adults who reported hazardous alcohol use and were sampled from an ED in the Pittsburgh metropolitan area, we can make meaningful inferences about their probability of binge drinking later that day. With more fine-grained and personalized GPS data, it may be possible to increase the predictive power of location information.In summary, we found that a combination of time of day and GPS-derived travel and location features provided interpretable patterns of young adults who are likely to report same-day BDEs. Our findings not only optimized the ML model with “windows of opportunity” but also identified “key location features” to trigger JITAIs to support the design of intervention strategies and messages for BDE prevention among young adults.to investigate windows of opportunity to prevent BDEs. Our results confirm that smartphone sensors can be used to predict imminent BDE events before their onset using same-day behaviors and comparing them with the behaviors during non-drinking and low-risk drinking events on weekdays and weekends. The key contributing features we found are time of day, travel patterns and boundaries , and accelerations of body movements, which could be used for triggering JITAI before the onset of BDEs and thus preventing the potential risks of BDEs.We attempted to predict imminent same-day BDEs in young adults with the use of smartphone-based sensors. We used ML coupled with XAI detection of BDEs is feasible using phone sensor data [moving toward the prevention of BDEs, we focused on predicting imminent (same-day) BDEs before their onset. In this feasibility study, we demonstrated how to build an ML model using passively sensed smartphone sensor data, which predicted young adult drinking behaviors , with 94.3% accuracy for weekday drinking behavior (WDXGBoost-W9D6) and 95% accuracy for weekend drinking behavior (WEXGBoost-W12D3). The prediction model using phone sensor data for both weekday and weekend drinking had higher accuracy relative to a baseline model using only day of the week to predict drinking behavior (60.6% accuracy). We also found that 9 hours (for weekdays) and 12 hours (for weekends) of phone sensor data collected at a 6-hour and 3-hour prediction distance  maximized the F1-score  for predicting BDEs. The analysis window size is important for estimating data storage needs and potential privacy risks of storing sensitive data on the phone for prediction. This ML prediction model advances our prior work using phone sensors to “detect” episodes of alcohol use that have already started [As our previous work confirmed that the sor data ,18, movi started ,18 by alThere were similarities in the most important phone sensor features for the “detection” and “prediction” of drinking events. For example, time  was important for both the “detection” ,18 and pspent longer duration in important locations were more likely to report BDEs  and drinking events  later that day. Furthermore, the participants who interacted with their smartphones more  and had longer call durations were less likely to drink alcohol later in the day. These results suggest that smartphone interaction activity and communication during the day might indicate work, school, and social obligations and activities that could help to regulate or constrain drinking behavior. By contrast, young adults who had fewer communication interactions using the phone had an increased likelihood of drinking later that day. The low level of communication activity involving the phone in the hours before a drinking event may reflect a sense of social isolation, which is a direction to be examined in future research. The most important smartphone sensor features contributing to drinking behavior prediction may serve as early warning signals, which have parallels in the mental health literature, where, for example, certain signs and symptoms signal increases in depressive symptom severity [In predicting drinking events, we found that young adults who traveled more  and severity .Further analyses using XAI indicated differences in the importance of features in predicting BDEs in young adults on weekdays relative to weekends. For example, the radius of gyration was a more important predictor of BDEs on weekends than on weekdays. The relevance of GPS-derived travel data in relation to episodes of alcohol use is consistent with prior smartphone sensor work with young adults . Our useThe ability to predict upcoming drinking events 3 and 6 hours before their likely onset during weekends and weekdays, respectively, using only smartphone sensor data could support JIT intervention , potentifarther from the onset of drinking, enhancing the motivation to limit drinking, or planning alternative healthy activities might be prioritized. In contrast, once drinking starts , specific techniques to reduce the risk associated with a BDE could be provided  [The drinking prediction models presented in this paper, testing a range of distances from the onset of drinking , suggest temporal changes that are the most predictive of drinking on specific days (weekends vs weekdays), which are relevant to tailoring intervention content —for exam drinks) . We enviOur results suggest that an eventual BDE prediction and intervention app requires a certain amount of sensor data to be temporarily stored on one’s smartphone. The phone needs to store a window of 12 hours (WEXGBoost-W12D3) and 9 hours (WDXGBoost-W9D6) of data to run the prediction algorithm for BDEs on weekends and weekdays, respectively. Older data can be removed on an ongoing basis to maximize data privacy. Furthermore, although our current system runs on a server, the prediction model is efficient enough that it could be executed directly on a smartphone, with data never leaving the smartphone. Regarding privacy and ethical issues in the collection of phone sensor data, there is a need to carefully review the data collection process with the participants, including the collection of specific types of sensor data  and the granularity at which data will be collected , and provide them with examples of the variables that will be derived  so that they can evaluate the potential risks to privacy and provide informed consent for data collection. The voluntariness of data collection and ability to opt out of data collection at any time need to be made clear and discussed with the participants when obtaining their informed consent. The project’s app used reminders  to keep the participants informed of phone sensor data collection. Secure methods of data transmission and storage  were used but are subject to potential breach, a risk which the participants need to understand.It is challenging to balance privacy preservation and the accuracy of knowing an individual’s location and travel patterns. The collection and analysis of GPS data raise privacy issues because the participant’s location  might be identifiable. The deidentification and level of granularity of data must be considered in protecting the privacy and confidentiality of the participants. Interestingly, based on our supplementary analysis . Fourth, we successfully built a population model to predict non-drinking events, low-risk drinking events, and BDEs in young adults; however, there might be individual patterns that our population model cannot capture, which may limit the generalizability of our model. In future work, we will collect a larger data set per participant with the integration of other phone data  and investigate the use of more personalized individual models. Our analyses focused on calls (incoming or outgoing) and SMS text messages as the main form of communication rather than communication through social media apps. Personalized models could result in a more accurate prediction system at the individual level.As is the case for any predictive model, deployment will result in some false positive and false negative predictions of drinking or BDEs. The potential “cost” of a false positive is that an intervention may not only be delivered without need, bothering an individual when irrelevant and degrading credibility, but also may lead someone to consider drinking when they did not originally intend to drink. Alternatively, not delivering an intervention when an opportunity to do so occurs  could result in perceived program unreliability; therefore, this should also be taken into account when selecting the optimal model.The studied sample comprised young adults who screened positive for hazardous drinking and were recruited from an urban area in western Pennsylvania. As such, the findings may not be directly applicable to other age groups, those with lower levels of drinking, and those from rural areas with different activity patterns.We position our work as a feasibility study, as our study population was limited to young adults. As we would expect that both the phone use behaviors and the drinking behaviors of young adults will differ from those of older adults, we do not claim that our approach or results will be generalizable to a population with different demographics. Instead, we suggest that future work needs to replicate the data collection and analyses reported here on a study population with a wider range of demographics. This replication will determine how generalizable our results are, both with a larger and more heterogeneous young adult population and with an older adult population.The XAI-integrated BDE model we developed could allow clinicians to assist in designing JITAIs for preventing imminent BDEs and thus their potential negative consequences. The specific sensor features contributing to high model performance highlight the potential for interpreting young adults’ behaviors only using smartphones in an unobtrusive way. As such, it should enable clinicians to derive advanced strategies for JITAI for young adults to prevent the onset of a BDE.To the best of our knowledge, this is the first study to attempt to develop a prediction model of same-day imminent BDE events using smartphone sensor data with a maximum accuracy of 95% (weekend model), which outperformed a baseline model that used only day of the week to predict same-day drinking in young adults. The best-performing prediction model was the XGBoost, using a 9-hour and 12-hour analysis window size at a prediction distance of 6 and 3 hours before the onset of a BDE on weekdays and weekend, respectively. To improve interpretability of the BDE model, we applied XAI  to determine key features and their contributions. The “windows of opportunity” and the key contributors  can be used by clinicians for designing JIT messaging to prevent imminent BDEs in young adults.


Neurodegenerative diseases cause developmental delays and loss of milestones in infants and children. However, scalable outcome measures that quantify features meaningful to parents/caregivers (P/CGs) and have regulatory precedence are lacking for assessing the effectiveness of treatments in clinical trials of neurodegenerative disorders. To address this gap, we developed an innovative, blinded strategy for single-arm trials with external controls using expert panel review of home video.We identified meaningful, observable, and objective developmental milestones from iterative interviews with P/CGs and clinical experts. Subsequently, we standardized video recording procedures and instructions to ensure consistency in how P/CGs solicited each activity. In practice, videos would be graded by an expert panel blinded to treatment. To ensure blinding and quality control, video recordings from interim time points would be randomly interspersed. We conducted a pilot study and a pretest of grading to test feasibility and improve the final strategy.The five P/CGs participating in the pilot study found the instructions clear, selected activities important and reflective of their children’s abilities, and recordings at-home preferrable to in-clinic assessments. The three grading experts found the videos easy to grade and the milestones clinically meaningful.Our standardized strategy enables expert panel grading of developmental milestone achievements using at-home recordings, blinded to treatment and post-baseline time points. This rigorous and objective scoring system has broad applicability in various disease contexts, with or without external controls. Moreover, our strategy facilitates flexible, continued data collection and the videos can be archived for future analyses.The online version contains supplementary material available at 10.1007/s43441-023-00568-4. Neurodegenerative diseases stem from insidious damage to cells and connections in the nervous system that are essential for cognition, behavior, mobility, autonomic functions, and communication . NeurodeGiven the wide range of affected developmental domains in children with such neurodegenerative disorders, a major barrier to clinical trials is identification of endpoints that are meaningful and important to parents/caregivers (P/CGs) and clinicians. These endpoints must be scalable and feasible to administer over the duration of the clinical trials as children age, and ideally, minimally burdensome for the already-taxed families.There is no universally accepted endpoint that has regulatory precedence for these conditions, nor one that covers all the impacts of the condition in terms of the way the children feel, function, and survive. We developed an innovative, treatment-masked strategy to assess the achievements of developmental milestones, using expert panel grading of at-home video recordings. Standardization of this process involved minimizing bias and ensuring continuity of assessment over time when remote evaluation was a more feasible option, or if pandemic-related closures necessitated remote evaluation. In the current study, we detailed the development process of our strategy, originally designed for a single-arm clinical trial and parallel natural history study (NHS) in children with neurodegenerative diseases. However, our strategy serves as a versatile framework for future research, adaptable to diverse disease populations and contexts.At the time of clinical trial planning, there was no standard clinical outcome assessment (COA) for developmental evaluation of children with neurodegenerative conditions of interest. We identified standardized activities to video record through literature review of disease symptoms, iterative interviews with P/CGs and clinical experts, scrutiny of developmental milestones on standardized scales such as the Bayley Scales of Infant and Toddler Development-III (BSID-3)  and VineDevelopmental milestones were identified that were meaningful to P/CGs and clinical experts and that children with neurodegenerative diseases either never develop or lose as the disease progresses. These were grouped by domain . This resulted in defined skills across a wide range of ability within a domain that served as the foundation for observing and documenting standardized, discrete activities regardless of age or developmental status. This list was further refined via iterative discussions with clinicians and COA experts to only include milestones that were considered highly observable and objective. They were required to be: (1) easily captured on home video, (2) feasible to observe on video of sufficient quality, (3) easily elicited by P/CGs through basic interactions with their children, (4) objectively gradable by experts in terms of performance, and (5) contributory to experts’ global impression of change (GIC) since baseline. Slight adaptations were further made to certain milestones to make them more appropriate for children with developmental delays due to neurodegenerative disease .P/CGs prompted and recorded activities by their child in a standardized fashion using a smartphone. The P/CG training materials had a brief manual for the study’s corresponding smartphone app , which included training videos for each selected activity. A laminated quick reference guide outlining exactly what to do for each activity and providing tips for improving video quality  was also provided . Each of the six skillsets included several basic standardized instructions regarding camera distance, position of child, actions to take during the video, skills being looked for, and examples of how the P/CG could elicit these activities from their children , and interested P/CGs identified by a patient advocacy organization were invited to participate. Eligible participants had the early or late infantile type of a specific neurodegenerative disorder for the clinical trial, and met all of the prespecified inclusion/exclusion criteria for the child  and for the P/CG . Training materials, a tripod, and required toys (lighted rattle and blocks) were shipped to participants. Within one week of completing their video recordings, P/CGs were scheduled for a one-hour, semi-structured interview conducted over the telephone to collect feedback on the clarity of the training materials and discuss any challenges with the video recording process.In May 2021, we also conducted a pretest of the expert grading process with three child development expert clinicians to test the feasibility of using video to objectively observe and assess milestones, as well as the appropriateness and practicality of the standardized grading rubric. Before grading, the three experts were trained in a 45-min session that introduced the clinical trial, the video capture pilot study for P/CGs, and the process to follow for viewing videos (in ShareFile) and for grading .These experts were provided with a secure login to a reviewer-specific list of videos in randomized order that was auto-generated using Excel randomization commands. They were instructed to view videos submitted by the P/CGs of the cases in the pilot study in the pre-determined randomized order, side-by-side with the grading rubric, so that they could check off any milestone observed in each video. Items on the grading rubric corresponded to the activities elicited of children by their P/CGs, along with additional milestones that would be commonly observed on the videos. For example, P/CGs were instructed to prompt their children to stand up and walk as independently as possible; expert reviewers could then evaluate the activity shown on the video with corresponding milestones such as “raises self to standing position,” “stands up alone,” “walks alone,” and “walks alone with coordination.” Children could also receive credit for demonstrating skills specifically solicited in other sections of the rubric, such as “turns head to sides,” if observed during a different activity.Each item on the grading rubric equated to one point, and each item was scored in a dichotomous fashion (achieved/not achieved) based on the rater’s observations of the videos submitted. The total count of milestones achieved (without duplication of points for milestones demonstrated in multiple videos) for each child at each graded time point in a clinical trial would enable derivation of change in the count of milestones achieved longitudinally (vs. baseline). The average change in milestones achieved could be compared between groups .After watching all the videos, raters were then asked to view two videos that were obtained from one of the participants over the course of one week and provide their GIC to approximate the study’s planned GIC process. They were given 10 days to complete all grading. Within three to five days of completing grading, a trained interviewer conducted telephone interviews with each expert rater, during which they were probed on the grading process effectiveness and efficiency, the logistics of viewing the video and checking off milestones on the rubric, any difficulties with the overall grading process and instructions, and the milestones selected for scoring. Results from the pilot study and the pretest of expert grading facilitated improvements before the formal design of the strategy was finalized.Five P/CGs consented to participate and were enrolled in the pilot study. All P/CG participants were mothers between 25 and 37 years of age, and the majority were college-educated. There were two healthy control participants, three and 15 months old, and three case participants with the degenerative disease, aged 10, 17, and 43 months. A total of 55 videos were submitted by P/CGs in the pilot study. Some P/CGs submitted more than one video for a single activity. Only the 30 videos submitted by P/CGs of the three cases were graded by the expert panel.All videos were captured and submitted during a single week for each child, and all submitted activities met at least eight of the 10 quality criteria  and/or daily activities . Activities requiring their child to be mobile  were more difficult and often required another person to assist.“It was kind of frustrating for me [at the clinic] … because he was not displaying all the skills that he does at home when he was in that clinical setting.”“You can see them in their comfortable environment and the way they act a lot more.”“I think I would prefer having the doctor look at the videos because the child is going to feel safer and more comfortable at his or her own house.”There was unanimous response from P/CGs that being able to remotely show their child’s recorded milestone videos to the doctor was preferred over detailed assessments at a clinic visit, as their child would often get fatigued during clinic visits, act differently and display fewer skills at the doctor’s office compared to at-home. Excerpts from P/CG interviews supporting their appreciation of remote assessment included:In the pretest of grading procedures, the three experts took five to eight hours to review 30 videos, with an estimated five to 10 viewings per video. They reported the two-minute video length to be appropriate, and the quality sufficient for scoring purposes. In general, raters found the videos relatively easy to grade and felt that the developmental milestones were clinically meaningful, observable and objective, and appropriate to assess change during a clinical trial. They also felt the milestones could be reasonably captured through home-based videos and were similar to what they would expect in a clinic setting. Raters reported the following video characteristics as barriers to accurate grading: recordings captured from a side profile when full frontal was required, videos not showing if the child was looking at the P/CGs vs. another object, videos cut too short thereby not allowing sufficient time for the child to fully demonstrate a skill , and audio capture that was not quite clear enough to discriminate subtle vocalizations when grading communication-related milestones. Experts also identified the need for clearer definitions of what constituted “tolerates attention,” “nasal” vs. “throaty” sounds, and “persistent reach,” suggesting definitions based on BSID-3 to be added.In the context of this small pilot study, masking to time point and treatment was not feasible. Nevertheless, experts felt that they would be effectively masked to treatment and likely time, but not to patient identity in a larger study due to the fact that they had graded that patient before. Certain characteristics potentially interfering with limiting recall were identified . Overall, raters agreed that being masked to treatment status and randomizing hundreds of videos from dozens of children across various time points, with videos of the same child appearing at least 10 videos apart, would be feasible and effective at minimizing rater recall. Experts agreed that the objective rubric helped eliminate bias when grading the videos.for at least 5 s” . To further clarify correct steps, professionally recorded sample videos were commissioned for the main trial app for P/CGs to review as ideal examples before attempting their own recordings.Using the pilot and pretest results, we improved the clarity of the activities and milestones, grading rubric, and training materials. Overall, “head control” was separated into its own activity, standing and walking were combined into one video activity (but would be separately graded), crawling and rolling were combined into one video, and the list of milestones in each activity and rubric checklist was reorganized by the order they appear in BSID-3, corresponding to order of difficulty. Moreover, rater feedback also resulted in the addition, deletion, and combination of some milestones. For example, “elevates trunk while prone—shifts weight” was deleted, while “rolls from back to sides” was added. “Undifferentiated throaty sounds” and “undifferentiated nasal sounds” were combined into one milestone. The updated activities and milestones are shown in Supplementary Table 1. Instructions for P/CGs were modified to better enable milestone completion and assessment, i.e., “hold out item for child to reach The grading platform was designed to automatically attribute points for directly related milestones when a more advanced milestone was achieved Table . For exaLastly, we made changes to the video scripts and the laminated reference guides. For example, the laminated guides were updated to state explicitly that it is acceptable to exceed two minutes of recording, if needed. P/CG instructions were updated to encourage them to submit videos demonstrating a child’s best abilities, even if there were minor interruptions or background noises from other children.Identical and standardized at-home video recording instructions are planned for implementation in both the clinical trial and parallel NHS. Videos will be submitted to a secure storage platform via the smartphone app, which will automatically notify P/CGs of their allotted time period (one week or more in duration) for video capture of required activities and send reminders. If they choose not to submit a video for an activity, the reason for not submitting one would be prompted , and study staff will have the option to extend or reschedule the window for video capture if appropriate. Once quality is reviewed and found acceptable, videos will be anonymized and made available in the expert reviewer portal. Site staff could request re-recording of specific activities if needed.To allow tracking, randomization of videos, and masking to study  and time , each video in the trial and NHS will be assigned a unique sequence number. Expert panel grading sessions will be delayed until study accrual enables separating videos of the same child by a minimum of 10 videos to minimize recall. Expert raters will have the flexibility to view each child’s video recordings as many times as they prefer during their review. In addition to checking off milestones observed, experts will also view side-by-side video recordings at baseline and a follow-up time  to assess GIC using a nine-point scale ranging from extremely deteriorated to extremely improved.For quality control, plans are to randomly intersperse 25% to 30% of videos from an interim timepoint twice to allow assessment of intra-rater reliability and to monitor raters for drift in grading over time  released a draft guidance on rare diseases emphasizing the need to “standardize the collection and handling of data to ensure quality and interpretability” and that “standardized operating procedures and quality assurance and quality control are essential” . Given tWhile there is minimal, if any, precedence for home videography as the basis for assessing effectiveness to gain approval for products, there is a great deal of precedence in the use of photographic and imaging techniques in drug development. This includes products using automated or expert measurement or grading of 2-D and 3-D photography or imaging, such as static head or bald spot images in alopecia , 10, skiRemote video assessments also have several disease-agnostic benefits for clinical trials and rare disease follow-up in neurodiverse populations. They offer increased flexibility for continued data collection in circumstances of interrupted visit schedules or decentralized trials aiming to minimize site visits . In lighOur video review and grading procedures allow the generation of rigorous, objective scoring by masking raters to study/treatment and to post-baseline time since treatment in the trial or since enrollment in the NHS. Expert feedback suggested that the masking efforts planned for the study were the most appropriate and effective approach to masking for such video review. Natural aging and change in appearance of children over time may further limit the ability to recall status of specific children over time and may further enhance the ability to mask to child identity. Nonetheless, it is worth noting that the success of randomized order and effective masking hinges on the availability of a large number of videos from both the trial and an NHS, which may be challenging in the context of rare disease if video recordings are captured infrequently.Our strategy can also be applied to studies outside of the United States. Even though not tested in the pilot study, local developmental experts or study staff could provide notes for activities documenting culture-specific gestures, vocalizations, and words in videos collected from families not speaking English, so that a centralized expert panel could review.Incorporating input from P/CGs and clinicians, we designed a standardized COA strategy using at-home video recordings that can be tailored to other diseases. Meaningful outcomes can be gleaned from expert grading of standardized at-home video recordings to assess the effectiveness of therapy. By implementing such procedures in both a clinical trial and a parallel NHS, treatment can be masked even for a single-arm, open-label clinical trial.Supplementary file1 (DOCX 21 KB)Supplementary file2 (PDF 160 KB)Below is the link to the electronic supplementary material.