N = 108 participants, 5492 observations; Study 2, N = 218 participants, 9994 observations). Study 2 presents a close replication with a fully remote delivery during the COVID-19 pandemic. During weeks when they received twice-a-day intervention reminders, individuals in the distancing interventions reported drinking less frequently than on control weeks—directionally in Study 1, and significantly in Study 2. Intervention reminders reduced drinking frequency but did not impact amount. We find that smartphone-based mindfulness and perspective-taking interventions, aimed to create psychological distance, can change behavior. This approach requires repeated reminders, which can be delivered via smartphones.Modifying behaviors, such as alcohol consumption, is difficult. Creating psychological distance between unhealthy triggers and one’s present experience can encourage change. Using two multisite, randomized experiments, we examine whether theory-driven strategies to create psychological distance—mindfulness and perspective-taking—can change drinking behaviors among young adults without alcohol dependence via a 28-day smartphone intervention (Study 1,  Creating psychological distance between unhealthy triggers and a person’s present experience—temporally, spatially, or socially,—may be an effective way to change behavior2. For example, creating ‘space’ from alcohol3, cigarette cues4, and unhealthy foods5 motivates healthier short-term choices in laboratory settings. Yet in everyday life, unhealthy triggers are abundant7.What tools can help people to create distance from unhealthy triggers and pursue healthier options as they go about their lives? A growing body of research highlights the promise of smartphone-delivered health reminders8. Since smartphones are ubiquitous and often with people as they go about daily lives, they offer important opportunities to test how to integrate theory-driven strategies from the laboratory into real-world settings9. Extending prior work, we focused on two popular psychological distancing strategies—mindfulness10 and perspective-taking11 to test the feasibility of smartphone reminders to change alcohol use behaviors among young adults. We deployed two multisite, randomized experiments: one prior to and another during the COVID-19 pandemic. The first experiment provided a proof concept; the second tested for scalability with a larger sample and without an in-person intervention component.Behaviors like alcohol use, smoking, and unhealthy eating are leading contributors to preventable disease and morbidity1 that often holds an integral role in social interactions, particularly among young adults12. In the United States, alcohol use rates are some of the highest during the early twenties13 and drinking is disproportionately pervasive on college campuses where 80% of students report consuming alcohol14 as opposed to 64.8% in the general public15. Although alcohol often forms a common part of the college experience, with expectations that social drinking will lead to positive social experiences16 and bonding17, drinking in excess may also be detrimental18. Alcohol may increase behaviors that put young adults at risk for infectious and chronic diseases, as well as their communities through productivity losses, anti-social behavior, violence, and accidents1. Unhealthy drinking habits often develop in college12 and pose risks for future alcohol dependence and associated health problems19. Accordingly, we sought to identify preventative strategies that aim to counter, or divert, drinking behaviors among college students. In this vein, we developed two interventions that draw on psychological strategies—mindfulness and perspective-taking—as two proof of concept ways to create distance from alcohol cues and promote behavior change.Alcohol use is a prevalent behavior20. Common mindfulness paradigms train individuals to take a step back and to accept their thoughts and feelings towards a trigger without judgment or avoidance. Theoretically, taking a step back may re-orient attention away from reactive thought patterns and dissuade unhealthy choices, possibly by de-automatizing responses to triggers21. Meta-analyses suggest that mindfulness training can be a promising component of substance use interventions23 across alcohol24, tobacco25, and cannabis and opiate use27, particularly within laboratory contexts and among clinical populations29. However, less work has leveraged mindfulness-based strategies as preventative efforts within more ecologically valid contexts, such as throughout daily life, and among non-clinical populations, such as healthy young adults. Among at-risk young adults, mindfulness interventions have produced mixed effects at reducing drinking35. Some studies suggest that a brief, in-person mindfulness training can decrease short-term consumption37. Others call for more frequent and higher intervention doses outside the laboratory31. Although distancing-based mindfulness strategies may be a promising path to behavior change, more research is needed on how to effectively incorporate these strategies into the daily lives of young adults, to possibly harness their preventative benefits38.The first popular strategy, mindfulness, involves creating space between a stimulus  and a person’s natural reaction to it11. Grounded in social learning theory39, perspective-taking paradigms typically instruct individuals to imagine how a target person may think, feel, or behave in a given situation40. Imagining how a target would react, for example in response to an alcohol cue, can create distance by re-orienting attention away from a person’s immediate reaction to alcohol and towards the desired target’s response. Separate bodies of work support this view. First, neuroimaging evidence suggests that adopting another target’s perspective can shift neural responses to parallel the target’s affective experience41. Second, behavioral work suggests that making salient the health behaviors of relevant targets, such as those of health conscious peers, can encourage the adoption of healthy behaviors42, possibly by calling into mind the subjective value of peers’ health habits43 or by increasing perceived self-other overlap44. These separate literatures motivate a test of whether approaching alcohol cues from the perspective of a low-drinking peer target, with the goal to create distance from alcohol cues, can change drinking behaviors. In line with this possibility, correlational research suggests that individuals who score higher (vs. lower) on perspective-taking report lower susceptibility to peer drinking influences45 and lower drinking46. Importantly, this research is cross-sectional and did not consider real peers as perspective-taking targets, nor their drinking behavior. Extending prior work, we suggest that experimentally prompting individuals to take the perspective of peers whom they perceive to drink less than themselves may offer an opportunity to create psychological distance from alcohol cues and help change drinking behaviors in day-to-day life.A second strategy to create psychological distance involves perspective-taking7. Indeed, current data is mixed, with some studies showing that brief psychological distancing trainings can motivate healthier choices37, and others suggesting that such trainings require boosts from ongoing reminders31. To address the possibility that repeated exposure is critical for the effectiveness of health communications in daily life47, we conducted randomized experiments using ecological momentary assessment (EMA), which collects intensive repeated measures in everyday settings48. We developed mindfulness and perspective-taking instructions designed to create psychological distance from alcohol cues and integrated them in the everyday lives of college students via smartphones. This approach allowed us to embed an experimental intervention while preserving a degree of ecological validity.Overall, the evidence reviewed above suggests that mindfulness and perspective-taking strategies may reduce drinking by promoting different types of psychological distance. However, less is known about how to effectively integrate these strategies into individuals’ natural environments where pro-drinking influences may be present49. Consistent with prior work, we modeled the frequency of drinking occasions and the amount consumed per occasion, as two separate outcomes, as both are independently linked to drinking habits and related health outcomes50. Our study focused on effects of psychological distancing reminders in general, and we had no a priori hypothesis about the relative effectiveness of the mindfulness versus perspective-taking strategies. We hypothesized that the frequency of alcohol occasions and the number of drinks per occasion would decrease on weeks when individuals received active intervention reminders versus control reminders. Further, we sought to replicate the effects of psychological distancing reminders on alcohol use frequency and amount with a larger sample and through a more scalable approach. Therefore, we conducted a second experiment that used fully remote delivery, which we refer to as Study 2. Study 1 provided a proof of concept during a regular time of in-person instruction when college students were on campus, prior to the COVID-19 pandemic. In turn, Study 2 presented a close replication study, conducted during the onset of the COVID-19 pandemic and throughout 2020.We randomly assigned participants to undergo a brief, in-person mindfulness, perspective-taking, or control training on how to respond to alcohol cues in their everyday life. Next, they received two smartphone reminders according to their respective condition and reported their alcohol use twice daily over the course of 28 days. We manipulated whether participants received psychological distance smartphone reminders (either mindfulness or perspective-taking) vs. control reminders, within-person over four alternating weeks, and examined changes in drinking frequency—the likelihood of having a drinking occasion, and drinking amount—the number of drinks per occasionM = 91.02%, SD = 12.72). In total, we collected 5492 data points on alcohol use. To measure the likelihood of a drinking occasion per assessment, participants responded to: “Since your EVENING/ MORNING survey, have you consumed any alcohol? (“No” or “Yes”). Participants who responded “Yes”, were asked to enter the number of standard servings of beer, liquor, and wine consumed:“Since the evening survey, how many standard servings (12 fl oz) of BEER/ (5 fl oz) of WINE /(1.5 fl oz) of LIQUOR have you had?”, using a numeric entry. To capture the total number of drinks per occasion, we summed responses for each beverage per assessment. See Refs.52 for previous use of these scales to capture alcohol use during daily life and the Measures section of the Methods for more details. These measures were also adapted from Ref.53 for twice-a-day morning and evening in situ assessment; with the original items previously validated against transdermal monitoring53. In Study 1, the average participant reported having an alcohol use occasion on 15.36% of all assessments and having 3.12 (SD = 1.70) drinks per occasion. Six participants (~ 6%) reported no alcohol use throughout the intervention period.In Study 1, participants were highly compliant with the protocol responding to a median of 94.64% of 56 EMA alcohol surveys sent to each person over the 28 days . In total, we collected 9994 data points on alcohol use. The average participant reported having alcohol on approximately 10.12% of all assessments and having approximately 2.00 (SD = 1.13) drinks per occasion. Seventy-eight participants (~ 35%) reported no alcohol throughout the intervention period, which mainly took place after students left campus during the COVID-19 pandemic. See Table In Study 2, we made use of the same alcohol use measures as in Study 1. In Study 2, participants were highly compliant with the protocol responding to a median of 91.07% of 56 EMA alcohol surveys sent  = 0.64, p = 0.526; Study 2: F = 0.22, p = 0.801. Participants in the mindfulness and perspective-taking conditions responded to more alcohol surveys on inactive vs. active weeks  vs. active weeks , t(70) = −2.99, p = 0.024; Study 2: inactive weeks  vs. active weeks , t(146) =—3.47, p =  < 0.001). To account for these differences in response rates on active versus inactive weeks, we controlled for the number of responses to survey prompts in analyses . We observed no group differences in age, gender and race  = 0.98, p = 0.376; gender: X2 = 2.75, p = 0.685; race: X2 = 3.06, p = 0.9307); Study 2 age: F < 0.01, p = 1; gender: X2  = 0.98, p = 0.613; race: X2 = 3.76, p = 0.878). With respect to differences in baseline drinking frequency, in Study 1, the perspective-taking group  reported fewer alcohol occasions in the month prior to the intervention relative the control group , highlighting a failure of randomization to create equal groups. We observed no other differences in baseline drinking frequency across other group comparisons , and no group differences in the number of drinks in the month prior to the intervention  = 0.508, p = 0.6033) in Study 1. See Table Next, we report group differences on key variables across both studies. Response rates during the intervention protocol did not vary by condition assignment: Study 1: F, 105 = 050, we separately examined two outcomes: frequency of drinking occasions and number of drinks per occasion . We report effects on drinking occasion frequency first and drinking amount second. The results are visualized in Fig. We tested the feasibility of psychological distancing reminders to reduce the number of drinking occasions and drinks per occasion on active, intervention weeks versus inactive, non-intervention weeks. To assess differences in drinking on active versus inactive weeks, we specified two separate multilevel hurdle models, one for Study 1 and another for Study 2. The main predictor was active week (vs. inactive week). Consistent with prior workp = 0.041; Table  p = 0.082). We found significant main effects in Study 2, such that participants in the mindfulness and perspective-taking conditions were significantly less likely to drink on active weeks (following active reminders) relative to inactive weeks (following control reminders), we found a directional main effect of the intervention reminders such that participants in the mindfulness and perspective-taking conditions were less likely to drink on active weeks (following active reminders) relative to inactive weeks (following control reminders) , we found no difference in the number of drinks consumed on alcohol use occasions on active weeks (following active intervention reminders) versus on inactive weeks (following control reminders) across both studies  week and distancing strategy in each multilevel hurdle model, for Study 1 and Study 2 separately. We found no significant interaction between active (vs. inactive) week and mindfulness (vs. perspective-taking) on drinking frequency  and in Study 2 , we found that individuals in the intervention groups showed a greater decrease in drinking frequency from inactive weeks to active weeks relative to the control group, both in Study 1 . The perspective-taking group reported fewer drinks per occasion relative to the control group in Study 1. However, we did not replicate these effects in Study 2 (see Tables S4 and S6). We are cautious to make strong claims about between-person effects given the possibility that random assignment did not fully overcome baseline differences between groups. In Study 1, we observed that the perspective-taking group reported drinking less frequently at baseline relative to the control group. Since we did not collect baseline drinking information for participants in Study 2, we cannot account for potential pre-existing differences between groups in Study 2 54.On average, across both studies, the frequency of drinking occasions decreased about 2%, on weeks when participants received twice-a-day psychological distancing reminders. In Study 1, drinking occasion frequency decreased from once every 6 days to once every 8 days when intervention reminders were present. In Study 2, drinking occasion frequency reduced from once every 9 days to once every 11 days. These effects are meaningful in light of broader goals to motivate positive behavior change in daily life from a preventative lens. Decreasing the frequency of alcohol occasions is not the direct goal of most alcohol harm reduction interventions among young adults, which often target binge drinking22, and the use of smartphones to promote behavior change56. Prior work has shown mixed effectiveness of smartphone interventions to change drinking behaviors, particularly among young adults57. Our two experiments extend the feasibility of theory-driven reminders that promote two kinds of psychological distance– mindfulness and perspective-taking– among college students without alcohol dependence. Consistent with prior work, our psychological distancing intervention effects on drinking frequency, may, to some degree, be contingent on repeated reminders. For example, Witkiewitz et al., found that randomizing college students into a 14-day smartphone-based intervention did not reduce drinking. However, among participants randomized to the smartphone-administered intervention, receiving more (vs. less) frequent personalized feedback modules, was associated with a lower likelihood of any drinking throughout the 14-day assessment31.Our investigation builds on bodies of research that consider psychological distancing strategies, such as mindfulness58, particularly in key moments when individuals encounter alcohol cues in day-to-day-life. For example, distancing reminders may make healthier options more salient, by creating mental space from automatic reactions to triggers59, or by moving attention towards a healthier peer’s perspective. Another possibility, drawing on the alcohol cognitive bias modification literature61, is that distancing reminders may help de-automatize cognitive biases to alcohol cues encountered in daily life. For instance, social drinkers62 may allocate quicker attention to, and make more positive subjective evaluations63 of alcohol  cues. Cognitive debiasing tools, such as psychological distancing, may help counter attentional and subjective evaluation biases and downregulate reactivity when approaching alcohol cues64. Similar to our findings, research that leverages cognitive bias modification to shift attentional bias away from alcohol also suggests little success to reduce drinking over a single training session65, but more encouraging results over repeated retraining66. Further, distancing reminders may also work by helping individuals notice when there is a need to regulate and/or how they can go about doing so, more broadly67.One possibility is that repeated psychological distance reminders help offload cognitive effort when evaluating health-related options69. However, future research that investigates these parallels in the drinking intervention context70 can help clarify the mechanisms through which psychological distancing strategies facilitate behavior change.With respect to intervention specificity, both the mindfulness and perspective-taking reminders reduced drinking frequency on active vs. inactive weeks to a similar degree. It is possible that different types of psychological distancing strategies, with repeated reminders, may be similarly effective in reducing the frequency of drinking occasions relative to an individual’s own baseline. Further, it is plausible that both interventions leverage a parallel psychological distance mechanism, though our data do not speak to this possibility directly. For example, neuroimaging research suggests that different forms of psychological distance are encoded similarly in the brain34. As such, future replication work may help isolate possible reactivity effects and build greater confidence in the distancing interventions.While promising, our intervention effects on reducing the frequency of drinking occasions should be interpreted with caution. We observed directional (Study 1) and significant (Study 2) within-person changes in drinking frequency from active to inactive weeks; with this change being greater than in the absence of active intervention reminders. However, participants in the intervention conditions did not necessarily drink less frequently than those assigned to the control condition, on average. One possibility is that random assignment did not completely overcome pre-existing differences in alcohol use between groups . There also remains the possibility that the distancing reminders altered drinking by making individuals more reactive to being monitored, though participants also received instructions about how to approach alcohol and reported their drinking behavior daily during “inactive” weeks. Thus, our comparison likely represents a relatively conservative test of potential effects of distancing reminders on drinking behavior. Nevertheless, self-monitoring alcohol use may promote behavior changeOur results should also be interpreted considering design strengths and limitations. First, we employed theory-driven interventions within an EMA design, which allowed us to capture intervention effects in participants' natural environments and overcome common retrospective biases that ask participants to recall information about longer periods of time . Next, the within-person manipulation allowed us to detect intervention effects while considering individuals’ personal drinking baselines. We employed this repeated measures design across two different samples, and across two different sites, thereby increasing the robustness of our findings. Further, we replicated these findings in a highly unusual and stressful time, the onset of the COVID-19 pandemic. Although intensive assessment often raises data compliance concerns, we observed little evidence of non-compliance in our data. The intensive sampling approach produced high response rates . These high response rates may support the feasibility of applying this approach for evaluating future interventions among college samples.71. To reduce concerns over self-reported drinking, future research could incorporate measures of passive transdermal sensors72, or blood alcohol content reports via smartphones73.It is important to note that our data cannot speak to long-term intervention effects on drinking beyond the length of 28 days, and our results may not generalize to samples beyond college students in urban college campuses in the Northeastern United States without alcohol dependence who are part of social groups. Specifically, we recruited students from pre-existing social groups (and included groups in which 80% or more expressed interest), with higher proportions of women than men. Although we controlled for non-independence of observations in statistical modeling, the non-independence may have confounded intervention effects in unmeasurable ways . Finally, when interpreting results, it is important to note that the psychological distancing reminders did not explicitly instruct individuals to decrease the quantity of alcohol consumed. As such, future work could test more explicit instructions to directly target the number of drinks when drinking. Finally, we used self-reported measures of drinking which may be subject to social desirability bias and may underestimate drinking levels74. The two mobile-health interventions tested here contribute to a growing literature on how to leverage mobile technology to administer psychological behavior change interventions among young adults. In two multi-site, randomized experiments we found that psychological distance-based reminders—drawing on mindfulness and perspective-taking—and delivered via smartphones, helped decrease the frequency of drinking occasions over the course of a month, both prior and during the onset of the COVID-19 pandemic. Intervention reminders specifically helped reduce drinking frequency but had little impact on the amount consumed per occasion. Future research may expand the use of psychological-distancing interventions employed via smartphones to motivate behavior change outside the lab.The present study responds to calls to develop more effective, theoretically-guided behavior change interventions75); and $55 for completing a 28-day EMA protocol with at least 70% response rate, compensated as Amazon gift card or cash card as per participant preference. In Study 2, participants were compensated $30 for completing an online survey and $60 for completing a 28-day EMA protocol via Amazon gift cards. Online surveys were conducted via Qualtrics, in-person tasks were presented using PsychoPy2, and the EMA prompts and participants’ responses were delivered via the LifeData app (www.lifedatacorp.com).We use data from two different cohorts of college students from the Social Health Impact of Network Effects (SHINE) Study which we refer to as Study 1 and Study 2. Details of recruitment, study design, and data analysis can be found in Supplement A. All research, methods, and study protocols were approved by the Human Subjects Electronic Research Application (HSERA) Institutional Review Board (IRB) at the University of Pennsylvania and were acknowledged by the Human Research Protection Office of the Department of Defense. All research, methods, and study protocols were conducted in accordance with the Human Subjects Institutional Review Board (IRB) at the University of Pennsylvania and the Human Research Protection Office of the Department of Defense. All participants provided informed consent before taking part in the study and were financially compensated. In Study 1, participants were compensated up to a total of $145. Compensation consisted of a $20 Amazon gift card to complete an online baseline survey; a $20 bonus Amazon gift card payment for at least 80% of the social group completing the survey; $50 for an in-person intervention training session, including an MRI scan session as part of a larger study (see Ref.75). However, recruitment was interrupted due to the COVID-19 pandemic, resulting in a final sample n = 108 when the study was closed for COVID reasons. For Study 2, our sample size was limited by the number of student groups originally recruited for Study 1 as part of a larger project (see Ref. 75), and all individuals who wished to enroll were included, yielding in a final sample of 218 individuals. See Supplementary Methods and Materials A for detailed information on participant enrollment and retention by condition.The target sample size for Study 1 (N = 240) was determined based on a power analysis accompanying an MRI session in the original grant application  and identified as white (53.7%), Asian (29.6%), Hispanic/Latino (4.6%), African American/Black (1.9%), and multiracial/other (10.2%). Study 2 sample comprised 218 individuals  aged between 18 and 42 years  recruited across two college campuses. Participants in Study 2 identified as white (37.2%), Asian (31.7%), African American/Black (7.8%), Hispanic/Latino (5.0%), multiracial/other (12.8%), and missing information (5.5%). In both studies, we recruited undergraduate students who were part of on-campus social groups , with more than 80% of the group expressing interest in participation. See more information on eligibility criteria in the ‘Procedure’ section and in the Recruitment section of Supplementary Methods and Materials B. Diagrams of enrollment and retention can be found in Supplement A, Figs. S1 and S2.Study 1 sample comprised 108 individuals  recruited across two urban college campuses in the Northeastern United States. Participants were aged between 18 and 28 years (75). Data collection for Study 2 replicated Study 1 but was modified to be completed fully remotely. In both studies, participants first completed an online, one hour long Qualtrics survey containing psychological, demographic and social network questionnaires on peer drinking behaviors, and other measures beyond the scope of the present report. Next, a subset of eligible participants were invited to attend an in-person (Study 1) session or to complete a virtual (Study 2) training session. In Study 1, eligibility for the intervention was determined by two components: a) the overall response completion rate of the social group and b) individual responses to questions in the baseline survey. Social groups were eligible to have their members invited to the intervention protocol if more than 20% of the group members completed the survey. Based on these criteria, 24 social groups were eligible. Of these groups, individuals were invited to complete the intervention protocol if they were: 18 years or older, had no history of serious medical issues, psychiatric hospitalization, or substance use disorders; reported drinking alcohol more than once a year; listed at least two people in their social group who drank the least in the group apart from themselves; were not studying abroad at the time, and reported being free from MRI contraindications including weighing less than 350 lbs; not claustrophobic, and not pregnant (See refs.77 for more information on an MRI session as part of the parent project). In turn, eligibility for Study 2 was open to any existing and new group members who completed any aspects of the broader project protocol. In Study 2, we excluded all individuals who had previously participated in the first intervention cohort. See Supplementary Methods and Materials B for more details.More detailed study procedures can be found in the project protocol . When participants responded “No” to having consumed alcohol, they answered questions about physical activity, caffeine use, and water consumption. These questions were matched for length with the follow-up alcohol questions to reduce the possibility that participants would report no alcohol use in order to minimize survey completion time. To select low drinking peer targets for participants randomized in the perspective-taking condition, we used peer nominations from the baseline social network questionnaire in response to: “Who in your group drinks the least?”. In cases when individuals nominated more than one peer who drinks less than them, targets were selected by picking a peer who was also nominated as close (“Who are you closest to?”) and who reported the highest drinking difference from the individual.We used participant reports of drinking during the 28-day ecological momentary assessment period. Participants were asked: “Since your EVENING/MORNING survey, have you consumed any alcohol? (“No” or “Yes” response option). Participants who responded “Yes”, were asked to enter the number of standard servings of beer, liquor, and wine consumed since the previous survey using a numeric entry. Responses for each beverage category were summed to obtain the total servings of alcohol consumed for each assessment  were winsorized to the next largest value—16 drinks per occasion. This step applied to 7 signals across 3 individuals out of 5492 observations in total. No outliers were observed in Study 2.78 using glmmTMB79 in R (Version 4.0.3) using the RStudio interface (Version 1.3.1093).To account for the nature of the alcohol use data which are often positively skewed and include many observations at zero, we used multilevel hurdle models. Hurdle models include a logistic regression to model the zeroes in the data as well as a count regression  to model the counts. All the zeroes  are modeled with the logistic regression and nonzero-counts  are modeled by a truncated negative binomial . Thus, these models allowed us to independently model whether a person drinks or not (logistic regression) at a given occasion and number of drinks when an individual drinks (count regression). We estimated hurdle models80; participant response rates, given observed differences in response rates on active versus inactive weeks; and signal count given overall decreases in drinking over time throughout the study period. Additional models controlling for demographic covariates are presented in the Supplement C (Table We specified two models (one for Study 1 and another for Study 2) to assess whether drinking frequency and amount vary among participants in the mindfulness and perspective-taking conditions (within-person) on weeks when intervention reminders were present or absent. The main predictor of interest was active vs. inactive week and the outcomes were frequency of drinking occasions and drinking amount. To further explore if the effectiveness of the reminders varied by condition type (mindfulness vs. perspective-taking), we included an interaction term: active week (vs. inactive week) x condition type. Given that observations are nested within participants, who are in turn nested within non-independent social groups, all multilevel models accommodated the nested nature of the data, and intercepts were allowed to vary randomly across people. In models presented in the main manuscript, we controlled for the following covariates: social weekend , due to differences between weekend and weekday drinking among college students C Table . We obseTo explore whether the main findings, i.e., changes in drinking occasion frequency from active to inactive weeks differed from non intervention-related changes, we compared drinking frequency change scores among participants in the interventions and the control condition. For each person, we calculated a change score that captures differences in the average proportion of drinking occasions on active intervention weeks versus inactive weeks throughout the 28-day study period. First, we first calculated the average proportion of drinking occasions on active weeks and inactive weeks, separately, by dividing the number of drinks reported throughout the active and inactive intervention period by an individual’s number of responses to alcohol reminders during the same time period. Next, we subtracted the proportion of drinking occasions on active weeks from the proportion of drinking occasions on inactive weeks.To obtain change scores for participants in the control condition, who did not undergo a within-person active-to-inactive week manipulation as part of the protocol, we randomly assigned two of the four weeks in the study period as pseudo “active” and the remaining two as pseudo “inactive” weeks. To counterbalance week order, as done in the active treatment groups, we randomly split half of the participants in an ABAB design, or a BABA design, which allowed us to match the two intervention designs as closely as possible. We repeated this random assignment 100 times to stabilize the estimates. Next, we averaged the change scores in drinking frequency (from pseudo-inactive to pseudo-active weeks) for the control participants across all iterations and compared the control group’s pseudo change scores to the intervention groups’ change scores. We first checked whether the proportion of drinking occasions were significantly non-Gaussian using the Shapiro Wilk test of normality and chose to perform non-parametric Mann–Whitney–Wilcoxon tests instead of t-tests because of nonnormality. We performed this comparison using unpaired two-sided Wilcoxon tests.Mindful that our identities can influence our approach to science, the authors wish to provide the reader with information about our backgrounds. With respect to gender, when the manuscript was drafted, nine authors self-identified as women, five as men, and one as non-binary. With respect to race, 13 authors self-identified as White, one as Asian, and one as Black. With respect to engagement with college students, when this study was conducted, two were doctoral students who teach and/or mentor other students, one was a research coodinator who mentors other students, five were postdoctoral researchers or research scientists who teach and/or mentor students, and seven were professors who teach and/or mentor students.Supplementary Information.


BackgroundMany residency programs do not accept the Comprehensive Osteopathic Medical Licensing Examination (COMLEX-USA) alone for osteopathic applicants. Furthermore, among those programs that do accept the COMLEX-USA, it is unknown how programs scale their minimum COMLEX-USA scores compared to their minimum United States Medical Licensing Examination (USMLE) scores.ObjectiveOur objective was to examine the variation of relative within-program differences between minimum USMLE Step and COMLEX-USA Level scores required for consideration by United States residency programs.MethodsWe performed a cross-sectional analysis of the Fellowship and Residency Electronic Interactive Database Access (FREIDA) database from April 2023, including the 10 specialties with the most training spots in 2022. These specialties were internal medicine, family medicine, pediatrics, emergency medicine, psychiatry, surgery, anesthesiology, obstetrics-gynecology, orthopedic surgery, and neurology. Within-program differences were calculated by subtracting the minimum USMLE Step 1 and 2 scores from the converted minimum USMLE Step 1 and 2 scores calculated from the minimum COMLEX-USA Level 1 and 2 scores using two conversion tools. We present differences as medians with interquartile ranges (IQR). Additionally, we report the proportion of programs with greater than 10-point differences for each step (1 and 2).ResultsOf the 3,364 accredited programs from the examined specialties, we included 1,477 in the Step 1 analysis and 1,227 in the Step 2 analysis with complete data. The median within-program difference between the minimum Step 1 score and the predicted Step 1 score was 12.0  using the Barnum and colleagues’ conversion tool and -1.7  using the Smith and colleagues’ tool. The median differences for Step 2 were 2.0  and -6.5  for each tool, respectively. Using the Barnum and Smith conversion tools, 937 (63%) and 435 (29%) programs had a greater than 10-point Step 1 score difference, respectively. Similarly, for Step 2, 564 (46%) and 515 (42%) programs had a greater than 10-point difference with each conversion tool.ConclusionThere is wide variation in the within-program differences between minimum USMLE and predicted minimum USMLE (from COMLEX-USA) scores. Many programs have greater than 10-point differences, which may be a source of bias in osteopathic applicant selection. United States (US) postgraduate residency programs often screen applicants using filters in the Electronic Residency Application Service (ERAS) ,2. ProgrThese minimum board score filters create several barriers for osteopathic applicants applying to residency programs. First, although not needed for medical licensure or admittance to residency, many programs expect osteopathic applicants to complete the USMLE to be considered . For exaFurthermore, for those osteopathic applicants who choose not to complete the USMLE, the COMLEX-USA scores are scaled differently and are not directly comparable to USMLE scores . TherefoNevertheless, COMLEX-USA and USMLE scores correlate well among osteopathic students who take both -10,12, aStudy design and settingWe performed a cross-sectional analysis of the American Medical Association’s Fellowship and Residency Electronic Interactive Database Access (FREIDA) database of residency programs on April 4, 2023. All included programs updated their program information between June 3, 2022 and December 20, 2022, except for 19 of 1,477 (1.3%) programs in the Step 1 analysis and 15 of 1,227 (1.2%) programs in the Step 2 analysis, which had missing last update days.ParticipantsWe included the 10 medical specialties with the most training spots in 2022, including internal medicine, family medicine, pediatrics, emergency medicine, psychiatry, surgery, anesthesiology, obstetrics-gynecology, orthopedic surgery, and neurology . We onlyVariablesVariables analyzed included minimum board examination scores , specialty, program type , percent of osteopathic residents (3-year average), and osteopathic recognition. The primary outcome was the within-program differences between the reported minimum USMLE Step scores and predicted minimum Step scores converted from their minimum COMLEX-USA Level scores.Statistical analysisMinimum Step scores were compared to minimum predicted Step scores calculated from Level scores using two conversion tools for each Step examination -10. BarnWe described continuous variables with medians and interquartile ranges (IQR) since all were nonnormal as determined by the Shapiro-Wilk normality test. We described categorical variables with counts and percentages. Confidence intervals (CI) for the median within-program differences between each program’s minimum USMLE and predicted USMLE scores were calculated from the standard deviation (SD) of medians from 1,000 bootstrap resamples. Bootstrap methods have the advantage of not assuming a normal distribution . We examWe did not perform any a priori sample size calculations because our objective was to examine the variation of within-program minimum board score differences and not perform hypothesis testing between reported and predicted USMLE scores. Nevertheless, where reported, CIs were considered significant if they did not include zero. Analyses were performed with R  . PackageOf the 3,364 programs in the FREIDA database from the included specialties, 1,477 (43.9%) met inclusion criteria in the Step 1 analysis and 1,227 (36.5%) in the Step 2 analysis with complete minimum board score data. The median minimum COMLEX-USA scores were 458  for Level 1 and 475  for Level 2. The median minimum USMLE scores were 210  for Step 1 and 215  for Step 2. The most common specialty was family medicine and the most common program type was community-based, university-affiliated hospitals  scores varied greatly between programs, and the variation differed by conversion tool Table . For exaCOMLEX-USA to USMLE conversion tools provide programs with an objective method to establish comparable minimum board scores, facilitating comparisons of academic performance between osteopathic and allopathic applicants -10. HoweLastly, these results generate further questions regarding how programs interpret applicant COMLEX-USA scores. The wide variability in differences between minimum USMLE and predicted minimum USMLE scores (converted from COMLEX-USA scores) suggests that there may be wide variability between programs regarding how they interpret the full range of applicant COMLEX-USA scores. Furthermore, this variability in interpreting COMLEX-USA scores may contribute to the lower match success for osteopathic students who do not complete the USMLE . StakehoLimitationsFirst, many programs do not report both minimum USMLE and COMLEX-USA scores. Therefore, the results of this study cannot be generalized to all residency programs, including programs that do not accept COMLEX-USA alone or do not disclose minimum board score requirements for both examinations. Similarly, programs were more often from family medicine and community-based, university-affiliated hospitals limiting generalizability. Also, this observational data cannot provide causal conclusions about the use of application filters or COMLEX-USA to USMLE conversion tools and match outcomes, nor can it provide definitive evidence for bias in the match process.We only evaluated the two most recently published COMLEX-USA to USMLE conversion tools; however, older tools exist ,23. TherThere is wide variation in the within-program differences between minimum USMLE and predicted USMLE (converted from COMLEX-USA) scores. These results suggest that within-program differences between minimum USMLE and COMLEX-USA scores may be a source of bias in osteopathic applicant selection.


We aimed to analyze the impact of histological chorioamnionitis (HCA) in the presence of preterm premature rupture of the membranes (PPROM) on obstetric and neonatal outcomes, and its possible predictability. A retrospective cohort analysis of PPROM cases (20–37 weeks) was conducted comparing the patients with and without HCA, seeking a predictive model of HCA using logistic regression. A total of 295 cases of PPROM were selected, of which 72 (24.4%) had HCA. The group with HCA had a shorter latency period and a greater number of clinical and laboratory criteria in the evolution. The group with HCA had a worse comparative result and presented: lower gestational age at delivery, lower average birth weight, lower Apgar scores, longer neonatal hospitalization, worse maternal clinical conditions and, higher rates of stillbirth, low birth weight (LBW), very low birth weight (VLBW), complications in pregnancy and childbirth, and cesarean delivery due to fetal distress or chorioamnionitis. A predictive model for HCA was developed, with the following independent variables: abdominal pain (odds ratio [OR] = 11.61), uterine activity  (OR = 5.97), fever (OR = 5.77), latency > 3 days (OR = 2.13), and C-reactive protein (OR = 1.01). With this model, an adequate receiver operating characteristic curve was found, with an area under the curve of 0.726, and some HCA probability curves were constructed for different clinical situations. In this novel study, we present a non-invasive predictive model, with clinical and laboratory variables, which may help in decision-making in a patient with PPROM. Preterm premature rupture of membranes (PPROM) is a condition affecting 2–3% of pregnant women , and accThe clinical diagnosis of chorioamnionitis is not always accurate, and the best marker of intrauterine infection remains Author. Therefore, the accurate identification of women with intrauterine infection continues to be a major challenge . AccuratThis condition has been recently described as intrauterine infection, inflammation, or both (triple I)  based onAccording to some authors, HCA would be the best marker of severity in prematurity, as these situations being associated with serious consequences for the newborn, such as lower gestational age (GA), weight, and Apgar score at birth; higher rates of orotracheal intubation; greater use of surfactant; greater risk for retinopathy of prematurity, respiratory distress syndrome, and early-onset neonatal sepsis (EOS); longer hospital nursery stay , which mTherefore, considering the lack of consistent data in the Brazilian scenario, and the absence of an accurate predictive model for this serious condition, we proposed to carefully analyze the available data of patients with PPROM and HCA at our hospital, with the aim of establishing a predictive model of HCA that can provide the identification of its best clinical and biochemical markers before birth.This retrospective cohort study included all consecutive PPROM cases admitted to the Obstetric Clinic of Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (HC-FMUSP), in São Paulo, SP, Brazil, between 2006 and 2011, with available data on delivery and GA at delivery of > 20 and < 37 weeks. We chose this gestational age period because this is the period in which we can propose conservative and expectant management. Although the gestational age between 20 and 26 weeks is for non-viable fetuses, there is a reasonable possibility of chorioamnionitis and maternal involvement, and the most adequate assessment of its risks can help in the conduction and orientation of the case. Furthermore, we chose this 5-year period because the protocol of the American Centers for Disease Control and Prevention (CDC)  started As it is a retrospective study which included reviewing of medical records, data were not always available for all patients.3 or increase of 20%); increased C-reactive protein (CRP) by 20%; abrupt decrease in amniotic fluid index (AFI), usually greater than 50%; alteration of the fetal biophysical profile, with absence of fetal respiratory movements [Patients were treated according to the previously published HC-FMUSP obstetric clinic protocol , 25, witovements . Regardiovements . This isovements , being dovements .Physical exam and fetal surveillance  were performed daily. In the assessment of amniotic fluid, we considered oligohydramnios and sever oligohydramnios when AFI < 5 and < 3 , 30, resHCA was diagnosed by acute maternal inflammatory reaction in the chorion and amnion, both in the ovular membranes and in the chorionic plate, which may be associated with fetal inflammatory reaction. The maternal reaction was characterized by polymorphonuclear neutrophils in the chorion laeve that migrate from the decidua parietalis to the chorion, or from the subchorionic intervillous space  to the chorion, moving toward the amnion. The diagnosis of fetal inflammatory reaction was made in the presence of fetal polymorphonuclear cells from umbilical vessels or chorionic arteries and veins in the chorion. The presence of an inflammatory reaction in the umbilical cord is called omphalitis or funisitis. This inflammation in the cord can affect only the umbilical vein, or the vein and arteries.Demographic, clinical, and obstetric variables were taken from the patients’ charts. The various clinical comorbidities of pregnant women, such as heart disease, hypertension, nephropathy, asthma, rheumatological diseases, and others were grouped into a single category as "comorbidities." We separated two clinical categories due to the possible association with preterm labor and PPROM: "diabetes” and “urinary tract infection.” Miscarriage was defined as gestational loss at < 20 weeks, and cervical incompetence as "the inability of the uterine cervix to retain a pregnancy in the second trimester in the absence of clinical contractions, labor, or both" .Finally, we sought to associate possible maternal and neonatal outcomes and complications with the presence or absence of HCA. As we have expectant management up to 36 weeks, we raised the possibility of terminating the pregnancy before that, either due to signs of fetal distress , premature labor, or chorioamnionitis. We also verified the presence of labor induction, type of delivery, and other maternal conditions at this moment. Regarding the newborn, we checked the gestational age at delivery, and divided the weight into low weight and very low weight, in addition to data from cord blood gases (when available); Apgar score at 1, 5, and 10 minutes after birth; and perinatal mortality. For the definition of a small-for-gestational-age (SGA) newborn, we consider the birth weight < 10th percentile of the Fenton & Kim curve .The data were arranged in an Excel table and analyzed using the IBM SPSS 23 statistical software. Initially, HCA was descriptively analyzed with numerical variables as means, and standard deviations and categorical variables as frequencies, proportions, and percentages.Patients with and without HCA were then compared using the chi-square test with Fisher’s correction for categorical variables, when appropriate. Normal distribution was confirmed for continuous numerical variables using the Kolmogorov–Smirnov test. All continuous numerical variables presented with p < 0.05 in normality tests; we therefore chose the Mann–Whitney U test for independent samples.Following this analysis, we developed a predictive HCA model by listing the most significant variables in univariate analysis, with p < 0.100. We used stepwise forward logistic regression to estimate the odds ratio (OR) and their respective 95% confidence intervals (95%CI). Additionally, we produced a probability curve for the HCA event with the predictive factors observed in the logistic regression using the following log-linear function:The quality of the predictive model was verified using a receiver operating characteristic (ROC) curve, with area under the curve (AUC) calculation.The research project was approved by the Internal Research Project Approval Committee (CIAPP) of the Department and by the Research Project Approval Committee (CAPPesq) of Hospital das Clínicas under approval no. 248.040, of 4/17/2013 and registered on Plataforma Brasil under the no. CAAE 13978413.2.0000.0068. As this was a retrospective research project, the requirement for written informed consent was waived. Medical record analysis and patient identification were carefully conducted.A total of 295 consecutive PPROM cases were selected for analysis. Of these, 72 exhibited HCA, representing a frequency of 24.4%. Of the 72 cases, 28 had omphalitis, with the infection reaching the fetus. This represented 9.5% of all PPROM cases and 38.9% of specific HCA cases.Of the 72 HCA cases, 22 had a clinical diagnosis of chorioamnionitis before the resolution of the pregnancy. There were eight additional cases of CCA without HCA. Thus, the overall rate of CCA would be 30 in 295 cases or 10.17%.Regarding the demographic characteristics and obstetric history, there was no statistically significant difference between the groups with and without HCA for most variables , except for the fact that HCA patients had prenatal care less frequently at the institution  with fewer appointments , in addition to a tendency to have lower education  and higher frequency of repeated miscarriages  and cervical incompetence .As for the mother, the cesarean section rate was lower, mainly due to spontaneous preterm labor, with a shorter hospital stay, probably related to the shorter latency period. However, the rate of fetal distress was higher, particularly when cesarean section was indicated. This explains a higher frequency of parturients in a regular general condition and a lower frequency of those in a good general condition, with a higher rate of complications  in the immediate postpartum. There was only one case of confirmed sepsis in the HCA group, but there were no cases of hysterectomy or maternal death. In addition, HCA patients had a higher ICU stay but without statistical significance.Logically, chorioamnionitis clinically diagnosed in HCA patients had a higher rate of resolution but it is noteworthy that only 10 of the 72 HCA cases were recognized as such before delivery, indicating resolution. An unpleasant odor was present in 20 of these cases at the time of delivery, confirming chorioamnionitis.Next, significant variables or with marginal significance in univariate analysis (p < 0.100) were selected for multivariate analysis, trying to group those that were similar. The following variables were selected through binary logistic regression for the HCA outcome: education level < high school, latency > 3 days, presence of fever (≥ 37.8 °C) at some point, fetal tachycardia (> 160 bpm), uterine pain, irritable uterus, GA at complaint, final AFI, final CRP, final leukocyte count, percentage difference in the last two leukocyte counts, percent difference in the last two CRPs, maternal heart rate on the day of delivery, and maximum temperature.This selection of variables was used for a stepwise forward regression, and we obtained an appropriate model, with a Hosmer and Lemeshow Test of 0.672, leaving four significant and independent variables in the final model, as shown in An ROC curve was constructed with this predictive model to evaluate the sensitivity of the method in predicting HCA, with an AUC = 0.726 (95% CI: 0.627–0.825) and p < 0.001.To assess the probability of the HCA event, we constructed two probability curves according to CRP variation and considering the main clinical factors of the final predictive model. In the first and second curve, we considered the presence  and abseOur rate of HCA (24.4%) was not remarkably high, particularly considering that all cases in the study were premature with a mean GA of 30.7 weeks on admission. A French multicenter study similar to ours analyzed 295 pregnant women with GA between 22 and 37 weeks and reported a HCA rate of 42.7% . A KoreaIn Brazil, there are few studies evaluating chorioamnionitis as a complication of PPROM. We identified only three, but with different GA compared to ours. A study of patients with GA < 24 weeks reported a CCA and HCA rates of 71% and 36%, respectively . AnotherOther Brazilian studies show higher rates of HCA than ours, ranging from 26.9%  to 40% , but witTherefore, our present study appears to be the first to evaluate this association and its prediction in Brazil.The diagnosis of CCA is not always easy. Often the diagnosis is made only retrospectively. The clinical picture is not always clear, and the diagnostic criteria are not uniform, even in Brazil. Although the ACOG does not establish unpleasant odor among its chorioamnionitis criteria , we use Indeed, there is debate about the best definition of chorioamnionitis because the underlying inflammatory condition is often not diagnosed in CCA. HCA would be easier, but it is only possible retrospectively, following birth. Bastek et al.  demonstrIn a review on the subject, Newton  also higTherefore, we decided to study HCA, with the understanding that it would constitute an appropriate marker of worse prognosis in PPROM. Indeed, our data support this assertion as they represent a more serious condition when HCA is associated with PPROM with a shorter latency period; lower GA at delivery; lower newborn weight; and higher rates of LBW, VLBW, fetal distress, unpleasant odor, delivery complications, low Apgar score at 1, 5, and 10 min, and stillbirth. Although maternal hospitalization was shorter, newborn hospitalization was longer, demonstrating a more severe condition and perinatal predominance . This corroborates with findings from some authors who report that HCA is worse than CCA –49, 51.Indeed, several authors indicated worse neonatal indices in patients with HCA, such as lower GA, LBW , 49–51, Such an increase in the risk of several neonatal complications should obviously be associated with a higher risk of perinatal death. Data from Han et al.  complemeHowever, if HCA is more severe, would we be able to identify it before delivery? Unfortunately, the initial condition of our patients on admission did not establish a clear difference between patients who would progress with or without HCA other than a lower GA at PROM and on admission. Furthermore, there was only a non-significant trend toward more bleeding on admission i.e., 15.9% vs. 8.5% (p = 0.09) for HCA and CCA, respectively. However, we observed a progressive clinical condition associated with HCA, with higher rates of severe oligohydramnios (AFI < 3); lower final AFI; high final CRP with a significant increase between the last two tests (582% vs. 20%); higher rates of leukocytosis with difference between the last two tests (33.1% vs. 6.5%) but not as significant as that with CRP; fever at some point but not necessarily on the day of delivery; a slightly higher maximum temperature (37.1 vs. 36.7 °C). There were also higher rates of maternal or fetal tachycardia, in addition to a higher mean maternal heart rate at some point or even on the day of delivery. The higher frequency of uterine contractions and lower frequency of abdominal pain are noteworthy. These signs are all considered in the clinical diagnosis of chorioamnionitis but they were not always present at the same time and to the extent expected; otherwise, the rate of CCA would be similar to that of HCA.Such changes in the clinical course are corroborated by other authors. Smulian et al.  describeA British study  at the KRyan et al.  also desOur data describe a different predictive model, in which not only CRP but other clinical variables were associated. In fact, few authors have made a predictive model of HCA in pregnant women with PPROM like we did, with clinical and laboratory variables in a non-invasive way. Only two authors built predictive models with clinical and laboratory variables. However, one of them applied to pregnant women with and without PPROM , and theMa et al.  conducteBeen et al.  evaluateThe other authors only describe predictive models in the laboratory. Two of these authors , 49 madeCobo et al.  analyzedKim et al.  retrospeStepan et al.  studied Perrone et al.  performeCaloone et al.  collectePPROM is a distressing clinical situation for both patients and obstetricians. It is not easy to balance the risk of prematurity on the one hand and the risk of intrauterine infection on the other. The lower the GA, the more difficult the decision to deliver or maintain the pregnancy. We believe that the results of this study can collaborate in the daily decision-making process of obstetricians who will be able to consider the risk of chorioamnionitis more precisely, based on not only laboratory but also on clinical data. This will provide a more adequate balance between the risk of infection and of prematurity. In this sense, evaluating the probability curves may help in the decision-making process at the bedside. Although this study is carried out in Brazil where it is unprecedented in its considerations, we believe that its results can help obstetricians worldwide. In conclusion, the present study is relevant not only in the Brazilian scenario, but also internationally, as its approach is unprecedented in considering clinical and laboratory results easily available on a daily basis in the prediction of a serious condition such as HCA. However, this predictive model currently requires validation in new samples of patients with PPROM due to greater antibiotic and corticosteroid use. We will be evaluating this in our new research project.


This paper focuses on image and video content analysis of handball scenes and applying deep learning methods for detecting and tracking the players and recognizing their activities. Handball is a team sport of two teams played indoors with the ball with well-defined goals and rules. The game is dynamic, with fourteen players moving quickly throughout the field in different directions, changing positions and roles from defensive to offensive, and performing different techniques and actions. Such dynamic team sports present challenging and demanding scenarios for both the object detector and the tracking algorithms and other computer vision tasks, such as action recognition and localization, with much room for improvement of existing algorithms. The aim of the paper is to explore the computer vision-based solutions for recognizing player actions that can be applied in unconstrained handball scenes with no additional sensors and with modest requirements, allowing a broader adoption of computer vision applications in both professional and amateur settings. This paper presents semi-manual creation of custom handball action dataset based on automatic player detection and tracking, and models for handball action recognition and localization using Inflated 3D Networks (I3D). For the task of player and ball detection, different configurations of You Only Look Once (YOLO) and Mask Region-Based Convolutional Neural Network (Mask R-CNN) models fine-tuned on custom handball datasets are compared to original YOLOv7 model to select the best detector that will be used for tracking-by-detection algorithms. For the player tracking, DeepSORT and Bag of tricks for SORT (BoT SORT) algorithms with Mask R-CNN and YOLO detectors were tested and compared. For the task of action recognition, I3D multi-class model and ensemble of binary I3D models are trained with different input frame lengths and frame selection strategies, and the best solution is proposed for handball action recognition. The obtained action recognition models perform well on the test set with nine handball action classes, with average F1 measures of 0.69 and 0.75 for ensemble and multi-class classifiers, respectively. They can be used to index handball videos to facilitate retrieval automatically. Finally, some open issues, challenges in applying deep learning methods in such a dynamic sports environment, and direction for future development will be discussed. The sports domain offers many opportunities to apply computer vision (CV) to help athletes, coaches, and teams reach their goals by enabling more efficient and detailed analysis and tracking of performance, or for the broadcasters to improve the experience for the viewers—however, the fast pace, complex dynamics, and interactions in many sports challenges implementing CV solutions. Additionally, achieving the high reliability required in commercial applications often leads to expensive solutions involving many specialized high-speed cameras, sensors, and support staff that are sustainable only at the top competitive level of the most popular sports, such as football, baseball, or basketball. Thus, it is of great research interest to develop more advanced and robust CV-based solutions with fewer technical constraints and requirements that would allow a much broader adoption of CV applications in both professional and amateur sports.In the research, there is a great focus on sports, such as basketball, soccer, and volleyball. For example, in , the autResearchers have recently studied group activity in team sports and individual actions. For example, in , the autThere are many publicly available datasets that contain data specific to the sports domain. Some examples of publicly available datasets are the UCF Sports Action Data Set , OlympicA more extensive overview of related work in action recognition in sports is given in . In the Therefore, the subject of our research in this paper is focused on the application of deep learning methods and building computer vision models for handball. The goal of this work is to train models for automatic player detection and player tracking, and recognition of the actions they perform and their temporal localization on handball videos recorded with only one camera, from a position accessible to spectators and coaches. The key issue is to make models that can be easily used by everyone, especially recreationists, young athletes, and athletes in small clubs, and everyone else interested in sports without expensive equipment and cameras to track their performance and improve the usability of recorded videos in the case of handball.This work is a continuation of our comprehensive research project in which we dealt with player and ball detection , trackinThe contributions of this paper include the analysis of two different strategies for training the action recognition models, namely, a comparison of a single multi-class model capable of assigning one action label to each sequence and of an ensemble of binary classifiers where each classifier can assign or not assign its corresponding label, resulting in multi-label classification.Handball actions vary widely in duration, from 10 to 80 frames, and have important segments in different parts of the video sequence. Since action recognition models require a fixed number of input frames, the influence of the length of the input sequence was separately analyzed on the performance of the classification of isolated actions and on the temporal localization of actions in unconstrained videos. In the case of temporal action localization, the considered time window may include only a part of the action or contain parts of different actions. All experiments were conducted on a custom dataset of annotated handball videos .The next section presents materials and methods used. The first subsection presents the most important methods used for semi-manual action dataset creation to accomplish the task of action recognition in handball, including object detection and person tracking. The second subsection explains action recognition and temporal localization tasks, followed by a description of the algorithms used in the experiments. The third section presents the action recognition and temporal localization experiments and fourth obtained results, followed by a discussion. Finally, the last section gives a conclusion with future research directions.To train the model for different computer vision tasks on handball scenes, it was necessary to create a custom dataset since there was no suitable dataset that could be used to train the model in the handball domain. Given that the creation of a set for supervised training is a long-term and expensive process, the goal was to apply an approach that facilitates and speeds up the creation of a set.The challenge in video tagging of handball scenes recorded during training and matches was that there are multiple players on the field at the same time performing different actions, moving in different directions, at different distances from the camera, and dynamically entering and exiting the camera’s field of view. The idea was to tag actions in videos using a person detector that automatically detects special location of players and tracking methods that follow their movements to help automatically recognize their actions. The annotation of the data set includes also manual temporal labeling where time sequences corresponding to a handball action are marked. As different actions can take place at the same time, multiple actions can be marked at some point in time, where there is no clear connection between the annotation and the spatial location where the action occurs. Player detection and tracking isolate ongoing candidate actions from a video. The actions are performed by individual players and lastly, for a sequence of consecutive frames. At the same time, the basic handball actions, such as “Shot”, “Jump Shot”, “Running”, “Dribbling”, etc., were temporally annotated using a video annotation tool. The next step merges the tracked player with the action taking place in that time and involves manual validation of the matches to build the final ground truth data for action recognition models. Expert-assessed matching of automatically selected video clips with the performed action is crucial at this stage because the athlete tracking model still does not have satisfactory accuracy, and the goal is to populate the dataset only with accurate data that will be useful for training the model.The resulting UNIRI-HBD_v2 dataset contains 4451 sequences in 9 action classes and a “Background” class with class distribution and average length, as shown in The described process of dataset creation is relatively complex, but it is important that once defined, the player detection and tracking model can be used multiple times to automatically supplement the dataset of handball actions.To recognize player actions and activities later, a handball player should be detected first, i.e., the player must be located on an image or consecutive video frame. This is a typical object detection task, where the goal is to locate all objects of a certain class  in the image and return the spatial location and spatial extents of the detected objects, usually represented with rectangular regions.During the preparation of the handball dataset, different configurations of Mask R-CNN , YOLOv3 Tracking handball players is a multi-object tracking problem because there can be 14 players simultaneously if a handball competition is analyzed, and even more in a training session. With an object detector, all the players’ locations would be known in the ideal case. However, there is still no correspondence between the detected player bounding boxes and the players’ identities. This problem can be solved using a tracking algorithm, or tracking-by-detection algorithm, that should assign a unique ID to each player that appears in a frame and maintains that ID while that player is in the scene. To accomplish that, a tracker can use the information obtained by the object detector, such as the dimensions and the location of the bounding boxes, along with their relative positions considering previous frames with the assumption that the objects should be near their last position or move according to a known movement model, e.g., in a linear fashion. Trackers may also use visual features extracted from within the bounding boxes if the overall appearance of the tracked object maintains some constant features that can be tracked even with changing pose or orientation. Cost functions are defined using both sources of information that serve as a quantitative measure of how different a bounding box is from the presumed position or appearance of the tracked object in the next frame. The Hungarian algorithm is used to find the globally optimal assignment of identification tracks to the bounding boxes of the players obtained by the object detector so that the cost function for the assignment reaches the global minimum.Tracking of players can be very demanding because players move quickly through the field and often change directions, challenging the assumption that the detected bounding box of a player in the next frame will be near or nearest to the last one. In addition, especially in the competition setting, the players wear similar clothes, often in colors that also appear in the “Background” , further complicating tracking of appearance features that must be specifically sensitive to player identity unrelated to clothes.In this work, we expand on the work of , where DBoT-SORT is a recent algorithm that achieved state-of-the-art results on multi-object tracking challenge MOT17 and MOT20 datasets . It usesThe trackers were tested on two sequences of handball practice from UNIRI-HBD dataset, with lengths of 48 s and 5 min 30 s, both at 60 frames per second, respectively. In both sequences, the camera is stationary, but in the first sequence, it zooms in at some point. The ground truth sequences were generated by detecting the players using YoloV3, “Running” the tracking using the Deep-SORT algorithm, and manually correcting the track assignments.Since the accuracy of the object detector greatly influences the tracking accuracy, in testing, both the Deep-SORT and BoT-SORT trackers were fed the same bounding boxes generated with the Mask R-CNN detector. Since here we are mostly interested in the tracker’s abilities to assign the player detection to correct track IDs correctly. In re-identification performance, only measures that deal with track assignment are evaluated. In contrast, the commonly used Multiple Object Tracking (MOT) evaluation measures, such as Multiple Object Tracking Accuracy (MOTA) or Multiple Object Tracking Precision (MOTP) concerned with bounding box correspondences and object detection performance are ignored. Namely, the number of identity (ID) switches  and idenAn identity switch means that a ground truth target was assigned to a track j and the last known assignment was to a different track k ≠ j. The IDF1 focuses on how long a target is correctly identified regardless of the number of mismatches and expresses the percentage of correctly identified detections over the average number of ground truths and computed detections.Once the players are tracked, actions annotated in time can be combined with the tracked players in a video.The final step involves connecting the action labels marked on a timeline with the detected player tracks. Timeline labels only contain temporal dimension and action identity but lack correspondence to the spatial location of the players performing the marked action. On the other hand, the detected player tracks locate a player in space and time in the video but lack information about the actions performed. The goal is thus to map the action labels and time ranges to corresponding player tracks, or rather, specific time ranges within a player tracking. However, since player tracking and detection are imperfect, a single-player track that should contain all appearances of that player in the video is, in fact, fragmented, so many short tracks with different IDs represent one player. Furthermore, due to missing player detections or wrong bounding boxes, the tracks may be interrupted where the player is still present on the scene or contain only part of the player that is visible in the scene. For these reasons, although an action is marked in the timeline, it is not guaranteed that a corresponding track is detected that contains the whole sequence of a player performing that action.For each time segment where an action occurs, there are normally as many tracks as players in the field, and usually more due to track fragmentation . The detected player tracks and corresponding boxes are treated as a new video sequence, where each frame is extracted from the bounding box in the original video . Since tOF) activity measure proposed in [x, y), the vector magnitude represents the speed, and its angle represents the direction of movement. The activity measure  to a video sequence. The usual assumption is that for each frame, only one pertinent label exists, i.e., only one action at a time. The recognition may be on a frame-by-frame basis, or on the level of the whole sequence, which is then also assumed to contain a single action. In the case of the unconstrained video of sports events, including handball, a more demanding task of spatiotemporal action localization is usually desired. The idea is to recognize which handball actions are performed, in which period (sequence of frames), and where the handball players performing individual actions or multiple players performing an activity are in the scene. A common approach for spatiotemporal localization, also used in this paper, is to reduce the problem to a series of action recognition tasks by generating candidate action sequences that are extracted spatially and temporally from the unconstrained video and applying the action recognition model to assign the appropriate action label. Since the generated candidate sequence does not always really show an action, the action recognition model should be able to handle that case. For that reason, in this work, we train the models on a “Background” action class which denotes a sequence that does not show a handball action or is defective due to poor player detection or tracking.Considering the complexity and duration of certain actions and activities, some can be detected using just one frame, but for more complex actions, a sequence of frames is needed for reliable recognition, such as a “Jump Shot” consisting of several physical movements, such as “Dribbling”, jumping vertically, and simultaneously throwing a ball. For that reason, models that take the time dimension into consideration are applied to the task of handball action recognition.In the following experiments, a group of models based on the Inflated 3D Network (I3D) architecture  were useIn this work, the final two layers of the I3D network streams were replaced to fit the number of classes in our dataset. The models were then trained on our dataset, starting from the Kinetics pre-trained models, with all the original layers frozen and only changing the parameters of the last two layers.The size of both RGB and optical flow frames was 224 × 224 pixels (px). In the original paper, the RGB input was prepared by first proportionally scaling the input video so that the smaller side is 256 pixels, then taking a random 224 × 224 px crop. However, in our case, since the input image is already a tight crop around a player, square cropping from a typically tall input would cut off too much of the image, such as the player’s hands, and thus lose important information about the performed action, so we opted to simply rescale the input frame to 224 × 224 px. For the optical flow stream input, the dense optical flow was first computed on the input video using Farneback’s algorithm , then a In the first experiment, I3D models were trained for recognition of 10 action classes  on UNIRI_HBD_v2 dataset.Since the ground truth actions in the dataset contain action clips of varying lengths, during training they had to be somehow reduced or expanded to the input length of the model. In previous research , we inveTherefore, the I3D models were trained on four different input sequence lengths, ranging from 10 to 60 frames, and using two different strategies to handle the varying sequence lengths, namely, cropping and decimation. In the cropping case, the set number of frames was extracted from the beginning part of the sequence. In the decimation case, an input sequence was reduced to the desired length by dropping frames throughout the sequence, so that the network “sees” the whole action, but at a reduced frame rate. In both cases, if the original sequence contained fewer frames than needed, the necessary number of frames was copied and inserted between existing frames to extend the sequence.The videos were randomly divided into training and testing sets in a ratio of 80:20 for each class. The I3D models were trained using the Adam optimizer with a learning rate of 0.00005, up to 30 epochs with a batch size of 10 on an NVIDIA RTX2080 GPU. To augment the dataset, during training, in some sequences, all frames were randomly flipped left-to-right so that for example a “Shot” towards the left becomes a “Shot” towards the right side.In the next experiment, a binary classification strategy was applied, so that for each class, a separate binary classifier was trained that decides whether a sequence should be labeled with that class or not. Each model was an I3D model as above, but with only two output classes.All binary classifiers were trained on the same training set as above; however, the class labels were modified, so that if, e.g., a “Jump Shot” classifier was being trained, all “Jump Shot” examples were labeled as true examples, and all examples of other classes were labeled as false. In this case, there was no need for a separate classifier for the “Background” class, which was represented by the negative output of each classifier. The input sequence length was 60 frames for all classes, taken from the beginning of the original sequence, except for “Catch” and “Throw” classes, for which the input length was 20 frames.Each model was applied to the test set and evaluated independently for its ability to classify the sequence into the correct class in terms of precision, recall, and F1 score. In this way, a sequence from the test set could be classified into zero, one, or more action classes, as it is possible that more classifiers assign their class label to the example. In the case of complex actions, such as “Dribbling” or “Passing”, this could be desirable . Additionally, since the input sequence length was adjusted according to individual classes, so it is shorter for “Catch” and “Throw” than for “Passing” classes, these classes can be handled better than in the multi-class model.The most demanding task is spatiotemporal action localization, where the goal is to detect where and when actions occur in an untrimmed video and output a series of bounding box detections with the associated class labels.For this task, we generate the candidate action sequences using the same object Mask R-CNN detector and Deep SORT tracker as in dataset preparation. The candidate sequences are then split into parts of equal duration in a sliding-window fashion  to be prThe evaluation was performed on 6 video clips from different camera angles, of a total duration of 5 min 31 s, that were not used in training the models.The evaluation was completed for each class separately, then the obtained results were averaged. A classified sequence was considered as true positive for a certain class if the intersection over union (IoU) overlapped between the ground truth sequence and the sequence classified into that class was at least 10% in the temporal domain. The required percentage was quite low because ground truth annotations may be much longer or shorter than the window length, e.g., a ground truth annotation of a “Throw” action in 10 frames and a window length of 60 would, in the best case, achieve an IoU of 16%. If the class is correctly assigned, but the IoU score is less than 10%, the classification is counted as a false negative. If a wrong class is assigned to a sequence, the classification is counted as a false positive for that class.On the UNIRI-HBD test dataset, we compared the player detection results of the models we tuned for player and ball detection on our dataset  with today’s improved versions of those models (Yolov7) to compare performances and to evaluate whether we need to update the data in the dataset we use to train the athlete detection model. The models were tested on a subset of 27 manually labeled images extracted from the videos of handball practice. The results of object detection in terms of average precision are shown in For player detection, the best results were achieved with the state-of-the-art Yolov7-e6e model, closely followed by the older Mask R-CNN model. The improvement between Mask R-CNN and Yolov7-e6e is much larger for the ball class, by nearly 15%, than for the person class, which was subjectively very good already with Mask R-CNN. Still, the ball detection performance needs to be more reliable for tracking the ball and should be still improved.The best results for ball detection were achieved with the custom Yolov3-PB model, with an average precision of 35.44, which shows that training on custom data greatly improves the results over the basic model; however, it is still insufficient to achieve reliable ball detection. This is expected as the ball is very small and moves very fast when in the air, causing motion blur that can alter its appearance or make it virtually disappear in the “Background” and is often occluded by players. Therefore, the model comparison shows that it is unlikely that a newer model or just retraining the model on a larger amount of data will solve the ball detection problem, and it is assumed that more specific model architectures will be needed.In the following tracking experiments, the players were detected using the Mask R-CNN detector, which was available at the time of creating the dataset, and the ball detection was not used. Since the annotation process is very time-consuming, not all videos were annotated. In the future, Yolov7-e6e or some other better-performing detector will be used for player detection in the rest of the videos.On the handball video from UNIRI-HBD test dataset, we compared the player tracking results achieved with Deep SORT models that were used with a fine-tuned player detectors (YOLOv3 and Mask R-CNN) with newer tracker BoT-SORT  that we The reported advantage of BoT-SORT in MOT challenges  does notSimilar with the ball case in object detection, it seems that tracking players in team sports, such as handball, differ enough from the general object tracking or tracking tuned to pedestrians, that a specialized approach is warranted that should utilize the constraints of the given sport.An example of parts of different chronological frames where the tracking is performed can be seen in Although both algorithms produce tracks that are much more fragmented than the ground truth, the resulting tracks are, in most cases, still long enough to contain a whole or most of a handball action.The F1 score of I3D_C models using the cropping strategy improves with an increasing number of frames from 10 to 60. However, the I3D_D models with the decimation strategy achieved the best result with 40 frames, slightly better than I3D_C with 60 frames.In the case of the decimation strategy, 10 and 20 frames had the same result for an F1 score of 71%, while the increase in frames from 40 to 60 frames recorded a drop in the F1 score from 78% to 75%. Changes in precision and recall metric scores behave in a similar way, with the crop strategy they increase with the increase in the number of frames, and with the decimation they achieve the best results at 40 frames.The sequence of 60 frames is around the average duration of video frames considering all the classes.To analyze the results in more detail, we used the confusion matrix, which makes it easy to see whether the classifier routinely confuses two classes. Looking at the confusion matrix, it is evident that large confusion occurs between more complex actions that include a common simpler action as its part. For example, both “Dribbling” and “Crossing” include a simpler throwing action. “Dribbling” is a complex action that involves “Running” while bouncing the ball off the floor, explaining the confusion with the “Running”, “Throw” and “Catch” classes. Similarly, “Crossing” is a complex action that consists of two “Passing” actions with a change of the players’ positions and a “Shot”, which explains the misclassifications between “Crossing” and “Passing”. The largest single confusion occurs between “Shot” and “Throw” actions, which are very similar and differ mostly by intensity and intent, a “Shot” is directed towards the goal and is usually performed with higher speed and intensity than “Throw” which signifies any other throwing of the ball, normally towards another player. “Catch” and “Throw” actions are also similar short interactions with the ball, which include a player raising his hand/hands to “Throw” or “Catch” the ball.Considering the results achieved when observing all 10 classes together, the experiments were continued observing only one class at a time, i.e., if a certain action occurred or not.In the second experiment, a separate I3D binary classifier was trained for each action for each sequence. The results of applying the binary classifiers on the test set are shown in For all but one of the classes, the multi-class classifier outperformed the ensemble of binary classifiers in terms of F1 score. The results for the remaining class, “Crossing”, were rather similar for both models. Relatively problematic is the complex “Passing” action that is composed of “Catch” and “Throw” actions. The ensemble classifier would have the advantage over the multi-class classifier in this case since the ensemble classifier is not forced to choose between the “Passing” class and its parts (“Catch” and “Throw”), such as the multi-class classifier is. Instead, each binary classifier for a “Catch”, “Throw”, and “Passing” could assign its label to the “Passing” sequence. In this way, it would be expected that the ensemble classifier has overall better recall but possibly worse precision than the multi-class classifier. Surprisingly, the opposite is true here, with the ensemble classifier having a much lower recall than the multi-class classifier for both “Catch” and “Throw” classes, and the same recall with lower precision for the “Passing” class.Additionally, the ensemble method has a large drawback in that it requires “Running” 10 models on the same example, and so requires at least 10 times the processing time than the multi-class model, and more if the time to load the models in the GPU memory is considered.The averaged results of action localization are shown in The I3D_40D model, which achieved the best results in the action recognition task in terms of all considered metrics, also performed the best regarding the precision and F1 measures for the action localization task. In terms of recall, the I3D_40C performed slightly better at a cost of lower precision. In most cases, for the same number of input frames, the decimation models achieve better results in terms of F1 measure than the models using the crop strategy, with the only exception being the model with 60 input frames.An example timeline visualization of detections for a model with 10 input frames and a model with 60 input frames is shown in The ensemble model surprisingly did not show the best result as in the action recognition task, even with the advantage of having different input lengths for different actions.This paper considered a computer vision pipeline for analysis of activities of handball players in videos, with the final task of spatio-temporal localization of handball actions in unconstrained video. The pipeline consists of several subtasks, where object detection and player tracking provide spatially localized proposals for the sub-task of action recognition. Similar pipelines were considered for sports, such as basketball  and vollIn the player detection task, both the Mask R-CNN and Yolov7 models already achieved good results without fine-tuning on the handball-specific dataset, with AP from about 85 to 91, depending on the configuration. However, the ball detection task was much harder, with the best AP result of 35 achieved with the Yolov3 model trained on our dataset. Training the ball detection on our dataset increased the average precision significantly, and a similar increase can be expected with newer object detection models, but the results still need to be satisfactory for real applications in handball analysis. Furthermore, since the possession of the ball performs a great role in interpreting the game situation, further specialized models for ball detection that go further than single-frame analysis should be investigated.Generating action proposals relies on player tracking in addition to player detection. The problem of multiple object tracking, particularly tracking pedestrians, is widely researched . CommonlI3D models with different strategies (multi-class and ensemble of binary classifiers) have been investigated for the action recognition task. The best results were achieved by the multi-class model with an F1 measure of 0.78 averaged over all classes, which outperformed the ensemble of I3D models, each trained as a binary classifier for one action class with the average F1 measure of 0.69. In a future study, removing the complex actions, such as “Passing” and inferring them from their parts (“Throw” and “Catch”), will be investigated instead of direct classification of the complex actions, as these proved to be the source of most classifier confusion.In many applications, such as video retrieval or statistics generation, it would be useful to find all actions by a certain player, which needs to be addressed in this study. However, this problem is linked to player tracking and should be considered in future work. In addition to classification accuracy, future work should focus on increasing the speed of classification, as the processing pipeline is still quite complex and computationally demanding. Approaching real-time processing would open new application possibilities in real-time statistics or highlight generation.Several experiments with the application of deep learning methods for video content analysis of handball scenes were presented in this paper. Player detection, tracking, and action recognition were considered incremental steps toward the spatio-temporal localization of actions in the video, but different experiments were shown for each task.For the action recognition task, I3D models are trained on a subset of input lengths. Next, the models are evaluated on previously extracted individual action sequences for action recognition and finally on the unconstrained video of handball training for action localization.The set of considered actions consisted of 9 handball actions of varying complexity, from a simple “Throw” action performed by one player to complex actions, such as “Crossing” that consists of several “Throw” and “Catch” actions, and actions that involve more than one player. The varying complexity and length of actions proved challenging for action recognition tasks, where a complex action could be misclassified as one of its parts or vice versa.For this reason, in future work, we plan to reduce the number of complex actions that are directly classified and focus on shorter and simpler actions while inferring the complex actions from the simple ones.It would also be useful for coaches to analyze not only different actions, but also the different execution of actions by a particular player. In this sense, we plan to combine the localization of actions with the identities of players, so that coaches can guide their players to better execution of actions thanks to the possibility of filtering actions by type or player.