Leishmania infantum (that causes VL) undergo drastic thymic atrophy, with significant reduction in thymocyte numbers, and decreased levels of intrathymic chemokines and cytokines, indicating that both lymphoid and microenvironmental compartments of the organ are affected. Lastly, recent data revealed that some probiotic bacteria or probiotic fermented milks improve the thymus status in a model of malnutrition, thus raising a new field for investigation, namely the thymus-gut connection, indicating that probiotics can be envisioned as a further adjuvant therapy in the control of thymic changes in undernutrition accompanied or not by infection.Undernutrition remains a major issue in global health. Low protein-energy consumption, results in stunting, wasting and/or underweight, three deleterious forms of malnutrition that affect roughly 200 million children under the age of five years. Undernutrition compromises the immune system with the generation of various degrees of immunodeficiency, which in turn, renders undernourished individuals more sensitive to acute infections. The severity of various infectious diseases including visceral leishmaniasis (VL), influenza, and tuberculosis is associated with undernutrition. Immunosuppression resulting from protein-energy undernutrition severely impacts primary and secondary lymphoid organs involved in the response to related pathogens. The thymus—a primary lymphoid organ responsible for the generation of T lymphocytes—is particularly compromised by both undernutrition and infectious diseases. In this respect, we will discuss herein various intrathymic cellular and molecular interactions seen in undernutrition alone or in combination with acute infections. Many examples illustrated in studies on humans and experimental animals clearly revealed that protein-related undernutrition causes thymic atrophy, with cortical thymocyte depletion. Moreover, the non-lymphoid microenvironmental compartment of the organ undergoes important changes in thymic epithelial cells, including their secretory products such as hormones and extracellular matrix proteins. Of note, deficiencies in vitamins and trace elements also induce thymic atrophy. Interestingly, among the molecular interactions involved in the control of undernutrition-induced thymic atrophy is a hormonal imbalance with a rise in glucocorticoids and a decrease in leptin serum levels. Undernutrition also yields a negative impact of acute infections upon the thymus, frequently with the intrathymic detection of pathogens or their antigens. For instance, undernourished mice infected with  Despite one of the global health priorities listed in the Sustainable Development Goals (SDGs) by the United Nations is combating hunger and ensuring sustainable food security and proper nutrition , malnutrMalnutrition thus constitutes a serious global public health issue, particularly in developing countries. According to a recent report  of the FDue to the pandemic, the prevalence of different forms of malnutrition has increased worldwide and it is estimated that these effects will be lasting, as already seen in 2021. In fact, the COVID-19 pandemic has widened and worsened inequalities between countries, affecting the livelihoods of an estimated 1.6 billion workers in the formal economy . Data frThe decrease in proper food intake results in a series of physiological changes, including: (i) growth restriction; (ii) reduction of fat, muscle, and visceral mass; as well as (iii) reduction in basal metabolic rate and energy expenditure . These aThe dynamic relationship between infection, nutrition and immunity is long recognized. Being a systemic disease, undernutrition also affects primary and secondary lymphoid organs, harming the immune system of malnourished individuals . Since tIn the present review we aimed at focusing the effects of undernutrition and infection upon one specific compartment of the immune system, namely the thymus, particularly the changes in the organ involving cellular and molecular interactions. Yet, before going into this point, it seems worthwhile to provide a general, yet concise background of the thymus and the generation of T-lymphocytes.+ CD8+ double-positive (DP) thymocytes are localized in the cortical region of the thymic lobules, whereas more mature CD4+ CD8- or CD4-CD8+ single-positive (SP) thymocytes are placed in the medulla , thymic dendritic cells (TDC), macrophages and fibroblasts, as well as non-soluble and soluble molecules such as the extracellular matrix (ECM) proteins fibronectin, laminin, type I and IV collagens; cytokines as interleukin (IL)-2, IL-6, IL-7, and IL-22; chemokines as CXCL12, CCL4, and CCL7; hormones as thymosin, thymopoietin, and thymulin; and different typical soluble components of nervous tissues, such as neuropeptides and neurotransmitters , 16.+ CD8+ developing thymocytes are rescued from programmed cell death upon interaction with self-antigens presented to the TCR by molecules of the major histocompatibility complex (MHC) by thymic microenvironmental cells. Positively selected thymocytes still in the cortex of the thymic lobules then move into the boundary of the cortex and medulla of the thymus for the presentation of self-antigens by the medullary thymic epithelial cells for the second time. In the medulla, thymocytes pass through the negative selection. In this case, differentiating cells undergo apoptosis if their TCR interact with high avidity with self-antigens coupled to the MHC class I or class II molecules expressed by microenvironmental cells in the organ (+ CD25+ Foxp3+ T cells (Treg), through a mechanism dependent on the TCR signaling avidity and duration , upon interacting with microenvironmental cells, particularly dendritic cells, and epithelial cells. Positive selection can be defined as the process through which TCR-expressing CD4he organ , 13. Difhe organ , 18. Altduration , 20.+ CD8+ cells, by ligation with the receptor CXCR4. The chemokine CCL25 also attracts immature thymocytes, although all thymocyte subsets are responsive, which is in keeping with the fact that its CCR9 receptor is expressed at all stages of murine thymocyte differentiation. Interestingly, CCL19/CCR7 interactions participate in thymocyte exit, illustrating that thymocytes may switch their responses to a given chemokine, tuning their migratory process , as compared to control animals , as summFinally, although not detailed herein, it is worthy to mention that deficiencies in vitamins, as well as in micronutrients  also promote thymic atrophy with cortical thymocyte depletion and changes in the microenvironmental compartment . Particu+ CD8+ stage of differentiation.In addition to the lymphoid compartment, the thymic microenvironment is affected in various undernutritional and infectious conditions. Morphological changes in the thymic epithelium from protein-undernourished mice include the decrease in the volume of the epithelial tissue in the cortex and in the medulla of thymic lobules from undernourished mice, as compared to well-nourished control animals . By contTrypanosoma cruzi-acutely infected mice, we also demonstrated changes in the expression of medullary and cortical specific TEC markers such as cytokeratins, as compared to controls, together with a general shrinkage of the thymic epithelial network , which in turn stimulates the adenopituitary gland to secrete ACTH (adrenocorticotrophic hormone) that enhances glucocorticoid secretion by the cortex of the adrenal glands. Physiologically, such axis is self-regulated by negative feed-back with glucocorticoids being able to negatively control the production of both ACTH and CRH. Yet, it has been demonstrated that maternal food restriction during the perinatal period or during lactation disturbs the activity of the HPA axis at weaning, with pups presenting reduced adrenal, thymus and liver weight, and increased circulating free corticosterone levels . DisturbMild maternal protein deprivation during lactation in rats affects thymic homeostasis in the young progeny, which presents lower body and thymus weights, significant alterations in CD4/CD8-defined T cell subsets and enhanced expression of leptin receptor ObRb in thymocytes. Although alterations in leptin circulating levels were not observed in this study, an increase in leptin signaling response of thymocytes from protein-deprived rats was described, together with a decreased rate of thymocyte spontaneous apoptosis when compared to controls . InteresIt is known that the thymic atrophy induced by undernutrition produces a negative impact in the immune response against infections , 66, 67.T. cruzi, Plasmodium and Leishmania  T cells due to increased apoptosis and premature egress of immature thymocytes  is a neglected disease that frequently afflicts children and malnourished populations in tropical and subtropical regions of the world. The disease is caused by be fatal , 181.Seminal prospective studies in children who acquired VL revealed that those who had a precondition of moderate to severe undernutrition before infection had 8.7 times greater risk of having classic and severe forms of the disease, whereas well-nourished children did not progress beyond subclinical infections . More reL. donovani and, consequently, anticipated the visceralization of the parasite  thymocytes, and remarkable alterations in total single positive subpopulations (CD4+ and CD8+) of the organ  proteins, cytokines, chemokines and hormones , 192. Weinfantum . Undernoinfantum .L. infantum infection and worsened by a precedent undernutrition condition.Also, early changes in protein abundance were observed in infected animals, and those alterations were exacerbated or annulated when animals were previously undernourished, reinforcing the deleterious role of undernutrition in the response to infection and revealing an unknown role of the thymus during VL . FunctioL. infantum, but amastigote nests were only observed in mice fed protein restricted diet  pathways, as well as upon the NF-kB transcription factor, especially by inhibiting IkB phosphorylation, thus hindering the transfer of NF-kB .Much less is known about the putative action of probiotics upon the thymus. In a mouse model of non-severe protein undernutrition, we found that the administration of a probiotic fermented milk as a re-nutrition supplementation did improve the thymic microarchitecture, recovering the corticomedullary differentiation in the thymic lobules, in conjunction with a decrease in thymocyte apoptosis. All the structural changes of thymus were accompanied by changes at the functional level, with increase in the numbers of mature and immature thymocytes and enhancement of cytokine production .L. casei CRL 431 administered in the drinking water to a high-fat diet consuming obese mice, improved the cellularity and functionality of the thymus, with an increase in IL-7 and IL-3 production as seen in In a second vein, studies in obese humans and in obese mice, showed a relation between obesity and involution of the thymus gland revealing the important role of nutrients intake on the anatomy and functionality of the thymus , 204. ThAs undernutrition remains a major issue in global health, studies on the pathophysiological aspects of this affection, associated or not with infection are yet important, particularly the mechanisms governing dysregulation of the immune response. In this respect, the immunosuppression resulting from protein-energy undernutrition severely impacts primary and secondary lymphoid organs involved in the response to a give pathogen. The thymus is particularly compromised by both undernutrition and infection, with a consistent cortical thymocyte depletion and important changes in thymic epithelial cells. Moreover, undernutrition yields a negative impact upon the thymus in acute infections, frequently with the intrathymic detection of pathogens or their antigens.The impairment of the thymic cortical area due to undernutrition during acute infections may alter physiological processes crucial for T cell development, such as (i) intrathymic selection of the T cell repertoire , (ii) thWS and PC conceived, wrote, reviewed, and edited the manuscript. JD, CM-G, GP, and DM-d-C wrote the manuscript. All authors contributed to the article and approved the submitted version.


Wearable sensors are an important tool in the study of head acceleration events and head impact injuries in sporting and military activities. Recent advances in sensor technology have improved our understanding of head kinematics during on-field activities; however, proper utilization and interpretation of data from wearable devices requires careful implementation of best practices. The objective of this paper is to summarize minimum requirements and best practices for on-field deployment of wearable devices for the measurement of head acceleration events in vivo to ensure data evaluated are representative of real events and limitations are accurately defined. Best practices covered in this document include the definition of a verified head acceleration event, data windowing, video verification, advanced post-processing techniques, and on-field logistics, as determined through review of the literature and expert opinion. Careful use of best practices, with accurate acknowledgement of limitations, will allow research teams to ensure data evaluated is representative of real events, will improve the robustness of head acceleration event exposure studies, and generally improve the quality and validity of research into head impact injuries. The summary statements were discussed, revised as necessary, and ultimately approved by more than 80% of the vote at the conferenceA head acceleration event (HAE) is defined as an event/incident that gives rise to an acceleration response of the head caused by an external short-duration collision force applied directly to the head or indirectly via the body in sport, recreational, military, or other activities of interest. Wearable devices are often both kinematically and field validated for direct HAEs and not indirect HAEs due to the limitation of reproducing indirect HAEs in the lab and identifying indirect HAEs on the field, respectively.Kinematic data must be filtered to remove potential false positive recordings and verify valid HAEs. Data windowing, video verification, and pre- and post-processing techniques aid in data validation. Individual verification of HAEs is challenging, and time consuming but improper data validation may lead to errors in estimation of exposure.Video verification serves as an independent verification of HAEs for a given application  and provides contextual information for HAEs. However, video should not be considered ground truth as the confidence in video verification depends on video quality and a robust labelling process. Guided and blinded video verification of head acceleration events are useful components to device performance in an on-field environment.Advanced processing techniques  have the potential to offer fast and reliable verification of valid HAEs. However, they are often developed for specific wearable devices in specific applications  and it is best practice to independently validate processing methods for use in the intended application.Before deploying head acceleration measurement devices in an on-field environment, users should establish data collection and analysis protocols according to the activity, resources, and research questions. Additionally, users should ensure 1) the devices are functional, 2) the batteries are charged, 3) the devices are attached securely to the individual, and 4) the wearable device is time-synchronized with other concurrent data sources .This work was part of the Consensus Head Acceleration Measurement Practices (CHAMP) project. The objective of CHAMP was to develop consensus best practices for the gathering, reporting, and analysis of head acceleration measurement data in sport. Subject matter experts were recruited to draft a series of papers on various aspects of the issue. As described in detail in a companion paper,59 In the early 2000’s, the first commercially-available device, known as the Head Impact Telemetry (HIT) System, was introduced as a means to monitor head acceleration data simultaneously and continuously from American Football athletes on the field in real-time.29 During the last two decades, numerous studies have been published using the HIT System to monitor HAEs in American football.49 Recent advances in sensor technology and the expansion of the commercial market have allowed researchers to broaden this scope beyond the sport of football by using sensors embedded in a mouthguard, attached to the skin at the mastoid process, worn on the head via headband/skullcap, or attached to a helmet.52Head acceleration events (HAEs) in sporting and military activities have been studied among researchers for over 60 years.Wearable devices are now ubiquitous in studies of HAEs. These devices have provided varying levels of ability to measure the kinematics of HAEs and an athlete’s exposure to HAEs across multiple games, training sessions and, in some cases, seasons. Despite their convenience and potential to advance our understanding of head accelerations, brain injury, and their sequelae, properly utilizing and interpreting data from wearable devices requires diligence. Additionally, as research on HAEs in sports develops, there is a great need for consensus on procedures surrounding validation and verification of HAEs measured by devices.in vivo.For accurate measurement of HAEs, wearable devices must be validated for (1) kinematics and (2) detection of events. In-lab validation against a gold standard  must be performed to assess the accuracy of kinematics measured from the underlying measurement sensors (i.e. the accelerometer), but the ability of devices to detect and record possible HAEs is best conducted in an on-field, real-world environment. The objective of this paper is to define minimum requirements and best practices for on-field deployment and validation of wearable devices to ensure data evaluated are representative of real events and limitations are accurately defined. Best practices covered in this document include the definition of a verified event, data windowing, video verification, advanced post-processing techniques, and on-field logistics, as determined through review of the literature and expert consensus; these best practices apply to device-recorded events collected Traditionally implementation of wearable devices in an on-field environment occurs following laboratory kinematic validation of the underlying sensors (i.e. the accelerometer). Though this is not strictly required, best practice recommendations presented herein assume the underlying sensors have been validated in the laboratory for head acceleration kinematics measurement and may be applied across a wide range of activities. This is because often wearable devices rely on the kinematic signals to identify the HAEs, implying some level of accuracy of the underlying sensors in their ability to measure or estimate head kinematics. Understanding these assumptions, on-field deployment and methods to validate wearable device data should be selected and utilized according to the intent of the research study. Justification of methods and associated limitations should be clearly defined and acknowledged when disseminating research results.via the body in sport, recreational, military, or other activities of interest.72 For military applications, HAEs considered herein are those that occur in the non-blast environment, though HAEs could be secondary to the blast, such as a direct head impact with a wall following a blast. Direct HAEs, i.e., head impacts, involve primary collision with the head or helmet. Indirect HAEs, i.e., without head contact, involve primary collision to the individual’s body resulting in inertial motion of the head. For example, this may include a fall to the ground or a body-to-body collision. These indirect HAEs are often associated with greater rotational head motions. HAEs can result in a wide array of linear and rotational head acceleration combinations depending on the location and direction of the collision force , of individual HAEs to quantify this performance. Wearable devices are often both kinematically and field validated for direct HAEs and not indirect HAEs due to the limitation of reproducing indirect HAEs in the lab and identifying indirect HAEs on the field, respectively. However, devices deemed valid from in-lab validation for direct HAEs may reliably measure kinematics of indirect HAEs due to the strict sensor requirements for impact scenarios. Similarly, methods to identify direct HAEs in the field can be applied to indirect HAEs.The definition of HAEs is important for sensor systems as they form the basis for assessing their performance, commonly defined in terms of sensitivity, specificity, or accuracy, and validity. On-field validation of wearable devices is predicated on an assessment of this performance as a measure of their ability to identify HAEs in the field. This fundamentally relies on the verification, though independent means .ods Fig. , from siData windowing is a relatively simple method for determining which device-recorded events are associated with HAEs. We have defined five steps of data windowing that incrementally reduce the device-recorded events to possible HAEs. Time-stamped device-recorded events are typically downloaded by date either from an online portal, the manufacturer, a sideline collection system, or the device itself. These data often contain device-recorded events for the duration the device was active  on a particular date. This allows the data to be “windowed” to remove events that occurred outside of time windows of interest. To window the data, at minimum study staff should be present during data collection to note time windows of interest. Current best practice is to have video footage of the data collection to facilitate review of, or confirmation of, data windows after the event. It is also best practice to have video synchronized with the clock of the device  to improve alignment between video and device time stamps.54 To identify unscheduled stoppages, most sports have a sound  to indicate when a stoppage has commenced and play has resumed during a game session, which can be used to establish timepoints by study staff or from time-synchronized video. In addition, some sports have game clocks that are paused during stoppages .Temporal windowing  is a method of retaining device-recorded events that occurred during specific time windows and removing events that occur outside of the time windows. The first step of temporal data windowing is to establish the start and end timepoints of a session Table , Step 1.The quality of windowing possible HAE data improves when using temporospatial information to further reduce the number of false events. The third step of data windowing involves establishing time windows in which certain groups  are active Table , Step 3.22 Conceptually, this method could be applied to field activities as some use GPS trackers,20 the data from which could be used to remove device-recorded events that were not spatially associated with the activity of interest. This tool may be challenging for applications requiring indoor data collection.The fourth step of data windowing requires obtaining individual participation time, which is when an individual is known to be participating in the activity of interest Table , Step 4.The fifth step of data windowing is to establish active participation time for an individual Table , Step 5.via time-synchronized video collected during the data collection. In-person monitoring of relevant timepoints  may be completed, and is the minimum requirement to complete data windowing;40 however, in-person notes often lack the preciseness needed for verification and full reliance on this method, in the absence of time-synchronized video collected during a session, may introduce error in recorded timepoints and eliminates opportunities for further verification. It also limits the metadata  that may be collected about the session. Additionally, in-person monitoring of events becomes increasingly more challenging with a greater number of individuals instrumented on-field and the size of the field of play/training arena. The usefulness of in-person monitoring may also depend on the activity, level of play, and resources available (including trained personnel). For example, in-person monitoring of header events in soccer may be conducted more easily than in-person monitoring of head impacts in ice hockey or football due to the level of contact and the speed and complexity of action that occurs during a session.The best practice for identifying relevant timepoints for data windowing is 73 This form of guided video verification is primarily used to remove false positive events , and carefully examine high severity sensor recordings that are most associated with mild traumatic brain injuries.60 This can often be used in conjunction with time windowing (which reduces the periods in which video verification is required), but itself requires substantial resources and coordination to deploy additional video recording resources for reliable verification. Outlined best practices in this section are intended to provide the framework for successful video verification.Video verification of HAEs serves several purposes in the deployment of wearable devices and their on-field validation. As an independent assessment of HAE incidence, video verification is most commonly used to confirm device-recorded events.18 and/or the type of HAE .54 Such contextual information can augment device measurements to categorise head acceleration severity in different types of events73 or can be used to develop machine learning classifiers for wearable devices to automatically differentiate events.78 In some cases, contextual information can also help determine issues with kinematic measurements  that are usually caused by external factors . Despite its many benefits, video verification remains an important but uncommon procedure in the HAE analysis pipeline, because video verification requires significant investment in personnel to video record events and identify events in the videos and some researchers have accepted the validity of devices at face value without thorough evaluation. In their 2020 review, Patton et al. estimated that nearly two-thirds of head impact studies did not perform or report observer or video verification of individual device-recorded events.52 Still, video verification remains a best practice method for verifying HAEs in order to validate the on-field capabilities of wearable devices and providing contextual data for further HAE analysis.During video verification of HAEs, contextual information on the HAE can also be described, such as the impact site 39 Recommended practice is to utilize additional broadcast quality video cameras  tracking the passage of play where most HAEs occur . This increases confidence during video verification, but also drastically increases cost and time for the video verification process. This may also be challenging at lower levels of sports  and in military environments, where there is limited infrastructure to facilitate multi-angle video collection. Placement of cameras depends on the activity being recorded, but best practice is to tune placements to maximize areas where potential HAEs may occur. It is also recommended practice to place cameras at heights >2 meters to reduce occlusions. As video cameras often record video at a distance, the resolution of video capture plays a critical role in the quality of video for verification. Most prior work utilizes 720 pixel–1080 pixel resolutions for sports such as American football and soccer.54 Greater video frame rates also provide more refined identification of the moment of impact, though as most wearable devices record at least 50ms around an impact, minimum frame rates of 30 frames per second (33 ms between frames) are acceptable.Effective video verification of HAEs depends on a robust pipeline to record and process video footage from sessions. Recording videos involves deploying video camera(s) to the field, but the number, placement, resolution, and frame rate of video recording plays a role in the quality of video footage for future assessment. In terms of number, a minimum of one camera is required to simply collect video of an event, though single camera systems suffer from player occlusions and ambiguities that introduce uncertainty in video verification. Thus, it is best practice to have at least two cameras capturing different views  such that one camera can account for occlusions or ambiguities in the other.etc.). Thus, it is recommended practice that all video, on-site reference clocks, and wearable device clocks are referenced twice, once before and once after an activity, to account for differential video and wearable device clock drifts. Such time synchronization is important to accurately capture the likely discrete event that triggers wearable devices, as often there can be multiple HAEs in quick succession . Clock referencing can be achieved by having a machine set the internal clock for video cameras and wearable devices, by video recording the machine clock,34 or by video recording intentional impacts applied to wearable devices.48 It is a best practice to regularly synchronize the machine clock for the wearable device with a global source .During deployment of video systems, it is critical that video cameras, on-site observer notes, and other devices (primarily wearable devices) are time-synchronized. Current best practice is to reference all video footage and wearable device clocks (HH:MM:SS) from a single machine at some point in time for each data collection (most often before the data collection). However, researchers should be aware that there is often variance between individual device clocks  discussion amongst the raters that reviewed the coding, or independently coded by an expert rater with more personal or professional experience with the activity. Proper training will likely minimize the number of inconsistencies requiring further review.Once collected, video verification is a laborious process that requires robust training protocols and consistent instructions on how individuals (raters) should verify or confirm HAEs. As video verification is a subjective process, it is best practice that instructions for rating HAEs be made explicit, for example clear instructions for identifying a direct HAE and how to label HAEs when the head is occluded by another object. At minimum, these rating instructions should be reported in literature so that they can be used in other on-field deployments for consistent ratings across studies. For on-field verification, at minimum one rater should video verify each timestamp of each device-recorded event captured by a wearable device that will be utilized in further analysis. These may not necessarily include all device-recorded events, such as in analysis of severe impacts where only high acceleration events are characterized.68 Low agreement within a single rater indicates high uncertainty or ambiguity in either the video or instructions.To ensure consistent and robust verification of HAEs, it is best practice to conduct intra-rater and/or inter-rater reliability analysis prior to video verification, depending on whether single or multiple raters are used respectively. Intra-rater reliability can be assessed for single raters by having a single rater review a video, wait for a period of time, and review the video once more. To assess inter-rater reliabilities, all video raters involved with video verification should review a common video to identify HAEs. The inter-rater agreement can then be used to ensure robustness of video verification procedures, as well as eliminate raters who do not have sufficient agreement with others.53 False negative events can occur due to linear acceleration magnitudes lower than pre-defined recording trigger thresholds77 or if events are misidentified by device algorithms and subsequently erased or not provided to the user.Thus far, video verification has been discussed in the context of verifying whether device-measured events are associated with HAEs, a process termed guided video verification. Guided video verification is important for identifying false positive HAE events, a component in validating on-field performance of wearable devices Table . However39 In the first stage, videos are reviewed by single raters with instructions to identify any possible instance of a HAE (high sensitivity) but are blinded to the head acceleration data. This effectively provides a collection of possible HAEs mirroring what wearable devices provide and thus a guided video verification can be subsequently performed in the second stage to confirm first-stage video-identified possible HAEs (high specificity). In both guided and blinded video verification, when multiple video angles are available, it is best practice to review videos together to determine whether an HAE occurred. Recommended practice is to first review videos independently, potentially by different raters, before they are assessed together. This primarily increases the confidence for HAEs that are independently identified in different video views.While blinded video verification provides additional an assessment of false negatives for a more complete on-field validation of wearable devices, it also requires more resources. For blinded video verification, at minimum each full video should be reviewed by a single rater to determine independently when HAEs occur. However, this approach is susceptible to individual rater biases (as blinded verification will likely use multiple raters due to the sheer volume of video), and thus recommended practice involves the utilization of a 2-stage video verification process.72 When an event is observed on video but not detected or recorded by the device, it is often referred to as a ‘possible false negative’. However, false negatives may be associated with observable and reported concussions. It is not common practice for studies to report the number of possible false negatives, but it is an important aspect of on-field validation to accurately estimate the HAEs experienced by an individual. Prior studies that have reported the number of possible false negatives have simply reported the number of events but have assumed these instances are ‘non-events’ and did not analyze them further.61 Alternatively, some studies have included these events in the calculation of impact rates or impact frequency, but acknowledge their exclusion from the analysis of head acceleration data.72One challenge that may arise when classifying HAEs during blinded video verification, is the ability to discern the amount of contact necessary to induce a wearable device recording of a HAE.76 Reasons for exclusion include insufficient film quality,74 the event occurred outside of the field of view,54or the characteristic of interest  was not clearly visible.23 In the event that a device recording was excluded because it occurred outside the field of view, there is a potential for underestimation of exposure, therefore, the methods, assumptions, and limitations need to be clearly described and acknowledged to aid in interpretation of the data by others.Collection of video footage for verification of events may be limited by resources and/or the environment being studied. Even when high quality, multi-angle camera views are collected, there are several challenges that must be considered. When reviewing HAEs, it may be challenging to distinguish direct HAEs  and indirect HAEs. Even with high quality video, there will always be uncertainty surrounding the distinction between direct and indirect HAEs. Many prior studies have excluded device recordings from analysis due to uncertainty of the event, or if the event was not clearly visible in film and only evaluated impacts that could be clearly verified with video; studies have reported a range of 6-75% of events recorded by devices that were ultimately included in analyses.Video verifying wearable device-measured HAE events remains an important tool or both the validation and development of wearable head acceleration measurement devices. However, it is important to keep in mind that video verification itself is a subjective process and should not be treated as an absolute ground truth, but rather an independent measurement of head impact and HAE exposure. Thus, great care must be taken to ensure robustness of video verification processes, much in the same way that HAE from wearable devices are expected to be validated. While we only present methods for identifying HAEs in video here, similar approaches can be utilized to ensure robustness of contextual analysis for HAEs, such as the impact site or the type of head acceleration. Indeed, in treating video verification as an independent measure for HAEs, similar techniques can be applied to cross-validate future novel methods for detecting HAEs, such as acoustic and proximity monitoring.81 but care must be taken with implementation of these techniques with acknowledgment of their limitations. Importantly, many commercial advanced post-processing techniques are proprietary, but their performance can and should be independently validated on the field in the setting of a deployment prior to extensive use. This will provide confidence in their further use to quickly identify HAEs and reduce the reliance on video verification.While both guided and blinded video verification are often used to verify which wearable device-recorded events are HAEs, this method is not universally accessible due to both the high time and cost requirements. Recent advances in post-processing techniques, such as machine learning, have allowed for automated classification of HAEs,in vivo80 and different applications result in distinct head acceleration characteristics (helmeted vs. unhelmeted sports).79 Thus, it is best practice to validate post processing techniques for each device in each specific application.The goal of advanced post-processing is to analyze wearable device-recorded events and determine which are associated with real events  and which are associated with false events, or non-HAEs. While a seemingly simple task, it is complicated by the use of different wearable device form factors and applications in different contexts . Commonly, methods are tuned to work for a specific wearable device in a specific application. This is because different wearable devices are known to have different performance 81 Sometimes these techniques can be used as a pre-process filter to determine whether to record data . More often, this information is collected and integrated into a post-process decision tree to eliminate a substantial number of false positive events; however, even these hardware-based methods are often not sufficient to eliminate all false positive events.78 This has led to a second approach relying on software post-processing methods to identify the unique features of HAEs from extraneous measurements.There are commonly two approaches to advanced post-processing. One approach relies on additional hardware or study design considerations that can help determine if a wearable device is being properly applied. For example, instrumented mouthguards can be equipped with a specialized proximity sensor to detect when the mouthguard is being properly worn as an additional confirmation of true head acceleration.81 Here, the support vector machine is commonly used and is often paired with frequency-based  or biomechanics-based  features. Deep learning classifiers are designed to learn features from the data, which allow them to identify complex relationships in the data for classification purposes.46 Here, traditional fully-connected neural networks and more complex convolutional neural networks are commonly used but require expertise in selecting appropriate hyper-parameters .Software post-processing methods often use machine learning algorithms that can be grouped into two distinct classes: feature-based classifiers and deep learning classifiers. The former relies on engineered features that are extracted from kinematic signals in order to differentiate HAEs.43 Even for research-based devices with peer-reviewed post-processing algorithms, it is still important to understand how these algorithms are developed and evaluated. To evaluate post-processing algorithms, the sensitivity and specificity are most often reported as they describe true positive rate and true negative rate, respectively (with video acting as an independent measure). As a best practice, all post-processing algorithms should report these two metrics as a measure of the algorithm performance.While most wearable devices incorporate some form of post-processing to identify HAEs, commercial devices often use proprietary algorithms.in vivo on the field, labels for true sensor-recorded events are often conducted using a guided video verification. Under this process, only events recorded by wearable devices are provided a true positive or false positive label for validating post-processing algorithms. However, based on previous blinded video verification work, it is known that many wearable devices do not record a substantial number of HAEs, likely because they do not produce accelerations above recording trigger thresholds.61 Sensitivity and specificity metrics often do not account for these missed events . Second, as the video verification process is subjective in nature, rater biases can be captured by software post-processing. Third, similar to this, wearable device measurements themselves during an observed HAE can be corrupted by other sources of error, producing kinematic signals that are not representative of head acceleration which in turn may corrupt post-processing pipelines. As an example, in mouthguards, it is known that the lower jaw (mandible) can bite down on mouthguards during a HAE and produce a characteristic high frequency oscillation measured by the underlying kinematic sensor independent of the head acceleration.72 These can be removed with device-specific techniques  but are predicated on understanding the origins of these erroneous kinematic signals.There remains some nuance to how these metrics are calculated that is important to highlight. This pertains specifically to the definition of the true positive HAE and true negative extraneous event. For data that are collected Advanced post-processing methods are a convenient way to automatically classify device-recorded events into true HAEs and false events and further facilitate data processing steps toward a dataset that is representative of real HAEs. This is particularly true for large multi-institution datasets where processes such as video verification are logistically impractical. Tiered approaches can be utilized for these larger datasets, wherein a subset of the data is set aside to develop and validate novel advanced post-processing techniques with video verification, and subsequently applied to the remainder of the dataset. Currently, advanced post-processing methodologies are specific to the wearable device and the application, but we provide best practices to assess their performance and highlight several nuances and limitations in their design.75 Wearable devices are being deployed to measure characteristics of gait and physical activities across a broad range of health conditions,66 but the assessment of usability is very limited.33 Searches using combinations of “usability”, “head impact”, “on-field”, and “concussion” identified few papers that reported on the system usability of head acceleration devices. Tierney et al.. (2021) reported their intention to conduct a study of player comfort and wearability with an instrumented mouthguard.71 Australian Football players responded positively to a wearability survey of the X-Patch sensor, but the authors noted on-field in-game issues, such as devices failing, detaching and being lost.43Finally, the last component to ensure quality on field data is in having proper deployment logistics. Wearable head acceleration measurement devices intersect with many disciplines, including ‘Digital health’ and ergonomics/human factors, and are not limited to biomechanics, sports science, or medicine.There is considerable interest in the use of wearable devices across many sports, from American football to cricket, as well as military applications. Each environment has a set of unique characteristics that must be considered when selecting a wearable device and its suitability for on-field deployment Table . There hvia the base station before use. If the base station does not display a battery life indicator, the user can ensure the devices were charging for an appropriate amount of time prior to the event of interest. It is important to note that battery life tends to decrease over time and it is recommended practice to periodically run battery life checks of random devices that are in use for multiple seasons. It is best practice for users to keep a daily log of device function on-field to record if/when a device is determined non-functional for the respective session due to battery death or sensor connection failure.Four critical on-field deployment best practices are to ensure that (1) the device is functional, (2) the battery (or device power supply) is charged, (3) the device  or device-mounted equipment  is attached securely and properly to the individual wearing the device, prior to data collection, and (4) the cameras are time-synchronized to the device clock or common source . For device functionality, the user can, at a minimum, verify each device turns on as it should. In many cases, the user can see if the device is connected to the base station in real-time. If this feature is available, the user can verify device connection. If the device transmits data to the base station in real-time, the user may impact the device such that the base station displays the impact to the user to ensure active communication. If the device does not have real-time data transmission, the user can follow manufacturer instructions to ensure the device is active prior to the data collection session. To assess battery life, users can verify battery change 8 Many devices are available, and each has its own attachment method . Manufacturers often provide instructions to ensure proper attachment, and it is important to follow these instructions prior to each data collections session. For example, to improve coupling of helmet-mounted devices, the user can conduct helmet fittings according to the helmet manufacturer’s guidelines to ensure proper fit. For mouthguard mounted devices, the user can ensure that the device fits the athlete comfortably and does not easily fall off the upper dentition when inserted. It is recommended practice to monitor proper fit periodically during the data collection period, when possible, and throughout the season to assess and address changes in fit or attachment. If this happens, it is recommended practice to have a protocol in place whereby users can remedy the deficiency during a stoppage in play that does not impact normal gameplay or practice.Finally, proper device attachment is critical.Number of staff and training The number of staff required will reflect the size of the team cohort, the ease of attachment, device-to-system connection and initiation protocols, and available pre-game or pre-practice preparation time. Staff will require training and a high degree of technical knowledge may be required, depending on the system. Consistent device deployment is one important aspect of obtaining reliable data, including fit/application of devices prior to deployment. Thus, it is important that training is standardized across all staff and staff follow standard operating procedures. Standard operating procedures should be available to staff at each data collection event for reference.Wearable device matches the sport and intentions of the study In some cases, research teams may be limited to a single device. But, if the research team has a choice between various devices, one of several factors to be considered is the sporting environment and the purpose of the study. Some devices are limited to helmeted sports and even certain types of helmets while other devices attach directly to the participant’s head. There are pros and cons to each device and each needs to be carefully considered, along with laboratory assessed device performance.Follow the manufacturers’ instructions for system operation and device attachment With an increasing variability in available wearable devices, users must be familiar with operation and attachment manufacturer instructions for their specific device. Users should be aware that device fit, and adherence could change over the course of a session or season and should consult the manufacturers recommendations to ensure consistency. Proper fit should be regularly monitored during the season to assess and address changes in fit or attachment. Where possible, proper fit should be monitored during the activity itself as well.Device reference checks Ideally, each device would be checked for functionality automatically during the device-to-system connection process. In addition, a quality assurance process should be considered to check that the device is calibrated and providing accurate and reliable measurements. A quality assurance process might include random sampling of devices for laboratory calibration checks and randomly assigned video verification of device-recorded events during the data collection period. This may help identify devices that may need attention or participant behavior that needs to be corrected . The sampling protocol would reflect the known error rates in the devices.Keep daily log of data collection activities At a minimum, event start and end times, device functionality, and individual attendance can be recorded in a daily log to track data collection activities and facilitate data review  data protection, data storage and record keeping, confidentiality, privacy, and informed consent are all important considerations The research team should ensure the study participants, or their legal guardians have provided informed consent and that data are protected and confidential. Study teams should avoid using participant names that could be displayed on a base station where a bystander could observe real-time impacts. Data sharing protocols need to be considered relative to local procedures and approved by the respective institutional review boards. The study team should consider whether data will be shared with individual participants or at a team or league level. If they are shared, it is best practice to ensure the participant understands the limitations of on-field head impact data and present the data in a way that is understandable to the lay person. The research team should also have awareness of device data storage policies  and ensure that local institutional review board regulations and policies are followed for storing human subject data.Hygiene All wearable devices will be exposed to body fluids. Therefore, it is best practice that infection control procedures are implemented for handling and use. This will generally involve gloves, cleaning, and disposal protocols. It is also best practice to clean devices according to manufacturer recommendations on a regular basis.54Data review It is recommended practice for study team members to conduct preliminary data reviews on a regular basis throughout the data collection window. Data review will assist in team and individual participant management but will also identify any systematic issues that require investigation.In addition to the on-field deployment best practices and general considerations for wearable device selection, the following list highlights important issues with on-field deployment logistics and best practices:


A structural equation model indicated that enjoyment mediated the pathway from instructional media to immediate posttest, and interest mediated the pathway from instructional media to delayed posttest score, indicating that these factors may play different roles in the learning process with immersive media. This work contributes to the cognitive affective model of immersive learning, and suggests that immersive lessons can have positive longitudinal effects for learning.This study describes and investigates the immersion principle in multimedia learning. A sample of 102 middle school students took a virtual field trip to Greenland via a head mounted display (HMD) or a 2D video as an introductory lesson within a 6-lesson inquiry-based climate change intervention. The HMD group scored significantly higher than the video group on presence ( Ana, a middle school geography teacher is in the process of deciding on how she will teach her class about climate change, and is considering using a nationally available inquiry based science learning (IBSL) intervention that lasts six lessons. In this intervention, students are assigned to groups and play the role of a research team that has to virtually travel to Greenland to investigate the evidence and consequences of climate change. They are then required to develop a number of experiments, and then present their results to a hypothetical UN panel of experts. The intervention is available in two versions where students can either experience climate change in Greenland with a 3D video presented via a head mounted display  or by watching the same content as a 2D video viewed on a projected screen . Ana’s school has recently purchased HMDs, but she is unsure of the potential added learning and motivational value of using them compared to the more conventional video solution.Ana’s dilemma is one that many educational stakeholders will be facing in the coming years. Although there is abundant literature on 2D virtual field trips in education  in education and their previous use in climate change education, followed by an introduction to the immersion principle in multimedia learning. Then, we will present the state of the art and gaps in the literature in the field of immersive learning, which motivate five hypotheses about if, and how, the immersion principle in multimedia learning works in the IBSL middle school intervention summarized above.With the increased quality and availability of affordable IVR systems involving head-mounted displays (HMDs), the number of regular IVR users in the UA. increased from 30.6 million to 45.3 million from 2018 to 2019, and were projected to be 55.3 million in 2020 , identified an increasing number of studies on the topic of climate change  and long-term (an average of approximately 3 weeks after the field trip) impact of using either a 3D video accessed through a HMD or 2D video to provide a virtual field trip as an early experience within a larger inquiry-based module on climate change. In addition, we investigate how affective factors including presence, enjoyment, and interest are involved in the learning process using structural equation modeling (SEM).Figure The second link in the model presented in Fig. Regarding link 3, presence can also spark situational interest in the learner  after the virtual field trip.Several studies have highlighted the possibility that assessing the short-term impact of leaning with IVR based lessons may not be capturing the actual value of using immersive technology . The students were either in 8th grade (n = 82) or 9th grade (n = 20) and were from four different public schools from different regions in a European country. The students reported their gender as 39 boys, 63 girls, and 0 non-binary. A sensitivity power analysis was conducted to estimate the minimum effect size detectable for an independent samples t-test between two groups  with 80% power, and α = 0.05 using G*Power 3.1  would be detectable with the current sample size. In a between-subjects experimental design, 49 students served in the HMD group  and 53 served in the 2D video group . The study was conducted in a European country with all materials in the local language.The participants were 102 students between the ages of 13 and 16 , an immediate posttest consisting of presence, interest, and enjoyment surveys and a knowledge test, and a delayed test consisting of the same knowledge items as in the immediate posttest. The presence scale consisted of five items adapted from the physical presence sub-scale in Makransky et al. . Three oThis is Climate Change: Melting Ice. Students began the virtual field in a helicopter which lands at a science base camp in Greenland, where they then follow former US Vice President Al Gore who was visiting a scientist at the base. Students were able to experience the consequences of the changing climate from a first person perspectives as shown in the top panels of Fig. The instructional materials consisted of a 3D HMD version or a 2D video version of a virtual field trip to Greenland to experience the consequences of climate change. The virtual field trip was built on a documentary by Dennis and Strauss  labeled,All students experienced the same educational content, with the only difference being the viewing device. The students in the 3D HMD group experienced the virtual field trip as a 360° video administered through Samsung S7 or S8 phones using the Samsung Gear VR head mounted display as shown in the bottom panels of Fig. The virtual field trip was an integrated part of a nationally available IBSL-based climate change learning intervention which was developed by a cross disciplinary group of stakeholders including teachers, educational psychologists, instructional designers, and pedagogical experts. This study was part of a national assessment that was designed to assess the educational value of the intervention, and this study describes the results for learning outcomes. Four teachers from different regions of the country (one teacher from each school) agreed to use the intervention and participate in the assessment of the intervention. The intervention consisted of six lessons which each lasted approximately 50 min and included three assessment time points, a prequestionnaire (at the beginning of the intervention), an immediate posttest  and a delayed posttest . Figure (2D video condition: n = 53) or as a 360° experience in a HDM (HMD condition: n = 49). Following the virtual field trip students took the immediate post-test. Students also completed survey items to measure presence, interest, and enjoyment. These two lessons were conducted on the first day of the study in all schools as can be seen in Fig. The six lessons were structured as follows. The first lesson consisted of a prequestionnaire, and the introduction to a fake news article. Students started by responding to the prequestionnaire, which included questions regarding gender, age, and grade-level. Finally, students read a fake news article followed by a plenary discussion highlighting a controversy around climate change. Lesson 2 consisted of an instruction of the scientific method and a virtual field trip to Greenland. The research team was present during this lesson and randomly assigned students to one of two experimental conditions using randomized ID numbers. The randomized ID number was used to place students to a working group of 3 to 4 students within their assigned condition, which they worked in from lessons 3 to 6. Students were then separated into two separate rooms based on their assigned condition, where they either experienced a virtual field trip to Greenland as a video on a projector screen in the classroom The remaining lessons (lesson 3–6) were designed based on IBSL and generative activity principles of multimedia learning ; however, there were differences based on dissimilar schedules across schools (School 1: 8.99 days (31 students); School 2: 13.17 days (20 students); School 3: 22.39 days (25 students); School 4: 33.89 days (26 students). See Appendix Table Hypotheses 1 through 5 were investigated using independent samples t-tests with presence rating (Hypothesis 1), enjoyment rating (Hypothesis 2), interest rating (Hypothesis 3), immediate posttest score (Hypothesis 4), or delayed posttest score (Hypothesis 5) as the dependent variables, and treatment condition (3D HMD or 2D video) as the independent variable. Furthermore, we investigated the cognitive affective model of immersive learning presented in Fig. M = 14.12, SD = 0.67; M = 14.15, SD = 0.69 respectively), or mean grade level . A chi-square test showed that the HMD  and 2D video  groups did not differ significantly in terms of the proportion of boys and girls, X2 (N = 102) = 0.090, p = 0.764. Furthermore, we calculated the correlation between the change in learning from the immediate test to follow-up test and the time between the two measurements to investigate if the amount of time between the lessons influenced learning. The correlation of r = 0.047 (p = 0.640) suggests that the time between the two assessments did not influence the results, so the data was grouped together in conducting further analyses.First, we investigated whether there were differences between the HMD and 2D groups on demographic characteristics. The HMD and 2D video groups did not differ significantly on mean age  significantly higher presence than the video group , t(100) = 7.171, p < 0.001, d = 1.43, which supports hypothesis 1. The next row of Table M = 4.65, SD = 0.63) reported significantly higher enjoyment than the video group , t(100) = 7.171, p < 0.001, d = 1.10, which supports hypothesis 2. Finally, the next row of Table M = 4.06, SD = 0.76) reported significantly higher presence than the video group , t(100) = 2.856, p = 0.003, d = 0.57, which supports hypothesis 3. Overall, these results support hypotheses 1, 2, and 3 and are consistent with the immersion principle, which states that immersion has an impact on affective processing in the learner.We predicted that the HMD group would produce higher presence (Hypothesis 1), enjoyment (Hypothesis 2), and interest (Hypothesis 3) ratings than the video group. These hypotheses were assessed using independent samples t-tests. As shown in the top row of Table t-test was used to test hypothesis 4 that the HMD group will score higher than the video group on the immediate posttest test in line with the immersion principle. As shown in fourth row of Table M = 18.29, SD = 4.04) scored significantly higher than the video group , t(100) = 3.072, p = 0.001, d = 0.61. We therefore conclude that there were significant differences between the HMD and the video conditions on declarative knowledge immediately after experiencing the virtual field trip which is consistent with Hypothesis 4 and the immersion principle from which it is derived.An independent samples t-test was used to test Hypothesis 5 that the HMD group will score higher on the delayed posttest than the video group in line with the immersion principle. As shown in the bottom row of Table M = 18.90, SD = 4.35) scored significantly higher than the video group  on the delayed posttest, t(100) = 3.507, p < 0.001, d = 0.70. Therefore, hypothesis 5 was supported. In general, the results support hypotheses 1 through 5 and indicate that there was a medium effect size difference between the HMD and video conditions on interest (d = 0.57), as well as immediate retention (d = 0.61), and long-term retention (d = 0.70), and a large effect size difference on presence (d = 1.43) and enjoyment (d = 1.10).An independent samples β = 1.212, p < 0.001, se = 0.172), indicating that the students in the HMD condition reported higher presence than those in the video condition as expected. Furthermore, presence was a significant antecedent of enjoyment , and interest  as predicted in the model. The correlation between enjoyment and interest was also significant . The path from enjoyment to immediate posttest score was also significant , but the path from interest to immediate posttest score  was not significant. Therefore, the SEM shows how immediate posttest score is significantly related to enjoyment, but not interest.We investigated whether the link between level of immersion and posttest scores is mediated by presence, interest, and enjoyment as depicted in the cognitive affective model of immersive learning in Fig. β = 1.294, p < 0.001, se = 0.340), but the path from enjoyment to delayed posttest score was not significant . The immediate posttest score was also a significant antecedent to delayed posttest score .Finally, the model illustrates a significant path from interest to delayed posttest score  scored higher on an immediate posttest and a delayed posttest than students taking a virtual field trip presented as onscreen video . In addition, the HMD group produced higher ratings of presence, interest, and enjoyment than the video group. Overall, these findings support the immersion principle in multimedia learning  through immersive technology can influence learning through different affective and cognitive processes including enjoyment and interest. Together both enjoyment and interest are hallmarks of capable, confident and enthusiastic learners , and enjoyment (d = 1.10), and medium effect size advantage on the outcomes of interest (d = 0.57), immediate retention (d = 0.61), and delayed retention (d = 0.70). Although this provides evidence for the educational value of immersive technology, stakeholders such as Ana will also have to balance other important factors such as the cost of purchasing and maintaining equipment, time constraints, practical considerations such as how to administer a HMD experience to an entire class, as well as privacy and safety issues that have all been highlighted as limitations when using VR in the classroom  between the immediate and delayed posttest. This was important for the design of the study because the immersion principle would suggest that students in the HMD condition would exert more effort in those lessons based on having more situational interest than students in the video condition. It was not possible to measure effort during these lessons in the current context, but future research should attempt to obtain process measures to further investigate this hypothesis.In the current study the video condition was administered as a collective session while the HMD condition was administered as an individual session. This decision was made to maintain ecological validity of how the media would be used in a classroom. More specifically, the teachers determined that giving each student access to the video on their own device would cause more distractions than using a collective video. Regardless of how a video is presented there is the possibility that students communicate and disturb each other; however, observations of the session indicated that this was not a great problem as students generally were very focused on watching the video in this experiment. One affordance of using HMDs is that teachers are able to immerse students in the lesson, making it impossible to communicate and disturb each other, which is not possible with a video session. Nevertheless, future research should compare using a video or HMD in individual sessions.Although we included enjoyment and interest in the SEM; many other factors influence the process of learning in immersive lessons. Such factors include interactivity, cognitive load, self-regulation, and particular individual differences variables such as spatial skills, which should be included in future research. Future research could also investigate other media as a comparison condition to HMD including real field trips, books etc., as the results of the current study are focused on differences between experiencing a virtual field trip in HMD compared to a 2D video.In conclusion, we have found that a virtual field trip experienced through a HMD is superior to the same virtual field trip when presented as a 2D video on the outcomes of presence, enjoyment, interest, as well as short-term and long-term retention in the context of a middle school IBSL climate change intervention. The SEM model showed that enjoyment directly mediates the results on the immediate posttest but not the delayed posttest. Alternatively, interest directly mediates the results on the delayed posttest test but does not the immediate posttest. The research provides additional support of the immersion principle in multimedia learning and initial findings that offer a better idea of how immersive lessons can influence long-term learning outcomes. We encourage future research to further test the cognitive affective model of immersive learning presented in this article and add to the model to eventually gain a stronger understanding of the variables and processes that play a role in learning with immersive technology.


To verify a test prototype of a novel flexible video laryngoscope in a difficult airway management simulator and to compare the efficacy of the flexible video laryngoscope with that of a conventional video laryngoscope.Fifteen clinical anesthesiologists performed endotracheal intubation with a flexible video laryngoscope and a conventional video laryngoscope in a difficult airway management simulator in the neutral position with intermediate and difficult mouth opening. The rate of intubation success, intubation time, and classification of glottic exposure were recorded. After endotracheal intubation, participants were asked to assess the difficulty of intubation of the two laryngoscopes.P = 0.025) and difficult (P = 0.005) mouth opening. The Cormack Lehane score of the flexible video laryngoscope was significantly lower than that of the video laryngoscope in the neutral position with intermediate mouth opening (P < 0.001) and difficult mouth opening (P < 0.001). There was no significant difference in intubation time in the neutral position with intermediate mouth opening (P = 0.460) or difficult mouth opening (P = 0.078). The difficulty score of endotracheal intubations with the flexible video laryngoscope was also significantly lower than that of the video laryngoscope in the neutral position with intermediate mouth opening (P = 0.001) and difficult mouth opening (P = 0.001).The success rate of endotracheal intubation with flexible video laryngoscope was significantly higher than that with video laryngoscope in neutral positions with both intermediate (Compared with conventional video laryngoscopy, flexible video laryngoscopy can provide superior glottic exposure and improve the success rate of intubation in a difficult airway management simulator.The online version contains supplementary material available at 10.1186/s12938-022-01043-1. Endotracheal intubation of patients with difficult airways has always been one of the most challenging tasks in clinical practice. Failed intubation can lead to airway injury and even death –3. PreviCurrent intubation tools mainly include direct laryngoscopes, visual laryngoscopes, visual rigid laryngoscopes, and flexible bronchoscopes. Among these intubation tools, the direct laryngoscope has long been the most commonly used, but it has inherent limitations because it requires a direct line of sight between the operator’s eyes and the glottis , which mAs an alternative to direct laryngoscope, video laryngoscope has developed rapidly in recent years. Video laryngoscopy relies on digital technology to transmit an image from the tip of the laryngoscope to a monitor, so it does not require a direct line of sight between the operator’s eyes and the glottis. Compared to the conventional direct laryngoscope, the video laryngoscope has a wide field of view and can clearly display the structure of the pharyngeal cavity. Previous studies have shown that visual laryngoscopy has a higher intubation success rate than direct laryngoscopy –11. At pAnother type of laryngoscope is the visual rigid laryngoscope, which mainly includes the Trachway video intubating stylet, Levitan FPS optical stylet, MultiViewScope, Optiscope video stylet, etc. –18. As aAs the distal end of the insertion part of the flexible bronchoscope can be flexibly changed to find the glottis by adjusting the control lever on the laryngoscope handle, it is a well-recognized airway management tool used in patients with difficult airways. However, the flexible bronchoscope is also a device for guiding intubation with endotracheal tubes, the insertion part of the flexible bronchoscope is thin, and the field of vision is confined. Sometimes it is difficult for the operator to distinguish the structure of the pharyngeal cavity. In addition, the suction function of the flexible bronchoscope is usually not efficient, and the lens is easily obscured by secretions or blood, resulting in blurred vision. Given these facts, a novel device that can ensure the efficacy of intubation in difficult airways is needed.As we have analyzed above, the previous intubation tools either have a wide field of view but are not flexible or are flexible but have a small field of view. Here, we introduce a novel difficult airway intubation tool designed and manufactured by us. This device keeps the basic structure of the flexible bronchoscope by making the flexible bronchoscope thicker and shorter, thus retaining its flexibility. A protrusion is added at the front end of the lens to increase its field of vision, thus inheriting the advantages of the visual laryngoscope with a wide field of view. Therefore, it combines the advantages of the "wide field of view" of the video laryngoscope and the "flexibility" of the flexible bronchoscope. At the same time, to enhance its suction function, the diameter of the insertion part and the suction channel were increased . As this modified laryngoscope can no longer guide intubation in endotracheal tubes like a flexible bronchoscope does, we installed an endotracheal tube guide device on one side of the laryngoscope. Through these reformations, the advantages of the previous intubation tools are retained, and their disadvantages are overcome.Currently, we have created the device as a test prototype and have named it a “flexible video laryngoscope (FVL)”. To further improve this instrument and prepare for clinical verification in the future, we carried out this preliminary study to verify the efficacy of flexible video laryngoscopy in difficult airway scenarios.A total of 15 participants were enrolled in this study. The details of the participant characteristics, such as sex, age, experience in anesthesia, experience in using a video laryngoscope and experience in using a flexible bronchoscope to perform intubation of difficult airway scenarios, are described in Table P = 0.025, 100% vs. 66.7%) or with difficult mouth opening .The success rate of endotracheal intubation with the FVL was significantly higher than that with a video laryngoscope, regardless of whether the patient was in a neutral position with intermediate mouth opening (P < 0.001) and difficult mouth opening (P < 0.001). There was no significant difference in intubation time in the neutral position with intermediate mouth opening (P = 0.460) or difficult mouth opening (P = 0.078). The difficulty score of endotracheal intubations with the FVL was also significantly lower than that of video laryngoscopy in the neutral position with intermediate mouth opening (P = 0.001) and difficult mouth opening (P = 0.001), as shown in Table The Cormack Lehane score of the FVL was significantly lower than that of the video laryngoscope in the neutral position with intermediate mouth opening (This study showed that the new flexible video laryngoscope (FVL) performs well in a difficult airway management simulator. All glottic exposures were grade 1, and the success rates for intubation were 100%, which were much higher than those of video laryngoscopy. The difficulty score of intubations with the FVL was significantly lower than that with the video laryngoscope. Although the intubation success rate of the video laryngoscope reached 46.7% in the neutral position with difficult mouth opening, the glottic exposure could only reach grade 3 (only the epiglottis but no portion of the glottis was visible). All tracheal tubes were blindly inserted into the glottis.At present, the main intubation tools used to address difficult airways are video laryngoscopy and flexible bronchoscopy. Video laryngoscopy has played an increasingly important role in airway management. Although there are many kinds of video laryngoscopes on the market, they mainly improve the exposure of the glottis by changing the shape and angle of the laryngoscope blade. Considering that sometimes the larger angulated blade shape cannot be inserted into the mouth of patients with their head in the neutral position and difficult mouth opening, we did not choose the larger angulated blade shape in this study. In addition, the visual laryngoscope we selected is most widely used in China.Because the insertion part of the flexible bronchoscope is flexible, it is suitable for various types of difficult airways. However, its disadvantages are also very prominent: it needs enough space in front of the lens; otherwise, the visual field will be small, and sometimes it is also difficult for the operator to distinguish the structure of the pharyngeal cavity, especially for those who are not skilled enough. In addition, the suction channel is slender, the suction function is poor, and the lens is easily obscured by secretions or blood.This new design inherits the basic structure and function of flexible bronchoscopy, that is, it retains its flexibility. On this basis, its insertion becomes thicker and shorter; consequently, the suction channel can be enlarged to improve its suction function, and the secretions in the oropharyngeal cavity can be removed rapidly to avoid lens contamination. After induction of anesthesia, induction agents and neuromuscular blocking drugs reduce the upper airway muscle tone, resulting in upper airway narrowing and obstruction. By adding a protrusion at the front end of the insertion part, the collapsed pharyngeal cavity tissue can be lifted, so the space of the pharyngeal cavity expands, which is helpful for the operator to visualize the structure of the pharyngeal cavity. Because the lifting force of the front end of the insertion part is not very strong, the protrusion cannot lift the epiglottis from the epiglottic valley in this difficult airway management simulator; it needs to go underneath the epiglottis and lift the epiglottis to expose the glottis.Since the insertion part of the FVL is thick and equipped with a protrusion at the front end of the insertion part, which makes it easier to expose the glottis, it cannot be placed inside the ETT as a conventional flexible bronchoscope that can be inserted into the trachea together with the ETT. We inserted the ETT by a guiding tunnel beside the insertion part and a matched guiding tube and stylet. This method changes the traditional idea of guidance within the ETT, which eliminates the ETT’s constraints on the insertion part’s diameter.Both the stylet and the guiding tube have certain flexibility. The reason why we use the stylet to guide the guiding tube is to make the guiding tunnel as slender as possible, so the insertion part will not occupy too much space in the pharyngeal cavity. Of course, the diameter of the guiding channel can be appropriately increased, so that the guiding tube can enter the glottis directly through the guiding channel. In this way, the operation of the stylet can be omitted, and the steps are reduced.In this study, we used a reinforced endotracheal tube. In the head neutral position, the endotracheal tube has a large curvature (approximately 90 degrees) when passing through the oral cavity to the posterior pharyngeal wall. The reinforced endotracheal tube is softer, and the shape is easier to change. It is easier to pass through the oropharyngeal cavity when sliding along the stylet.In addition, the FVL has a ventilation channel, which can be connected to a high-flow oxygen source during intubation or to a high-frequency ventilator for high-frequency ventilation during intubation.In this study, we did not compare this design with flexible bronchoscopy because the advantages of the FVL cannot be fully verified in a difficult airway management simulator. For instance, the function of suction and the enlargement of visualization of pharyngeal cavity needs to be verified clinically.We want to emphasize that this FVL is only a prototype, and some details have not been completely finalized. Although this difficult airway management simulator can better simulate the situation of difficult airways by changing the mouth opening and neck movement and reliably verify the performance of the FVL in difficult airways, the real situation in the clinic is much more complicated. For example, the narrowing and obstruction of the upper airway after the induction of anesthesia, the interference of secretions on the lens, etc., all need to be further verified clinically.This study showed that the new flexible video laryngoscope performs well in a difficult airway management simulator. All glottic exposures were grade 1, and the success rate of intubation was 100%, which was much higher than that of video laryngoscopy. Moreover, the difficulty score of FVL intubation was significantly lower than that of video laryngoscopy.The local ethics committee  determined that the verification of this novel flexible video laryngoscope and video laryngoscope in the difficult airway management simulator did not require ethical approval. Fifteen anesthesiologists who had worked for more than 5 years were enrolled in the study with written consent for the anonymous use of their data and completed a questionnaire about their personal information and experience in airway management.A difficult airway management simulator .The rate of successful intubation was tested by the McNemar test. The comparison of intubation time, the difficulty score of intubations and the Cormack–Lehane score was compared by the Wilcoxon test. For all comparisons, a β = 0.1 and a = 0.05, a sample size of no less than 6 participants was needed.According to the preliminary experiment in the neutral position with difficult mouth opening, based on the sample size estimation of the paired design, the success rate of endotracheal intubation with FVL was 100%, and the success rate of endotracheal intubation with video laryngoscopy was 33.3%. For a two-tailed comparison, with Additional file 1. The intubation video of the flexible video laryngoscope.


The majority of online videos about shoulder injection treatment showed low reliability, overall quality, and educational value. Additionally, a new scoring system is required to accurately evaluate musculoskeletal injection videos for educational purposes.This study aimed to evaluate the reliability, overall quality, and educational value of online videos for learning the techniques related to shoulder injection treatments and analyzing the usability of video evaluation tools for musculoskeletal injections. Online video searches were performed in February 2022 using the terms “shoulder injection”, “glenohumeral joint injection”, “acromioclavicular joint injection”, and “subacromial bursa injection.” Included videos were scored by modified DISCERN (mDISCERN), global quality score (GQS), and shoulder injection score (SIS). Correlations between scoring systems were analyzed. Of the 150 videos, 49 (32.67%) contained highly reliable information. Regarding the assessment of overall quality by the GQS, 109 (72.67%) videos were of low quality. Regarding SIS, 114 (76.00%) scored not >5, of which 77 (51.33%) scored <3. Most of the SIS domains were fully explained in <40% of the included videos. A weak positive relationship was noted between the mDISCERN and SIS (r Shoulder pain is one of the most prevalent problems seen in primary care. Shoulder disorders are considered the third-most common cause of musculoskeletal problems and affect approximately one quarter of the worldwide population . These dHowever, physicians have reported low confidence in performing injection treatments owing to inadequate training ,5. JointRecently, online videos have emerged as a good alternative for physicians to learn clinical skills with the advancement of the internet . The intHowever, previous studies evaluating the quality of medical information on social media have mainly focused on medical videos for patients . There wAlthough some studies that have evaluated online videos have developed novel evaluation tools , a majorTherefore, we tried to evaluate the reliability, overall quality, and educational comprehensiveness of online videos on YouTube and Google, with the goals of educating medical professionals on shoulder injections and identifying whether the current evaluation tools for online videos are appropriate for shoulder injection techniques for medical professionals ,17,18.We selected the following three specific shoulder injection techniques: glenohumeral joint injection, acromioclavicular joint injection, and subacromial bursa injection. These are commonly used injection techniques for patients with shoulder pain . To inclThe exclusion criteria for videos were as follows: duplicate videos, videos uploaded before 1 January 2017, or over 5 years prior to the study, and non-English videos. Irrelevant videos  and very low-quality videos to score were also removed by authors’ judgments. The remaining videos were included for further analyses. A flowchart of the video selection process is shown in The videos searched on YouTube and Google that were evaluated for this study were accessible to everyone. Ethics committee approval was not required because this study did not include human participants or animals.On the day of the search, the basic descriptive characteristics of each video were collected. These included the title, uploader, and uniform resource locator, numbers of views, likes, and comments, duration (seconds), and posting days. Since November 2021, YouTube has not released the number of dislikes to ordinary viewers; therefore, we were unable to collect them. The number of views, likes, and comments of videos identified through Google searches, except for videos that were also searched on YouTube, were not collected owing to the lack of information. Based on the method described in a previous study, the viewing index  was calculated whenever possible . It. It18]. The SIS was developed by the authors of the current study to evaluate whether a beginner physician who has not previously performed shoulder injections, or who has less experience, can learn shoulder injection techniques through videos. The SIS contains the following 10 domains: indication, needle selection, injection materials, patient position, surface anatomy, sterilization, needle approach and insertion, injection target, contraindication and caution, and post-injection management . Each doTwo independent reviewers (H.E.C. and C.W.J.), who specialize in physical medicine and rehabilitation and have more than seven clinical years of shoulder injection treatments, evaluated each video using mDISCERN, GQS, and SIS after being trained to analyze the video in the same way. The content and information in each video’s footage were reviewed. If the video contained more than one shoulder injection technique, only the part of the test that was intended to be evaluated was assessed. Discrepancies in scores for the same video between reviewers were resolved by consensus until an agreement was reached.p > 0.6. All analyses were performed using the RStudio software (R version 4.1.2). Statistical significance was set at p value < 0.05 for parameters other than the interrater reliability.Descriptive data are presented as numbers (percentages) and means ± standard deviations. The Shapiro–Wilk test was applied to approximate the normality of data, and the Wilcoxon rank–sum test was performed to investigate the association between two groups. Pearson’s correlation coefficient was used to determine the relationship between mDISCERN and SIS, as well as GQS and SIS. Interrater reliability was measured separately for the scoring of the mDISCERN, GQS, and SIS using Cohen’s weighted kappa coefficient, with significance set at A total of 1455 and 261 results were identified using YouTube and Google searches, respectively. After applying the exclusion criteria, 150 videos were selected and analyzed . The basThe mean numbers of views and posting days were 6664.86 ± 26,270.10 and 808.30 ± 490.44 days, respectively. The mean VI was 7.57 ± 26.72. The mean numbers of likes and comments on the video were 56.37 ± 242.60 and 4.82 ± 24.38, respectively. The mean duration of the video was 167.92 ± 292.51 s. The average reliability (mDISCERN), overall quality (GQS), and educational comprehensiveness (SIS) scores of all videos were 2.01 ± 1.15, 1.88 ± 1.04, and 3.07 ± 2.91, respectively. The kappa scores indicated good agreement between the reviewers, showing that the interrater reliabilities for mDISCERN score, GQS, and SIS were 0.83, 0.85, and 0.83, respectively.Of the 150 videos selected, 49 (32.67%) contained highly reliable information . Regarding the assessment of overall quality by the GQS, 41 (27.33%) videos were of moderate , or high  quality, which indicates that 109 (72.67%) videos were of low quality. Regarding the SIS, 114 (76.00%) videos scored no more than 5, with 77 of these (51.33%) scoring <3. Only 15 (10.00%) videos had a score of more than 7 . Except for the patient position (66.67%), the remaining domains have been fully explained in <40% of the included videos .In particular, 19 (12.67%), 28 (18.67%), 33 (22%), 29 (19.33%), and 36 (24.00%) videos contained detailed information on post-injection management, information about needle selection, injection materials, explanation of surface anatomy, and proper sterilization, respectively. The mean scores of each SIS domain are summarized in The interrater reliability was significant for all scoring systems .p = 0.03) and mean duration (p < 0.01).Most videos (68.67%) covered the contents of the image-guided shoulder injection treatment, whereas the remainder covered the contents of the blinded shoulder injection treatment. When comparing the two groups, the Wilcoxon rank–sum test revealed significant differences in the mean VI (p < 0.01). A significant difference was noted in the mean GQS, representing the overall quality of the video . The mean SIS of the blinded injection group (5.34 ± 2.55) was significantly higher than that of the image-guided injection group (2.01 ± 2.46) (p < 0.001). The mean scores for each domain are presented in The mean mDISCERN values for the blinded injection and image-guided injection groups were 2.38 ± 1.03 and 1.84 ± 1.17, respectively  were from the medical professional group, and 22 videos (15%) were from the non-medical professional group. Significant differences were found in the VI, views, and likes between the two groups.p = 0.001 for mDISCERN, p = 0.004 for GQS, and p = 0.001 for SIS). The mean scores for each domain of SIS are also presented in Results of all three scoring systems were significantly higher in the videos of medical professionals than those of non-medical professionals (2 = 0.38). Moreover, a moderate positive relationship was observed between the GQS and SIS (r2 = 0.49). The correlation coefficient by domain is summarized in Among the 10 domains, correlation coefficients of 0.5 or higher in both comparisons are indication, needle approach and insertion, and contraindications or cautions. Meanwhile, the domains that showed a weak positive correlation between the mDISCERN and SIS were needle selection, injection material, surface anatomy, sterilization, and injection target. Needle selection, surface anatomy, and post-injection management showed a weak positive correlation between the GQS and SIS.In this study, we assessed the reliability, overall quality, and educational comprehensiveness of all videos on shoulder injection treatments uploaded within 5 years. We found that only 32.7% of the videos contained highly reliable information, and 27.3% contained only 10% of high-quality videos. Only one video received five points in the mDISCERN, and no videos received five points in the GQS. Moreover, we found that the majority of videos on social media did not contain sufficient information to educate beginner physicians on shoulder injection techniques. Practical aspects, such as detailed information on which needle to choose, how to prepare the medicine for injections, and where to inject on the surface, were often neglected.Several studies analyzed the videos of other clinical procedures on YouTube. In a study in which a total of 50 videos of epidural steroid injection were examined from the patient’s point of view, only 22% of the videos had highly reliable information, and only 34% were moderate to excellent in quality . SimilarInterestingly, when broken down by the uploader type, the VI and likes of the medical professional group were significantly higher than those of the non-medical professional group. Although not statistically significant, the number of comments was higher in the medical professional group. This was consistent with the results of previous studies ,31. ConsAccording to a review paper, pre-existing tools such as the mDISCERN and the GQS, which were not initially designed for video assessment, are still being used frequently ,15. The One notable finding from our study is that videos of the image-guided injection group tended to have lower educational comprehensiveness than those of the blinded injection group. We suppose that this is probably because video providers assume that viewers who make an effort to search for those videos recognize basic information regarding shoulder injections. However, considering this, the videos are insensitive. Several videos about image-guided injection only focused on showing the procedure itself, without providing explanations. Basic injection sites, other than the description of image-guided injection, were often ignored. Furthermore, descriptions of the image, such as the anatomy of structures acquired in an image and how to obtain such images, were often neglected. Since image acquisition and description are essential for image-guided injection, the video cannot be considered educationally appropriate without at least explaining them. Providers of videos related to image-guided injection should make more efforts to create higher-quality videos.Our study had an inherent limitation. The SIS was self-produced by the authors and did not receive peer review from other researchers. Although the authors have numerous musculoskeletal injection training experiences as experts who have been performing musculoskeletal injections for >7–20 years at university hospitals, more systematic and reliable injection evaluation tools need to be developed in the future. Another limitation is that we did not analyze all educational videos related to shoulder injection on the internet. Although we searched videos using four different keywords on two of the most popular platforms, Google and YouTube, there can be other educational platforms that may contain video materials that we could not include. Considering the dynamic nature of online platforms, many educational videos are produced and posted in real-time, and future studies including more videos are needed.The majority of social media videos for shoulder injections, especially those about image-guided injections, were not suitable for educating beginner physicians due to their low educational comprehensiveness. Regardless of whether the videos are adequate for educational purposes, viewers prefer those produced by medical professionals. Therefore, medical professionals should pay more attention to uploading comprehensive high-quality videos for educational purposes. Meanwhile, the most used pre-existing tools, mDISCERN and GQS, were not enough to assess the educational comprehensiveness of social media videos. This allowed us to identify the need for a new scoring system to accurately evaluate musculoskeletal injection videos for education purposes. Future research needs to evaluate videos regarding other injections, and examine the development and demonstration of new evaluation tools.