Eyewitness misidentification accounts for 70% of verified erroneous convictions. To address this alarming phenomenon, research has focused on factors that influence likelihood of correct identification, such as the manner in which a lineup is conducted. Traditional lineups rely on overt eyewitness responses that confound two covert factors: strength of recognition memory and the criterion for deciding what memory strength is sufficient for identification. Here we describe a lineup that permits estimation of memory strength independent of decision criterion. Our procedure employs powerful techniques developed in studies of perception and memory: perceptual scaling and signal detection analysis. Using these tools, we scale memory strengths elicited by lineup faces, and quantify performance of a binary classifier tasked with distinguishing perpetrator from innocent suspect. This approach reveals structure of memory inaccessible using traditional lineups and renders accurate identifications uninfluenced by decision bias. The approach furthermore yields a quantitative index of individual eyewitness performance. Eyewitness errors contribute to wrongful convictions. Here, the authors present a lineup procedure that reveals the structure of eyewitness memory, reduces decision bias, and measures performance of individual witnesses. For example, more than 350 people, many serving lengthy prison sentences, have been exonerated in the United States because their DNA was found to be incompatible with evidence collected from the crime scene. In ~70% of these cases, misidentification by eyewitnesses contributed significantly as evidence for conviction2.Eyewitness identification has long played a valuable role in criminal investigations and prosecutions. Despite this value, our society has been confronted in recent years with many rank failures of eyewitness testimony4. A major focus of research on this topic has been the manner in which an eyewitness lineup is presented11. The traditional simultaneous (SIM) lineup is composed of  six facial photographs shown at the same time  confounds the strength of the recognition memory signal with the decision criterion, leaving the outcome susceptible to unrecognized bias.The performance of eyewitnesses under the SIM and SEQ paradigms reflects recognition memory12. This approach has enabled some important insights into eyewitness memory under different lineup conditions16. A desirable alternative is to estimate recognition memory signals themselves, which would enable criterion-independent analyses of the relative strengths of these signals. We propose here a lineup procedure that does so. Our procedure employs two powerful experimental techniques that are rooted in the history of scientific study of perception and memory: perceptual scaling and a signal detection method known as receiver operating characteristic (ROC) analysis.Recent laboratory studies of SIM and SEQ lineups have attempted to overcome this problem using expressed confidence as a proxy for the eyewitness decision criterion or by introducing explicit biasing instructions17, they have not been applied jointly to the eyewitness identification problem.Broadly considered, the goal of perceptual scaling is to map the relationship between a set of physical stimuli and the corresponding responses of an observer’s perceptual system. The goal of signal detection analysis in this context is to determine the optimal performance of a perceptual system tasked with classification of a set of stimuli. Used together, these tools enable us to scale the strengths of recognition memory signals elicited by lineup faces, and to quantify the best possible discriminability of those signals. Although these tools have been previously combined in laboratory studies of perceptual discriminationWe utilize this approach to reveal detailed structure of eyewitness recognition memories for a complete set of lineup faces. We then use these memory signals to measure how well an optimal statistical classifier can distinguish the perpetrator from an innocent suspect. Performance of the optimal classifier compares well to traditional lineup procedures, with the added benefits that identification decisions made by the classifier are not subject to unrecognized bias, that it renders a numerical index of performance for individual eyewitnesses, and that lineup filler choices are quantitatively assessed for fairness. For all of these reasons, this approach has enormous potential as a research tool for evaluating effects of other variables on eyewitness performance and as a practical tool for unbiased investigation and prosecution of crimes.18, which is a longstanding behavioral technique designed to scale stimuli along a psychological continuum, such as perceptual similarity to a target stimulus. In essence, the method estimates the central tendency and variance of the perceptual signals elicited by each stimulus, from which it is possible to draw criterion-independent conclusions about perceived similarity. This method has been used in many practical applications, such as product marketing and preference testing20, optometric assessment (“which is clearer: lens one or lens two?”)21, and hearing-aid evaluation22. In cases where the target exists in memory alone, such as with eyewitness identification, the method estimates the central tendency and variance of recognition memory signals elicited by the cue stimuli.Our scaling procedure employs Louis Thurstone’s Method of Paired ComparisonsN = 15) of six face images (designated F1–F6), one of which (F1) was the suspect  lineup consisted of three presentations of all possible pairings  condition the target (F1) was included in the lineup, along with five fillers who matched the physical description of the perpetrator. The target-absent (TA) condition differed from the target-present condition solely in that the target was removed and replaced by another face (F1*) that matched the physical description of the perpetrator. PAR lineups were administered for both TA and TP conditions and the perceptual scaling analyses described below were performed separately for each condition. Results of scaling in TA and TP conditions were subsequently combined using a signal detection procedure described in the section “Signal detection analysis of recognition memory signals” below. This procedure yielded a criterion-independent measure of the probability that an optimal binary classifier can distinguish between the perpetrator and an innocent suspect.Superior lineups are naturally defined as those that lead to a high probability of correctly identifying the perpetrator (target) and a low probability of incorrectly identifying an innocent suspectn = 20; 32.26%), significantly exceeding chance performance (p = 0.016). By this simple measure, the target face was “identified” in the PAR lineup at a rate comparable to that in the SIM  was significantly lower than in the traditional lineups .We used our scaling procedure initially to determine the top-ranked face within the set of six TP PAR lineup faces Fig. , which iSIM Fig.  and SEQ SIM Fig.  procedurAlthough generally supportive of the paired comparison approach, this top-rank frequency analysis neglects the richness of data produced by perceptual scaling and reveals nothing about the consistency of recognition memory signals elicited by each face or the degree to which signals vary between faces. In particular, top-ranked status alone does not address the extent to which that face is scaled significantly above all of the other faces in the lineup.N = 62). Average voting scores are plotted in Fig. p = 0.00002), on average, which reflects the fact that this target face consistently scored highly, even though it was not the top-ranked face for every subject.To illustrate these additional features of the PAR data, we asked what the relative rankings were within the set of six TP lineup faces for the entire population of subjects assigned to that condition N = 6. Averagep < 10−6). Notably, the average score for target face F1 was also highest amongst subjects who performed incorrectly . This counterintuitive finding reflects high consistency of target face judgments among our subjects. (Subjects in the incorrect group often ranked face F1 as their second choice.) More generally, the observed scaling of voting scores reveals informative structure of recognition memory signals not accessible using traditional lineups.We further divided these data by correctness of subject responses. Voting scores for subjects who preferred target face F1 (correct subjects) are plotted in Fig. tly Fig.  . Average voting scores are plotted in Fig. The target-absent condition was composed of a PAR lineup identical to the target-present condition, with the exception that the target face was removed and replaced by another face (F1*) that matched the physical description of the perpetrator. The purpose of this manipulation was to determine, in conjunction with the TP condition, the relative probabilities of selecting the target vs. a non-target face. As for the TP condition, we asked what the relative rankings were within the set of six TA lineup faces for the entire population of subjects assigned to that condition (p = 0.036). We observed only one significant scaling difference between voting scores in TP and TA conditions: face F4 was scaled higher than F2 in the TP condition and the reverse was true in the TA condition. In both conditions, F2 and F4 were the filler faces ranked least similar to (and thus least confusable with) the perpetrator.The relative voting score distributions for the remaining TA filler faces were also similar to those seen for the TP conditions  We applied this method and assessed classification performance for every decision criterion, thus deriving a criterion-independent measure of the ability of an optimal statistical classifier to make lineup identification decisions based on perceptual scaling data from a population of witnesses. Because of measurement uncertainty, performance of the optimal classifier will always be less than perfect, but it is the best performance that can be achieved with the information available.Our perceptual scaling data provide insights into the structure of recognition memory without asking each subject to make a unique choice and thus without engaging a decision criterion for identification. Using estimates of central tendency and variance obtained from perceptual scaling, identification is elegantly redefined as a problem of statistical inference. The distributions of votes cast for each lineup face Figs.  and 6 ca29, which is a method to visualize relative probabilities of correct vs. incorrect identifications as a function of decision criterion.  While ROC curves have been employed previously to evaluate eyewitness performance with traditional SIM and SEQ lineups31, our use of this approach is distinctive because it is applied to a complete set of recognition memory signals, derived from each subject in a manner that avoids decision bias. By contrast, traditional lineups must integrate decisions across subjects and often use expressed confidence as a proxy for decision criterion.The outcome of our signal detection analysis is conveyed in the form of ROCIn the next section, we present ROC curves obtained from these analyses. These curves fall into two categories: those derived by averaging across the subject population and those reflecting individual subject performance.Our population analyses are based upon data obtained from a large number of face pairings presented to each of many subjects. These analyses reveal general tendencies in the data in a form that facilitates comparison with traditional lineup studies, which necessarily yield population-based ROCs.8, the six face-specific probability measures were then averaged to yield the probability of incorrectly classifying a TA lineup face as target.We first applied our signal detection analysis to perceptual scaling data from both TP and TA lineups. For the TP condition we determined for every decision criterion the probability of correctly classifying the target face as target. In the TA condition each of the six lineup faces was drawn from the same parent distribution, in the sense that they were all chosen to match the physical description of the perpetrator. For this reason, there was no single face designated as the innocent suspect; we considered the likelihood that any one of the faces in the TA lineup could be identified incorrectly. We thus determined separately for each face the probability (for each decision criterion) of incorrectly identifying that face. Following the same protocol used for this purpose in traditional SIM and SEQ lineupsp < 10−6) is a criterion-free index of the degree to which an optimal statistical classifier is capable of correctly distinguishing between the perpetrator and an innocent suspect based on scaling data from the entire subject population. The recognition ROC conveys the same performance metrics that are commonly reported in studies of traditional SIM and SEQ lineups, but does so by exploiting the full structure of recognition memory for lineup faces, while at the same time avoiding the ambiguity of witness’ decision criteria for identification.Correct and incorrect classification probabilities, computed as described above for each decision criterion, are plotted on the ordinate and abscissa of the ROC shown in Fig. 8). The sequential and simultaneous lineup results from this earlier study are plotted together with our recognition ROC in Fig. With these PAR lineup performance metrics in hand, we can compare the outcome of our approach to published data on performance obtained using traditional lineups. As noted above, we found our lineup to be unfair based on perceptual scaling, since there are significant differences between the responses elicited by our lineup faces Figs.  and 6. WWith the exception of the target face, all faces included in TA and TP lineups were drawn from the same parent distribution of faces, in the sense that they were all chosen to match the physical description of the perpetrator. Incorrect identification of a filler in the TP lineup should on average occur with the same likelihood as incorrect identification of any face in the TA lineup, provided that the presence or absence of the target in the lineup does not alter the scaling of the remaining faces.p = 0.036). We did observe one order reversal between the two conditions  predict that the presence or absence of the target in the lineup should not alter perceptual scaling of the remaining faces. Consistent with this prediction, the correlation of voting scores for the five non-target faces that are common to both TP and TA conditions was high .Results for individual subjects obtained using the simulated TA condition are illustrated in the form of recognition ROCs Fig.  for the 15. The PAR lineup, by contrast, yields an objective quantitative index of performance, based on discriminability of recognition memory signals elicited by lineup faces. The PAR lineup can therefore be used to compare performance of individual eyewitnesses in a criterion-independent fashion. In particular, the method offers an objective performance criterion by which witnesses can be triaged based on their certainty.Measures of criterion-independent discriminability are not obtainable from individual subjects  using traditional lineups. Inferences about the correctness of identification in such lineups are often derived from witness statements of confidence, based on the observation that a confident witness is more likely to make an accurate identificationThe forgoing analyses suggest several key benefits of the PAR procedure relative to traditional SIM and SEQ lineups. All of these benefits are tied to the fact that the PAR procedure is not limited to a single report of identification, but rather renders rich quantitative detail about the structure of a witness’ recognition memory for all lineup faces. Application of signal detection analysis to the scaled relationship between lineup faces and memory allows us to circumvent covert decision biases and render criterion-independent measures of discriminability. Results of our initial analysis of PAR population data are comparable with analyses of population data obtained using traditional lineups, though we stress the critical role of lineup fairness in making such comparisons. Indeed, one of the key features of our perceptual scaling approach is that it provides an immediate quantitative measure of lineup fairness, which we propose as a valuable tool for future studies of eyewitness performance.Another important benefit of the PAR approach is that performance measures can be computed directly from the distributions of voting scores obtained using a single target-present lineup. We suggest that this new capability can eliminate the need  for testing an additional group of subjects who view only target-absent lineups in order to obtain a measure of incorrect identifications. Our findings support this proposal. Finally, our approach is not limited to population data analysis, as is the case for traditional procedures. Rather, the PAR lineup delivers a quantitative measure of performance based on individual eyewitnesses, which could be of enormous utility in assessing the value of individual eyewitness evidence.33. In this sense, using signal detection analysis to compare recognition memory signals is directly analogous, for example, to a forensic fingerprint examiner’s use of standardized statistical methods for comparison of fingerprints34.For all of these reasons, we suggest that the PAR procedure may prove useful both as a research tool for evaluating effects of other variables on eyewitness performance and as a practical tool for investigation and prosecution of crimes. In a court of law, we envision that the PAR approach would augment direct testimony of an eyewitness with expert testimony on the classification of stimuli, as is routinely done in other forensic disciplinesDespite its many merits, there are potential concerns associated with the PAR procedure. For one, this procedure requires multiple viewings of each facial photograph, which could lead to new face memories that confound or compete with that of the perpetrator, and thus interfere with comparative judgments over the course of the lineup test. We believe such effects to be minimal or nonexistent because all faces appeared equally often in the PAR lineup and performance was comparable to that seen with traditional lineups.A second potentially problematic feature of the PAR lineup is the fact that the witness never explicitly identifies the perpetrator. It is easy to imagine defense counsel’s vociferous objection on these grounds. Rather than a scientific problem, this concern reflects a deep-seated cultural problem associated with our criminal justice system. The problem stems from naïveté about how people make decisions, which has long bedeviled eyewitness testimony and rules for reporting and testimony by forensic examiners. In particular, human decisions based on sensory information are necessarily probabilistic. A witness viewing a traditional SIM or SEQ lineup always confronts some uncertainty, but the decision is often rendered and interpreted as certain. Yet because the decision criterion employed by an eyewitness viewing a traditional lineup is unknown, and potentially influenced by undetermined perceptual, social or cultural factors, we have no immediate knowledge of the amount of uncertainty that the witness was willing to bear in making an identification. In other words, the longstanding practice of encouraging witnesses to say “he’s the one” is where the problem lies.The PAR lineup offers a practical solution to this cultural problem. It cannot overcome the limits imposed by uncertainty, but it acknowledges this uncertainty and offers a quantitative index of it. A jury presented with this numerical evidence is in a more informed position to determine guilt, as compared to a jury presented with the faux certainty of a traditional lineup identification.The experiment consisted of two parts. In the first part, every subject was presented with the same video recording of a mock crime: a 42-s fragment from the theatrical film God Bless America . In the second part, conducted on the following day, each subject was presented with one of three lineup types. Subjects viewed the same six lineup faces, regardless of lineup type. One of the lineup faces was that of the actor who played the role of perpetrator in the crime video. This face was designated the target and lineups that included this face were termed target-present. The other lineup faces were fillers selected based on certain attributes  that matched the description of the perpetrator. In all three lineup types, subjects were told that the target may or may not be present in the lineup. As noted below, some subjects in the PAR lineup condition viewed a target-absent lineup, in which the target face was replaced with another face that matched the description of the perpetrator.The simultaneous (SIM) lineup Fig.  consiste7.The sequential (SEQ) lineup Fig.  consisteThe paired comparison (PAR) lineup Fig.  consisteTraditional simultaneous and sequential lineups were included in this study solely for the purpose of comparing the relative frequencies of top-ranked faces Fig.  across tWe applied the PAR lineup procedure in two experimental conditions: target-present (TP) and target-absent (TA). The TP lineup was composed of the target face and five fillers that matched the description of the perpetrator. The TA lineup was identical to the target-present lineup, with the exception that the target face was replaced with another face that matched the description of the perpetrator.A lineup was “rejected” when the witness failed to identify a single face in the lineup. In our study, SIM and SEQ lineups were rejected when the subject did not explicitly select a face. The PAR lineup was rejected when the voting scores of two or more face images tied for the top rank.We employed two established experimental techniques to analyze data from PAR lineups: perceptual scaling and signal detection analysis. The former allowed us to estimate the recognition memory signals elicited by each lineup face and the latter enabled us to determine the best possible discriminability of the perpetrator relative to an innocent suspect.The complete set of votes collected from each subject was analyzed as shown in Fig. To evaluate whether differences between voting scores for different face images were statistically significant, we performed bootstrap analysis of the voting score distributions in every subject group . The probability of obtaining a difference of voting scores by chance was estimated by subtracting the higher-scores distribution from the lower-scores distribution and computing the fraction of negative values of the difference.We assessed discriminability of the perpetrator vs. an innocent suspect using a standard process of statistical inference. As noted in the main text, voting score distributions serve as estimates of the recognition memory signals elicited by lineup faces. These distributions were used to compute the relative probabilities of correctly classifying the target as target (correct identification rate) and incorrectly classifying a non-target as target , as a function of decision criterion.The results of our signal detection analyses are conveyed in the form of receiver operating characteristic (ROC) curves, which plot as a function of decision criterion the probability of correctly classifying the perpetrator vs. the probability of incorrectly classifying an innocent suspect. Signal detection analyses were carried out and ROC curves were produced separately for population and for individual subject data.29, which is by moving the decision criterion along the dimension underlying the voting score distributions and computing corresponding classification probabilities for the perpetrator  and for innocent suspects . Hit rate was estimated by computing the fraction of correct identifications of the target under each decision criterion in the population distribution of votes in the TP condition. False alarm rate was estimated by computing the fraction of incorrect identification of a non-target under each decision criterion in the population distribution of votes in the TA condition. These classification probabilities are integrated into the area under the ROC curve (AUC), which thus serves as a criterion-independent index of the degree to which an optimal classifier can correctly distinguish target and non-target faces based on the distributions of population voting scores.Using data from the aforementioned target-present and target-absent conditions, we determined the population ROC curve by standard methodsWe determined the error of estimating the population ROC curve using bootstrap analysis. We resampled the voting scores making up the two distributions used to derive the ROC: one for the target in the TP condition and the other for non-targets in the TA condition. On each cycle of resampling, we derived a new ROC (resampled ROC) and computed its AUC. This way we obtained a distribution of resampled ROC curves and a distribution of resampled AUC indices. We used the distribution of resampled ROC curves to determine the 50 and 95% regions of error of ROC marked in Fig. As described in the main text, the voting scores for individual subjects can be used to estimate the means of recognition memory signals elicited by lineup faces, and the slope of the voting function can be used to quantify the variance associated with the recognition signals. From these statistical moments it is possible to compute classification probabilities for target and non-target faces.39. Paired comparison scaling procedures are among the few methods known to satisfy this LIIA requirement41. In other words, the outcome of a lineup with five non-targets should be unaffected by the presence or absence of the target (the top vote-getter in the population average). It follows that voting score distributions obtained from those five non-targets should enable us to evaluate the probability that a non-target is wrongly classified as target. This argument, together with supporting results summarized in the section “Individual subject analysis of eyewitness data” in the main text, justify our approach to derivation of individual ROC.As noted above, we derived the population ROC using the TP condition to determine the probability of correctly identifying the target, and using the TA condition to determine the probability of incorrectly identifying a non-target. Individual subjects, however, can only participate in either TP or TA conditions. We therefore derived single-subject ROCs using data from the TP condition to determine probability of correct identification, as before, but using a simulated TA condition to determine the probability of incorrect identification. In this simulated TA condition, the target face was removed from analysis. Removal of the target face is legitimate for this purpose if it does not alter scaling of the remaining lineup faces. Statistical decision theory has shown that removal of a voting option (the target in this case) from consideration is legitimate if it satisfies the stringent mathematical criterion of local independence of irrelevant alternatives (LIIA), in which scaling is robust over removal of the top vote-getterThis individual subject procedure is analogous to that used for population data, with two exceptions: first, that a simulated TA condition was used to estimate false alarm rates (as we explain in the main text); second, that the following procedure was used to compute mean and variance of voting scores for each face:Step 1. We set the means of the probability distributions μi, i = {1...6} to the subject’s six measured voting scores and assumed that (a) the variances σ2 of these distributions were the same for every face; (b) the distributions were normal, N. We estimated each subject’s recognition memory variance (σ2) in the next step.Step 2. We estimated each subject’s recognition memory variance (σ2) by matching the slope of the voting function produced by the model (defined in Step 1) to the slope of the measured voting function, as follows. Using the subject’s model, we simulated the PAR experiment assuming different values of σ. For every tested value of σ, we simulated the comparisons among 45 pairs of images. For each pair, we sampled one response from each of the two simulated distributions that corresponded to two images in the pair. The image whose sample had a larger value was judged as more similar to the target. By this process, we derived the simulated voting matrix and voting function for the subject, following the procedure described in Fig. σ, and found the value σ* for which the slope of the simulated voting function was most similar to the slope of the measured voting function for this subject. Hence, σ*2 was an estimate of the subject’s recognition memory variance.Step 3. As for population data, the target-present and the (simulated) target-absent analyses were applied to voting score distributions in order to determine, respectively, the probability that the target is correctly identified as target and the probability that a non-target is incorrectly identified as target.  These correct and incorrect probabilities were computed for each decision criterion and plotted as ROC curves. The AUC for each curve serves as a criterion-independent index of the degree to which the optimal classifier can correctly discriminate target and innocent faces based on the voting score distributions obtained from a single witness. As such, the individual-witness AUC is a quantitative index of individual witness performance.Similar to our analysis of population ROC curves, we used the distribution of individual ROC curves produced at Step 3 of the above procedure to determine the 50 and 95% regions of error of ROC marked in Fig. Mock crime and lineup face images were viewed in a quiet light-tight behavioral testing room. All visual stimuli were rendered in color on a computer monitor  and viewed from a distance of 62 cm using a chin rest. The video recording had the resolution of 1280 × 720 pixels at the screen size of 40 × 18 cm  and rendered at the rate of 25 frames/s. Face image resolution was 167 × 167 pixels. Each face image had a screen size of 7 × 7 cm . Experiments were controlled using the software platform MATLAB, Release 2017b, The MathWorks, Inc., Natick, Massachusetts, USA.Subjects were primarily (96%) undergraduate students from the University of California, San Diego, who received course credit for their participation. A small number of subjects (4%) were college-age individuals from the La Jolla community, who received monetary compensation for their participation. Two hundred and two (202) subjects participated in the experiments : 62 subjects were presented with the PAR target-present lineup and 70 subjects were presented with the PAR target-absent lineup; 33 subjects with the SIM lineup, and 37 subjects with the SEQ lineup. All subjects had normal or corrected-to-normal visual acuity.In accordance with ethical standards set by US laws and regulations, we protected the welfare, rights, and privacy of human subjects who participated in these experiments. Subjects were informed of their rights as experimental participants and they provided written consent. The human subject protocol was reviewed and approved by the Human Subjects Institutional Review Board of the Salk Institute for Biological Studies (protocol #17-0002).Further information on research design is available in the Peer Review FileReporting Summary


COVID-19 has thrust video consulting into the limelight, as health care practitioners worldwide shift to delivering care remotely. Evidence suggests that video consulting is acceptable, safe, and effective in selected conditions and settings. However, research to date has mostly focused on initial adoption, with limited consideration of how video consulting can be mainstreamed and sustained.This study sought to do the following: (1) review and synthesize reported opportunities, challenges, and lessons learned in the scale-up, spread, and sustainability of video consultations, and (2) identify transferable insights that can inform policy and practice.We identified papers through systematic searches in PubMed, CINAHL, and Web of Science. Included articles reported on synchronous, video-based consultations that had spread to more than one setting beyond an initial pilot or feasibility stage, and were published since 2010. We used the Nonadoption, Abandonment, and challenges to the Scale-up, Spread, and Sustainability (NASSS) framework to synthesize findings relating to 7 domains: an understanding of the health condition(s) for which video consultations were being used, the material properties of the technological platform and relevant peripherals, the value proposition for patients and developers, the role of the adopter system, organizational factors, wider macro-level considerations, and emergence over time.We identified 13 papers describing 10 different video consultation services in 6 regions, covering the following: (1) video-to-home services, connecting providers directly to the patient; (2) hub-and-spoke models, connecting a provider at a central hub to a patient at a rural center; and (3) large-scale top-down evaluations scaled up or spread across a national health administration. Services covered rehabilitation, geriatrics, cancer surgery, diabetes, and mental health, as well as general specialist care and primary care. Potential enablers of spread and scale-up included embedded leadership and the presence of a telehealth champion, appropriate reimbursement mechanisms, user-friendly technology, pre-existing staff relationships, and adaptation (of technology and services) over time. Challenges tended to be related to service development, such as the absence of a long-term strategic plan, resistance to change, cost and reimbursement issues, and the technical experience of staff. There was limited articulation of the challenges to scale-up and spread of video consultations. This was combined with a lack of theorization, with papers tending to view spread and scale-up as the sum of multiple technical implementations, rather than theorizing the distinct processes required to achieve widespread adoption.There remains a significant lack of evidence that can support the spread and scale-up of video consulting. Given the recent pace of change due to COVID-19, a more definitive evidence base is urgently needed to support global efforts and match enthusiasm for extending use. There is global interest in video consultation services, including Skype, Teams, FaceTime, and other Voice over Internet Protocol (VoIP) media, to facilitate synchronous patient-to-provider video communication in health care ,2. StudiEvidence on the use of video consultations in health care is mixed. There is a rapidly growing literature on the feasibility, safety, acceptability, and effectiveness of video consultations across clinical areas  includinA small number of studies have explored the technological, contextual, and practical challenges to be overcome if video consulting is to become more widespread. One multilevel qualitative study, conducted in the English National Health Service (NHS) and undertaken by our team, examined the development, implementation, and use of video consultation services . FocusedAppreciation of the potential for longer term sustainability of this new service model is crucial in the context of the unfolding COVID-19 pandemic, which has brought a rapid need for alternatives to face-to-face contact and hence for the spread and scale-up of video consultations. We therefore conducted a systematic review of the opportunities and challenges to widespread implementation  of video consultation services in health care, asking the following questions:What theoretical frameworks have been used in this literature and for what purpose?What opportunities and challenges have been identified in the literature on the spread, scale-up, and sustainability of video consultations?What transferable insights can be useful for policy and practice? What questions remain unanswered?Our main concern is to identify and evaluate challenges to the scale-up and spread of video consultations from the existing research literature and, from this, to inform the rollout and longer term sustainability of video consulting and the research agenda that can support it. Spread, scale-up, and sustainability are often used as interchangeable terms without a standard definition or adequate theorization see . In thisIn December 2018, we systematically searched 3 databases: PubMed, CINAHL via EBSCOhost, and Web of Science. The search was updated in March 2020. Our search strategies, developed with the help of a research librarian, used a mix of keywords, Medical Subject Headings (MeSH), and Major Headings (MH), as provided in Inclusion criteria are summarized in Search results were imported into Zotero. Duplicates were removed, while retaining information from each search and preventing false duplicates from being merged. Each record was first screened by title and abstract . Remaining articles were screened by reading the full texts , with any disputes resolved through discussion and consensus with JW. At each stage, articles were eliminated if they did not meet the inclusion criteria . Full-teWe extracted data into a Microsoft Excel (Microsoft Corp) spreadsheet. At a descriptive level, we extracted data on study location, setting, clinical focus, definition of telehealth, service model and type of technology, and research design . We extrWe worked inductively to surface the challenges to and opportunities for the spread and scale-up of video consulting across studies. Sensitized by the Nonadoption, Abandonment, and challenges to the Scale-up, Spread, and Sustainability (NASSS) of health and care technologies framework , we thenWe piloted and refined this process on one paper and then extended the process across the rest of our data set. Using content analysis, we compiled a descriptive overview of the opportunities and challenges identified in the papers, including frequency distributions. We then applied a more analytical lens to synthesize and interpret our findings using the NASSS framework .Initial searches identified 4484 unique articles published between January 2010 and December 2018, with 193 articles identified for full-text screening and 12 initially included in the final review. An updated search in March 2020 returned 185 new articles for screening, of which 5 were identified for full-text screening. One new relevant article was identified, bringing the total records included in the review to 13 .Diverse research designs were employed in the included studies, including interpretive case studies , structuThere was ambiguity in the use and variations of the term telehealth, with three studies giving no definition and others defining telehealth variably depending on study focus . The terPapers described 10 different video consultation services in 6 regions: Australia , Cabo VeThe different approaches and frameworks used in the papers reviewed reflect different logics of change. This has implications for the way spread, scale-up, and sustainability are operationalized, studied, and conveyed. Many of the articles included in the review emphasized a logic of change underpinned by implementation science . Bauer eThe above studies primarily report the use of frameworks as part of evaluating, rather than guiding, spread and scale-up efforts. Another two articles employed frameworks to support spread and scale-up efforts, although they did not draw on relevant approaches such as the Going to Full Scale framework, 3S infrastructure, or the Dynamic Sustainability framework -60. In aComplexity was mentioned in a few articles, although not as a key organizing framework. Alami et al  analyzedSocial science theory was also employed, although rarely in a highly theoretical mode. In their study on telemental health services for rural American Indian communities, Brooks et al  used difA total of 38 opportunities to scale-up, spread, and sustainability and 47 unique challenges were reported across the 13 articles. The most common opportunities included the availability of clinical and/or nonclinical telehealth champions or coordinators (n=8), provider acceptance (n=4), absence of billing or licensure restrictions (n=3), adequate funding (n=4), and strong interorganizational communications (n=4). The most common challenges were lack of technical telehealth-specific support for clinical staff (n=6), need for redevelopment of workflows and organizational routines (n=6), financial pressures (n=5), and lack of training (n=5).Not all articles explicitly reported opportunities and challenges: 10 (77%) reported both challenges and opportunities ,37-41,43We synthesized the data extracted from the 13 articles according to the 7 domains of the NASSS framework  Figure .This domain encompasses the clinical and sociocultural aspects of the health condition and associated comorbidities, acknowledging that not all individuals with the same condition would benefit equally from health technologies . The comOpportunities for and challenges to the spread and scale-up of video consulting by type of clinical condition were reported in detail in only one study ,38. For The technology domain encompasses the materials, data, knowledge, and supply features of the technology . This doBrooks et al  also repAcross all papers reviewed, available technologies for video consultations posed challenges for establishing remote models of care as well as spread and scale-up. Authors reported that support and high-quality training were necessary to achieve widespread growth ,32-43. TThe value proposition domain is concerned with whether or not the technology is worth developing or introducing for clinicians, patients, and suppliers . It can A weak or poorly articulated value proposition posed challenges for scale-up and spread (n=2). For example, Martinez et al  reportedThe upstream supply of technology  was reported as a facilitator by Greenhalgh et al , who expThe adopter system considers the staff, patients, and caregivers and their potential and desire to adopt and to continue to use a technology . Staff cSpread and scale-up were enabled by an adopter system with engaged and committed staff who were given protected time by the organization to conduct video consultations, as mentioned by Interian et al : “havingChallenges to and opportunities for widespread implementation for patients and caregivers were reported in 6 articles -34,38,42Despite the abundance of challenges to and opportunities for widespread implementation reported from the provider perspective, there was little in-depth consideration of the impact of patient and caregiver adoption on widespread implementation and how the work and acceptance of these groups may have influenced widespread and sustainable implementation. Of the articles that did report patient or caregiver challenges -34,38,42The organization domain considers the capacity of the organization to innovate, its readiness for change, the nature of the funding decision, the extent of change in routines, and the work needed to implement change in the organization as it relates to the new technology . These eComplexity in the organizational domain was managed more effectively when telehealth champions were available, as reported in 11 of the reviewed articles -37,39-43The wider context domain considers the political, regulatory, professional, and sociocultural aspects of video consulting implementation . These fEven when the political environment was more stable, papers reported that national strategies did not always emphasize the potential for scale-up and spread against other priorities. Take the following report from an evaluation in Norway, which suggests that national approaches “were mainly focused on messaging services and electronic exchange, while less attention was paid towards telemedicine services, especially videoconferencing” . There wAbsence of or ambiguity in reimbursement was another common contextual barrier that influenced wider spread of video consultations . In a UKThe final domain, embedding and adaptation over time, focuses on the scope for adaptation and the resilience of the organization in the face of implementation and potential spread and scale-up . One stuFactors impacting resilience, “the intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances so that it can sustain required operations, even after a major mishap or in the presence of continuous stress” , and adaThis systematic review contributes valuable insights about the potential for scale-up, spread, and sustainability of video consulting and a novel interpretation through the application of the NASSS framework. A key finding is the paucity of evidence thoughtfully articulating challenges to the scale-up and spread of video consulting, which sits uncomfortably alongside the current global enthusiasm for expanding the use of video consulting services. The review also reveals an absence of concrete operationalization and theorization of scale-up and spread, exemplified by the absence of analysis and concrete definitions and terminology, as well as a lack of appreciation of complexity . The articles view spread and scale-up as the sum of multiple implementations, rather than theorizing the distinct processes required to achieve widespread adoption.Findings from the review pointed to potential enablers of the spread and scale-up of video consulting services, notably regarding the influence of the presence of a telehealth champion, especially during initial spread , as wellThere are gaps in the existing evidence on the spread and scale-up of video consulting. This was particularly the case in relation to interorganizational networking, which is critical in enabling and sustaining innovation ,30,64, aAs patients, health care organizations, and nations continue to look toward video consultations, it is essential to continue to theorize this domain. The absence of consistent theorization was demonstrated in this review by the absence of consideration of how challenges and opportunities might interact with each other and influence the process. One proposal to address this oversight is the theorization of scale-up and spread as social processes , prioritThe COVID-19 pandemic is producing what is essentially a “natural experiment” as alternatives to traditional face-to-face consultations become a necessity to prevent viral transmission . InteresThis review is limited by likely bias within the articles, bias inherent in the reviewing process, and the lack of theorization in the included studies. Publication bias is a long-standing and recognized phenomenon in health research  and likeThe strengths of this review include the comprehensive searching methods used to identify a breadth of published research . The analysis and synthesis of the listed challenges has also been framed by a comprehensive theoretical framework.Our review contributes an important synthesis of knowledge about the spread, scale-up, and sustainability of video consulting. Our analysis, guided by the NASSS framework, has enabled us to examine a diverse body of evidence on video consulting services that have scaled or spread to more than one setting. This addresses a fundamental gap in the literature, with prior studies typically focusing on individual technologies or services, rather than organizational implementation and spread across settings .To date, limited attention has been paid to the spread and scale-up of video consulting. Previous systematic reviews have tended to isolate areas of specific interest, focusing on selected aspects of video consulting, such as patient satisfaction ,72, feasThe studies we reviewed are broadly positive about video consulting. However, the small sample sizes, select nature of samples, and high losses to follow-up call into question any unqualified conclusion that video consulting is “effective.” The trials that have been undertaken on video consulting have provided few or no data on the organizational complexities of implementing a technology-based service, and do not address the question of how video consultation services become embedded in real-world settings.To date, there have been only a handful of rigorous and theoretically grounded qualitative or mixed methods studies that explore the emergence of video consultation services naturalistically. Such studies have yet to systematically study spread and scale-up beyond the initial implementation.This systematic review identified opportunities and challenges for the scale-up and spread of video consulting. The application of the NASSS framework surfaced complexity in a number of domains, notably characterizing the importance of organizational support and the wider system in the scale-up and spread of video consulting services. Many articles viewed spread and scale-up purely as the sum of multiple implementations, rather than explicating and theorizing the distinct  processes required to achieve widespread adoption.Given the recent impetus to implement video consulting services at pace and scale due to the COVID-19 pandemic, a more definitive evidence base is urgently needed to support global efforts, and match policy enthusiasm for the widespread use of these technologies. We strongly encourage decision makers and researchers to embrace relevant theoretical lenses that can aid spread and scale-up and ensure the future sustainability of what looks set to be a significant part of future health care delivery.


Atta bisphaerica grass-cutting ants build giant nests with hundreds or thousands of large chambers. Here, we assessed whether the total volume of fungus chambers and other nest parameters grow at close or similar proportions to worker numbers in the A. bisphaerica colony. The fungus garden biomass, external area, and total fungus chamber volume increased at rates similar to that of worker numbers. The total volume of chambers, and external area increased at a similar proportion to that of the number of workers, probably due to the fungus garden allocation needs of the colony. On the other hand, the number of fungus chambers, number of entrance holes, and nest depth increased more slowly than the number of workers. Thus, this study demonstrates that A. bisphaerica nest development regarding the total volume of chambers is similar to the increase in the number of colony workers.Atta grass-cutting ants (Formicidae: Myrmicinae: Attini) are found in the Cerrado biome and build giant nests with hundreds or thousands of large chambers. We assessed for Atta bisphaerica grass-cutting ants whether the total volume of fungus chambers and other nest parameters grow at close or similar proportions to worker numbers in the colony. Data on fungus garden biomass, population, external area, number of entrance holes, number of fungus chambers, total fungus chambers volume, and nest depth were obtained during colony growth/nest development. Our results reveal that the fungus garden biomass, external area, and total fungus chamber volume increased at rates similar to the increase in the number of workers. The total volume of chambers, and external area increased at a similar proportion to the increase in number of workers, probably due to the fungus garden allocation needs of the colony. The number of fungus chambers, number of entrance holes, and nest depth increased less than 4-fold for every 10-fold increase in the worker number. In addition, the height of the fungus chambers increased as the width increased, a pattern not observed for tunnel height and length, and the chamber volume increased according to worker number. Thus, this study demonstrates for A. bisphaerica that nest development in terms of chamber volume is similar to the increase in number of colony workers, and this contributes to a better understanding of Atta nest architecture. Pseudomyrmex and Camponotus build nests on trees from preformed cavities and organic materials [Pheidole, Odontomachus, and Pogonomyrmex dig less than a hundred small chambers [Atta genus, dig complex subterranean nests with hundreds/thousands of large chambers [Nests, burrows, traps, galls, and several structures that animals build are essential for the animal’s survival and are products of natural selection ,2. Some aterials ,8. In thchambers ,10,11. Hchambers ,13,14.Atta), it was proposed that the high mortality of semi-claustral queens was sufficient to support the natural selection of fully claustral queens [For leaf-cutting ants  [A. bisphaerica are characterized by a lower number of fungus chambers when compared to species that cut dicotyledons, such as A. sexdens [A. bisphaerica do not make specialized chambers for waste depositing (waste chambers), as is the case for the grass-cutting ant A. capiguara [The availability of food resources could influence an increase in worker numbers and fungus gardens for colony growth ,20. Foodforaging . Thus, tworkers) . Nests o sexdens . In addi sexdens . On the apiguara .A. bisphaerica nests; they also showed the number of entrance holes and tunnels that were influenced by the availability of food resources [Atta, the mature colonies are large and contain from several thousand to over a million workers [Previous studies showed the external area pattern and the number of chambers and tunnels in mature esources ,21. Furtesources . Althoug workers .Pogonomyrmex and Aphaenogaster ants  increased more slowly than the worker population [A. bisphaerica according to the number of workers, and it contributes to a better understanding of Atta nest architecture.In ants in general, the nest size is related to the number of workers, given that they build their nest in some proportion to their needs ,25. Howepulation . In thisBrachiaria spp., with patches of Paspalum spp., the preferred nesting places of grass-cutting ants.This study was conducted at Lageado Experimental Farm  and Santana Farm , Botucatu, São Paulo, Brazil. On the farms, the pasture was formed using A. bisphaerica nests were mapped since their nuptial flight in October 2014, according to Forti et al. [i et al. . After 2i et al. . In addii et al. ,23.The fungus garden and population were collected in plastic pots ranging from 0.5 to 15 L, after opening each chamber. The fungus garden was immediately taken to the laboratory to obtain a fresh weight and weighing 20% to estimate the dry weight. Drying was carried out in an oven with air circulation at 50 °C for 24 h, and the total fungus garden dry biomass of each nest was obtained by extrapolating dry mass values to 100%. The population of each nest and chamber was conserved in 70% alcohol, and 10% of the collected volume was used to count the number of workers. Workers were separated and counted, and the total population of each nest was obtained by extrapolating the number to 100% volume.Fungus chambers found according to excavation were counted and their width, height, and distance to the surface were measured. The foraging tunnels and tunnels that connect the chambers were measured in width, height, length, and depth. The tunnels that connect chambers were counted and measured in nests up to 32 months, due to a large number of these structures in older nests. In the nests aged 44 and 56 months, the dimensions of the foraging tunnels were obtained from three tunnels in each quadrant of the nest.2 (ch + r0.67), in which ‘r’ is chamber base radius and ‘ch’ is cylinder height, measured by subtracting the maximum height of the chamber from its radius, ch = h − r [The volume of each fungus chamber was estimated based on the cylinder volume. However, as the chambers are rounded, a correction factor was used for calculating the volume, V = πr = h − r . The totLinear regression analyses of log-transformed variables were used to determine the fungus garden biomass growth and nest architecture development in relation to the increase in number of colony workers. The nest architecture variables evaluated were external area, number of entrance holes, number of fungus chambers, total fungus chambers volume, and nest depth. Bivariate scaling analyses (log–log) of worker number with (a) fungus garden biomass, (b) number of fungus chambers, (c) number of entrance holes, (d) external area, (e) total fungus chambers volume, and (f) depth of nests were performed to test allometric relationships. The growth in fold number in workers was also used to determine the proportion of growth for every variable. In addition, the dimensions of chambers and tunnels were subjected to regression analysis, and data on architecture parameters, number of workers, and fungus garden biomass according to colony age  were subjected to descriptive statistics to obtain the mean, standard deviation, and maximum and minimum values. All analyses were performed with Statistica software version 7.0  .A. bisphaerica nests aged 2 months had only one entrance hole, external area around 200 cm2, fungus chamber with 160 mL volume, and depth of 18 cm. At 8 months of age, the nests increased to two fungus chambers , external area to 360 cm2, and reached a depth of 0.9 m  . Fungus A. bisphaerica nests had chambers with a volume ranging from around 200 mL to 14 L , it was speculated that the population divided into groups and small chambers  benefits the colony in communication [The proportional increase in total fungus chamber volume and external area with the worker number increase is attributed to the colony’s need for space to mainly allocate the fungus garden. Several studies support the hypothesis that the fungus garden and presence of ants in preimaginal stages within the colony are incentives for the workers to expand the space and build new chambers ,30,31. Athe nest ,14. In tnication .A. bisphaerica nests is the result of chambers and tunnels dug to grant access to the environment  that assist in nest ventilation [The external area (loose soil) of ironment a,c. For the nest . On the st holes . In A. cchambers . As a betilation .Atta colony needs a large fungus biomass for alates production  [Atta queens have maximum internal reserves, with a mass ranging from 400 to 800 mg [The increase in fungus garden biomass proportional to the worker number increase is possibly an energy strategy for the colony ,36. A mannually) . In theio 800 mg . Nuptialo 800 mg ,38. In ao 800 mg ,39.A. bisphaerica grow in dimensions (height as a function of width) and volume increases of around 3-fold for each 10-fold increase in worker number; however, their size is limited and does not exceed 14 L volume , given that soil temperature is negatively correlated with soil depth . The symA. bisphaerica nest structure is isometric according to worker number, with an increase in total volume of chambers at a similar rate to the increase in the number of workers. These findings indicate that larger dimensions of Atta nests are not only the result of a large population, as previously thought, but the fungus garden is also a relevant factor. Future studies should be conducted with Atta species that nest in forests (closed environments), and potential changes in the communication of these ants that may have arisen from the larger nests should also be investigated.This study demonstrated that the development of


N = 331 young adults . Demographics, alcohol consumption, subjective intoxication, and hangover severity were assessed for the past three days. In addition, mood  was also assessed. N = 143 participants reported to be hungover on the day of assessment, N = 122 participants reported to have been hungover the previous day (‘yesterday’), and N = 87 participants reported to have been hungover two days before the assessment (‘2 days ago’). The analyses revealed that baseline mood and mood while drinking had no relevant effect on the amount of consumed alcohol and did not significantly contribute to hangover severity. However, hangover severity was associated with significantly increased negative affect, particularly with higher levels of subjective stress on the day of the hangover.The aim of this study was to investigate whether baseline mood and/or mood while drinking have an impact on alcohol hangover severity. A survey was held among  Negative baseline mood and mood while drinking have been reported to influence alcohol consumption. This is reflected by reports that situational factors (experiencing negative life events and stress) and intrinsic factors  can be associated with negative mood ,2. SubseDrosophila has suggested that alcohol hangover/tolerance development and the response to stress may be partly mediated by the same molecular pathway [The functional link between stress and alcohol consumption has repeatedly been demonstrated via changes in the hypothalamic–pituitary-adrenal (HPA) axis, especially with respect to the corticotropin-releasing factor (CRF) ,9. Speci pathway .Due to the phenomenological (affective) and functional  overlap between mood, stress, and hangover, mood and subjective stress may be predictors of hangover severity. Despite the close link between mood and alcohol consumption, only very few studies have investigated the effects of baseline and/or current mood on experiencing hangovers.Hangover is the most commonly reported negative consequence of alcohol consumption  and has N = 1266 subjects , reported positive correlations between hangover symptom frequency and psychosocial factors, including negative life events, neuroticism, guilt about drinking, feeling depressed while drinking, and being angry while drinking. In the second paper, Harburg et al. [N = 1104 subjects were re-analyzed, the correlations between hangover symptom frequency and mood while drinking were less pronounced. Significant correlations were reported for neuroticism, guilt about drinking, drinking to escape, negative life events, and feelings of depression and anger while drinking. A stepwise linear regression analysis including all of the assessed variables was performed separately for men and women. Both analyses yielded a model with only modest predictive validity (19% in men and 21% in women) for the variance in the reported frequency of hangover symptoms. The analyses showed that with regard to the contribution of individual variables to the explained variance, guilt about drinking was the strongest predictor of hangover severity (9% in men and 11% in women), followed by neuroticism (4% in both men and women), being angry when high/drunk (3% in men and 2% in women), and negative life events (2% in men and 1% in women). In men, being depressed when high/drunk (1%) and the amount of consumed ethanol (<1%) further contributed to the model. In women, additional significant predictors were being younger (2%) and having experienced being drunk for the first time at a younger age (1%). Together, these findings suggest that mood during drinking has relatively little impact on experiencing hangover symptoms.While it is commonly reported that mood and emotions are negatively affected during the hangover state ,19,20, tg et al.  excludedHowever, there are several issues that complicate the interpretation of the results presented by Harburg et al. ,22. For Taken together, it is important to replicate and improve the study by Harburg et al. by including a valid assessment of hangover severity to infer whether mood while drinking has an impact on the presence and severity of next-day alcohol hangover. Therefore, the current study aimed to verify and extend the observations by Harburg et al. in an international sample of young adults by applying a 1-item overall hangover severity scale, which is regarded to be superior to composite symptom scores . SpecifiIn August 2018, a survey was conducted among an international sample of young adults who came to Fiji either for work or holidays. Both men and women within the age range of 18–35 years were included. The young adults were approached at Wailoaloa Beach and asked to complete a survey.Subjects who were willing to participate and sufficiently understood the English language completed the survey on location. The location was chosen because a relatively large number of young adults congregated here to spend a holiday or relax after work.The survey was anonymous and subjects did not receive an incentive for completing the survey. The investigator was present to clarify any issues arising from English not being the participants’ mother tongue. The study was conducted by Utrecht University, informed consent was obtained from all subjects, and the Ethics Committee of the Faculty of Social and Behavioral Sciences of Utrecht University granted ethical approval .The survey collected demographic information, including age, gender, height, and weight to compute body mass index (BMI), and usual weekly alcohol intake. The survey contained guidance about standard drinking sizes, and how to convert, for example, a bottle of wine into standardized alcohol units, which contain 10 g of alcohol each.To assess the past year’s immune status, the Immune Status Questionnaire (ISQ) was completed . CurrentNeuroticism was assessed with the neuroticism scale of the Eysenck Personality Questionnaire-Revised Short Scale (EPQ-RSS) ,36. The In addition to demographics and baseline mood, various other assessments regarding alcohol consumption and mood were made for the past three days . For each of these days, subjects reported their alcohol consumption. Both the number of alcoholic drinks and the time frame of consumption were assessed. The estimated BAC was computed with a modified Widmark equation . SubjectStatistical analyses were conducted with SPSS . Mean and standard deviation (SD) were computed for each variable. Outlier data  were omitted from the analyses.For each test day, participants were independently allocated to the ‘no hangover’ or ‘hangover’ group. This was based on the reported absence (score 0) or presence (score 1–10) of a hangover for that particular day. Thus, group sizes differed between the three days, and individual subjects could be allocated to the hangover group on one day, but to the no hangover group on another day, depending on the reported presence and absence of hangover for that particular day.All statistical analyses were conducted separately for each of the three days. Most study outcome variables did not follow a normal distribution. Therefore, nonparametric statistics were used to analyze the data. To compare demographics and baseline mood between the hangover and the no hangover group, independent-samples Mann–Whitney U tests were used.p < 0.05. Linear stepwise regression analyses  were conducted to determine which variables  were significant predictors of (a) having a hangover (yes/no) and of (b) hangover severity (1–10 score on the single-item hangover severity assessment). Further, linear stepwise regression analyses were conducted to determine which of the assessed variables were significant predictors of (c) the amount of consumed alcohol and of (d) subjective intoxication on the evenings preceding the next-day alcohol hangover. Analyses were conducted for the whole sample, and for men and women separately.Spearman’s rho correlations were computed between drinking variables and mood outcomes. Results were considered significant if N = 331 subjects. Their demographics and baseline mood are summarized in N = 143 subjects (43.2% of the sample) reported having a hangover on the day of the assessment (referred to as ‘today’ throughout the article). N = 188 subjects who did not report having a hangover. The comparisons revealed that subjects with a hangover scored significantly higher on some of the baseline mood scales and perceived immune fitness compared to subjects who reported no hangover. However, it should be noted that the magnitudes of the observed differences were small .The survey was completed by Variables related to alcohol consumption and mood (rated separately for mood during drinking and mood while hungover) are summarized in p < 0.0001), between hangover severity and total sleep time , and between hangover severity and sleep quality . There were no significant correlations between hangover severity and baseline mood or neuroticism. Ratings of mood while drinking did not significantly correlate with hangover severity. However, significant correlations were found between hangover severity and ‘fatigue, sleepiness’ experienced during hangover , between hangover severity and ‘stress’ experienced during hangover , and between hangover severity and ‘guilt about drinking’ experienced during hangover . A similar pattern of outcomes was seen for the other two days that were assessed (As can be seen in assessed .2) and the beta coefficient (β) are also included.The analysis of the current day revealed that three variables accounted for 45.2% of the variance in overall hangover severity. With regard to the variance explained by individual variables, subjective intoxication was the strongest predictor of hangover severity (43.0%), followed by baseline fatigue (1.5%) and sleep quality (0.7%). The addition of the ‘next day’ variables mood and guilt experienced while hungover yielded a model where four variables accounted for 56.1% of the variance in overall hangover severity. Subjective intoxication was again the strongest predictor of hangover severity (43.0%), followed by fatigue while hungover (8.7%), guilt about drinking while hungover (4.0%), and stress while hungover (0.4%). Regression analyses for the other two days of assessments yielded comparable results, as subjective intoxication was always the best predictor . Table 3The demographic and mood variables that affected alcohol consumption are summarized in A summary of all findings is presented in The current study aimed to verify and extend the observations by Harburg et al. in an international sample of young adults by applying a 1-item overall hangover severity scale to investigate whether baseline mood, mood while drinking, as well as mood during hangover, were associated with and/or predicted current and retrospective hangover severity. In contrast to previous reports ,21,22, tThe confirmatory regression analyses further supported our conclusions. The obtained models revealed that subjective intoxication (drunkenness) was the most important contributor to hangover severity. In comparison to that, mood while drinking had no relevant impact on hangover severity. Mood while hungover and guilt about drinking while hungover significantly contributed to the model predicting hangover severity, but it should also be noted that stress, fatigue, and guilt during hangover are most likely the consequences, rather than the cause of hangover severity.Regression models predicting subjective intoxication revealed that the most important contributing factor was the amount of consumed alcohol. For both subjective intoxication and the amount of alcohol consumed, the regression models revealed that baseline mood and mood during drinking only had a small contribution to the models, if any . At first sight, this might be regarded as comparable to the findings reported by Harburg et al. , who repThe observation that subjective intoxication was the most important predictor of hangover severity is in line with results of previous studies ,42,43. OResearch on the relationship between smoking and the presence and severity of hangover is limited, and this is an important topic for future research. Finally, we examined possible sex differences in variables contributing to hangover severity. Previous research showed that the presence and severity of hangover symptoms did not relevantly differ between men and women at comparable BAC levels ,50. In t


The demographic, clinical, and anesthetic data, with data on pain severity and opioid administration at the postoperative anesthesia care unit (PACU) on postoperative days (PODs) 1, 3, and 5, were compared between the groups. The pain score and opioid administration did not differ at the PACU or on PODs 1, 3, or 5. The day of PCA discontinuation, time to pass flatus, length of the hospital stay, and incidence of nausea/vomiting, dizziness, and headache also did not differ between the groups. Fentanyl PCA with 80 mg of nefopam as an adjuvant did not have a superior analgesic effect after laparoscopic colorectal cancer surgery.Despite rapid advancements in laparoscopic surgical techniques and perioperative management, postoperative pain remains a significant clinical issue. We examined the analgesic efficacy of nefopam as an adjuvant in patient-controlled analgesia (PCA) for acute postoperative pain in patients undergoing laparoscopic colorectal cancer surgery. We retrospectively analyzed the medical records of 120 patients who did or did not receive 80 mg of nefopam as an adjuvant in fentanyl PCA; they were allocated to the nefopam ( Despite the rapid advancements in laparoscopic surgical techniques and perioperative management, postoperative pain remains a significant clinical issue ,2,3. Poshttp://cris.nih.go.kr/cris/index.jsp, KCT0005037). The need for individual consent was waived by the institutional review board, as this was a retrospective study involving medical record review.We retrospectively reviewed the electronic medical records of patients who underwent laparoscopic colorectal surgery from January 2015 to April 2020 at a single center. The patients ranged in age from 27 to 74 years. All patients had tumors within the colon or rectum and had undergone laparoscopic colectomy or anterior resection . The exc®, Ace Medical, Seoul, Korea) with a basal infusion rate of 0.5 mL/h, a bolus of 1 mL, and a lockout interval of 15 min. The drugs used for PCA comprised a 100 mL mixture of 20 µg/kg fentanyl in normal saline, whereas the nefopam addition to PCA comprised a 100 mL mixture of 80 mg nefopam  and 20 µg/kg fentanyl in normal saline. At the end of surgery, patients were administered 4 mg/kg of intravenous sugammadex and/or 0.03 mg/kg of intravenous pyridostigmine, and 0.002 mg/kg of intravenous glycopyrrolate. The reversal agents were decided by the anesthesiologists considering the neuromuscular block status. After extubation, patients were moved to the postoperative anesthesia care unit (PACU) where they received a further bolus of intravenous fentanyl at 0.5 µg/kg when the numeric rating scale (0 = no pain to 10 = absolutely intolerable pain) score was greater than 3. Postoperative opioids at the general ward included intravenous fentanyl, meperidine, or hydromorphone, as well as oral medications, including oxycodone and tapentadol [Patients did not receive premedication before surgery. After admission to the operating room, standard monitoring  was performed, including oxygen saturation, electrocardiography, end-tidal carbon dioxide, pulse oximetry, bispectral index (BIS), and non-invasive blood pressure. Anesthesia was induced intravenously using 40 mg of 2% lidocaine, 2 mg/kg of 2% propofol, 0.5–1 µg/kg of fentanyl, 0.05 mg/kg of midazolam, and 0.6–0.8 mg/kg of rocuronium. The patient was endotracheally intubated using a Macintosh laryngoscope after approximately 3–5 min of mask ventilation and the loss of all four twitches upon train-of-four ulnar nerve stimulation. Following tracheal intubation, anesthesia was maintained using 1.5–3.0 vol% sevoflurane and a bolus injection of 0.5–1 µg/kg fentanyl to maintain hemodynamic parameters within 20% of baseline values and a BIS of 40–60. The lungs were ventilated with 50% oxygen with air. This was adjusted to maintain an end-tidal carbon dioxide level of 30–40 mmHg. All surgeries were performed by one of six specialized colorectal surgeons who followed similar techniques for colorectal cancer [pentadol . Opioid pentadol . Pain sct-test, or Fisher’s exact test. We extracted the mean arterial pressure (MAP), heart rate, and BIS before intubation (T1), at the start of surgery (T2), at the end of surgery (T3), and after extubation (T4); these parameters were compared using the t-test or Wilcoxon rank-sum test. We compared the pain scores at PACU arrival and discharge, as well as on PODs 1, 3, and 5 with rest and movement. Moreover, we compared the opioid administration during anesthesia, at the PACU, and on PODs 1, 3, and 5 using the Wilcoxon rank-sum test. The chi-squared test was used to compare the day of PCA discontinuation, time to pass flatus, length of the hospital stay, and the postoperative incidence of nausea/vomiting, dizziness, and headache. A p-value less than 0.05 was considered statistically significant.All data were analyzed using SAS 9.4 . Data are expressed as the mean ± standard deviation (SD) or number (proportion), as appropriate. Demographic, clinical, and anesthetic data for the two groups were compared using a chi-squared test, n = 60) and those with PCA that did not contain nefopam . Demographic and clinical data are summarized in Of the 237 patients who were assessed for eligibility, 117 were excluded based on the exclusion criteria. Thus, data of 120 patients were analyzed. The patients were categorized into two groups, namely those with PCA containing 80 mg of nefopam  of patients. ASA, American Society of Anesthesiologists; NRS, numeric rating scale; Nefopam group, patients who received nefopam as an adjuvant in patient-controlled analgesia; Non-nefopam group, patients who did not receive nefopam as an adjuvant in patient-controlled analgesia. * p-value < 0.05 was considered statistically significant.All data are presented as the mean ± SD. T1, before intubation; T2, start of surgery; T3, end of surgery; T4, after extubation; Nefopam group, patients who received nefopam as an adjuvant in patient-controlled analgesia; Non-nefopam group, patients who did not receive nefopam as an adjuvant in patient-controlled analgesia. * p-value < 0.05 was considered statistically significant.All data are presented as the mean ± SD. NRS, numeric rating scale; PACU, postoperative anesthesia care unit; POD, postoperative day; Nefopam group, patients who received nefopam as an adjuvant in patient-controlled analgesia; Non-nefopam group, patients who did not receive nefopam as an adjuvant in patient-controlled analgesia. p-value < 0.05 was considered statistically significant.All data are presented as the mean ± SD. PACU, postoperative anesthesia care unit, POD, postoperative day; Nefopam group, patients who received nefopam as an adjuvant in patient-controlled analgesia; Non-nefopam group, patients who did not receive nefopam as an adjuvant in patient-controlled analgesia. p-value < 0.05 was considered statistically significant.All data are presented as number (%) of patients. Nefopam group, patients who received nefopam as an adjuvant in patient-controlled analgesia; Non-nefopam group, patients who did not receive nefopam as an adjuvant in patient-controlled analgesia. In the present study, we aimed to evaluate whether nefopam as an adjuvant in fentanyl PCA exerts a superior analgesic effect in patients undergoing laparoscopic colorectal cancer surgery. Postoperative pain severity and opioid administration were not reduced during the acute postoperative period and neither was the incidence of postoperative nausea/vomiting, dizziness, headache, or somnolence. We suspect that these results are related to the low nefopam dose for prolonged analgesia, and/or the timing of nefopam administration for preemptive analgesia.The ERAS protocols in surgery for colorectal cancer are designed to improve patient outcomes ,27,28. ESeveral studies have reported on the timing of nefopam administration for postoperative analgesia. Preoperative administration of 20 mg of nefopam reduced exertional pain during 2 to 72 h of follow-up after colon cancer surgery, although it did not reduce postoperative fentanyl consumption . FurtherThis study has several limitations. First, we did not convert the postoperatively administered non-opioid analgesics  into opioid equivalents because of conversion difficulties, although it can affect postoperative pain. Additionally, we did not enroll patients who had taken non-opioid analgesics during anesthesia. These exclusions negatively affect the generalization of the usual perioperative experience. Second, we did not measure the daily delivered and rescue bolus doses of PCA. Third, the sample size of our study was small. Fourth, we did not use propensity score matching. To minimize selection bias, we applied the exclusion criteria of nefopam for patients in the non-nefopam group. However, there may have been possible unknown confounding factors in the selection of patients in the non-nefopam group.We observed no superior analgesic effects using 80 mg of nefopam as an adjuvant in fentanyl PCA during the acute postoperative period. Further randomized controlled studies are needed to determine an effective and safe dose and administration regimen for nefopam to be used as an adjuvant to fentanyl or other opioids in advanced multimodal analgesia for PCA.


Using the principle of saturation as a foundation, we describe and validate a simple-to-apply method for assessing and reporting on saturation in the context of inductive thematic analyses. Following a review of the empirical research on data saturation and sample size estimation in qualitative research, we propose an alternative way to evaluate saturation that overcomes the shortcomings and challenges associated with existing methods identified in our review. Our approach includes three primary elements in its calculation and assessment: Base Size, Run Length, and New Information Threshold. We additionally propose a more flexible approach to reporting saturation. To validate our method, we use a bootstrapping technique on three existing thematically coded qualitative datasets generated from in-depth interviews. Results from this analysis indicate the method we propose to assess and report on saturation is feasible and congruent with findings from earlier studies.Data saturation is the most commonly employed concept for estimating sample sizes in qualitative research. Over the past 20 years, scholars using both empirical research and mathematical/statistical models have made significant contributions to the question: How many qualitative interviews are enough? This body of work has advanced the evidence base for sample size estimation in qualitative inquiry during the design phase of a study,  As Morse pointed out more than 20 years ago, “saturation is an important component of rigor. It is present in all qualitative research but, unfortunately, it is evident mainly by declaration” Since we had available the total number of codes identified in each dataset, we carried out one additional calculation as a way to provide another metric to understand how the median number of interviews to reach a new information threshold related to retrospectively-assessed degrees of saturation with the entire dataset. In this case, once the number of interviews to reach a new information threshold was determined for each run of a dataset, we divided the number of unique themes identified up to that point by the total number of unique themes. This provided a percent–or degree–of saturation for each run of the data, which was then used to generate a median and 5The results from the bootstrapping analyses are presented by dataset, in Tables more interviews than the given median, depending on the run length of choice. This is indicated by a superscript +2 or +3.Note that, as described in the example above, the number of interviews in the run length is not included in the number of interviews to reach the given new information threshold, so the total number of events needed to assess having reached the threshold is two or three +2, while for a run length of three we would say saturation was observed at 7+3, both at the ≤5% new information level. Using the total number of themes in the dataset retrospectively, the number of themes evident across 6–7 interviews corresponded with a median degree of saturation of 78% to 82%.For Dataset 1 , at the +2 and 14+3. In other words, at run length 2, it took 11 interviews, plus two more to confirm that no new information was contributed. At run length 3 it was 14 interviews plus three more to confirm no new information. The number of themes evident across 11–14 interviews corresponded with a median degree of saturation of 87% to 89%.At the 0% new information threshold, the median number of interviews to indicate saturation were again consistent across bases sizes, varying only by the run length. The median number of interviews required were 11+2) and 7–8 interviews at run length 3 (7+3 or 8+3). The number of themes evident across 6–8 interviews corresponded with a median degree of saturation of 79% to 82%. At the 0% new information threshold saturation was indicated at the same points as in Dataset 1: 11+2 and 14+3, consistent across all base sizes. In other words, no new information was observed after a median of 11 interviews using a run-length of 2, nor after 14 interviews using a run length of 3. Here again, despite a different total number of themes in the overall dataset, the number of new themes evident across 11–14 interviews corresponded with a median degree of saturation of 87% to 89%.The results for Dataset 2 were nearly identical to Dataset 1 . Saturat+2 and 16+3, consistent across base sizes. The number of new themes evident across 12–16 interviews corresponded with a median degree of saturation of 69% to 76%.Dataset 3  containeprospectively during the data collection and analysis process or retrospectively, after data collection and analysis are complete. A key advantage is that the metrics are flexible, affording researchers the ability to choose different degrees of rigor by selecting different run lengths and/or new information thresholds. Similarly, the method allows for different options–and greater clarity and transparency–in describing and reporting on saturation.In this paper we present a way of assessing thematic saturation in inductive analysis of qualitative interviews. We describe how this method circumvents many of the limitations associated with other ways of conceptualizing, assessing and reporting on saturation within an in-depth interview context. The process can be applied either th%ile) 11–12 interviews might be needed, tracking with existing literature indicating 12 interviews are typically needed to reach higher degrees of saturation.Based on the bootstrapping analyses we can draw several conclusions. The first is that the results are within the range of what we would have expected based on previous empirical studies. Using the ≤5% new information threshold, our findings indicate that typically 6–7 interviews will capture the majority of themes in a homogenous sample (6 interviews to reach 80% saturation). Our analyses also show that at the higher end of the range for this option . If analyzing data in real time, the results of this initial assessment can then determine whether or not more interviews are needed.Run length has an effect on the outcome, as one would expect. The longer the run length, the greater number of interviews required to reach saturation. The size of run length effect is smallest–very minimal–if employing the ≤5% new information threshold. The practical implication of this finding is that researchers can choose a longer run length–e.g., three interviews (or more)–to generate a more conservative assessment of saturation.The new information threshold selected affects the point at which saturation is indicated, as one would expect. The lower the new information threshold–and therefore the more conservative the allowance for recognizing new information–the more interviews are needed to achieve saturation. From an applied standpoint this finding is important in that researchers can feel confident that choosing a more stringent new information threshold–e.g., 0%—will result in a more conservative assessment of saturation, if so desired.There are, of course, still limitations to this approach. It was developed with applied inductive thematic analyses in mind–those for which the research is designed to answer a relatively narrow question about a specific real-world issue or problem–and the datasets used in the bootstrapping analyses were generated and analyzed within this framework. The applicability of this approach for qualitative research with a different epistemological or phenomenological perspective is yet untested. Another potential limitation of this method relates to codebook structure. When conducting an inductive thematic analysis, researchers must decide on an appropriate codebook organizational scheme  or in several commonly used increments . Likewise, while an odds ratio of 1.2 may be statistically significant, whether or not it’s meaningful in a real-world sense is entirely open to interpretation.Selecting and interpreting levels of rigor, precision, and confidence is a subjective enterprise. What a quantitative researcher accepts, for example, as a large enough effect size or a small enough p-value is a subjective determination and based on convention in a particular field of study. The same can be said for how a researcher chooses to report and interpret statistical findings. P-values can be expressed either in absolute terms Click here for additional data file.


Serious consequences leading to the increase of infectious diseases and mortality of children justifies the importance of interventions for eradication of malnutrition. Thus, this study aimed to provide a model for the prevention of malnutrition among children under 5 years old (CU5) in Iran.This paper is part of a bigger study, conducted in 2017 using a mixed methods approach. A model for prevention of malnutrition in CU5 was proposed based on the earlier steps, with the cooperation and consultation of experts and specialists. In the final step, a Delphi method was used to determine the validity of the proposed model.The main dimensions of the prevention model of malnutrition for CU5 in Iran included four level: basic causes, interventions, outcomes and impact. The proposed interventions are presented based on twelve areas: structural, intersectoral, political, economic, sanitary, health-oriented, research, educational/cultural, evaluation related, production, infrastructures and legal. Based on these areas, 118 solutions were finally selected for the final model. This model is designed based on the current conditions in different regions of Iran, the factors related to child malnutrition, affective context on policy making, the content of previous policies, the process of policy making in Iran, and key stakeholders and actors in policy making.In order to prevent malnutrition, the causing factors should be identified and resolved. The adopted policies should be, more seriously, based on the presence of key stakeholders and actors. Most of the existing nutritional problems among children are because of inappropriate consumerism culture and habits in families and its transfer to children. Early years of childhood, as the most important years of life, can affect the health status of whole life seriously , 2. The Malnutrition is a condition in which a deficiency, excess or imbalance of energy, protein and other nutrients occurs . ApproxiThis type of malnutrition is caused by many factors such as inadequate care in pregnancy period, low literacy of family members, inadequate community malfunctions , polluted environments, poor nutrition of the household, frequent and severe infections, inadequate supply of food, and poverty, especially . DespiteConsidering the fact that stunning, underweight and wasting are among the most important indicators of malnutrition, the current study aimed to use these indicators to investigate malnutrition among children. In 2018, globally 149 million CU5 (22%) were stunted  and 49.4Using the indigenous models, specific for each region of world, needs to be considered to reduce malnutrition and improve the situation. Hence, given the importance of malnutrition in children, the aim of this study was to develop a model for the prevention of malnutrition among CU5 in Iran.This paper is a part of a bigger study that was conducted in 2017 using a mixed methods  approach in four steps , individuals with the experience of executive jobs (at least 5 years), and finally, having effective experience related with policy making in this area. Snowballing method was also used to detect the eligible samples for entering the study.Data collection tool comprised of three sections: demographic information of respondents, policy options, and also a section for recording the participants’ responses about the proposed options. The validity of the questionnaire was evaluated through the opinions of experts in this field. To assess the reliability, a pilot study was done on five individuals. A summary of the objectives along with a proposed model was sent to the experts in two rounds. The results of the first phase were analyzed and a revised model was developed based on the comments. It was then sent back to the experts and to collect their final comments. To prepare the final model, the Delphi step findings were analyzed by descriptive method. If the agreement was over 75%, the policy option was approved and if the agreement was between 50 and 75%, the policy option entered the next round. Moreover, the agreement below 50% would lead to the rejection of options . At the In order to design the primary model Fig. , the finThe first and second phases of the Delphi attended by 25 and 20 individuals respectively. In Table The prevalence of malnutrition among Iranian CU5 is much lower than the global average and is in relatively good condition. About contextual causes, mother education level, father education level, child gender, birth weight, and age group were mentioned as the most important factors in the literature.Based on evidence, the main factors in malnutrition are low socioeconomic status of family, parental education level, household health index, health literacy, nutrition culture, maternal characteristics such as BMI, nutrition during pregnancy, and the number of childbirths, child characteristics such as age, gender, birth weight and common infectious diseases –25. ConsThe interventions are suggested to be classify based on the area, scale, type, or time to reduce the children malnutrition. Considering the contextual causes, the most important intervention could be maternal measures. Maternal interventions, particularly during pregnancy, may have intergenerational effects and on birth weight and child growth . Other iLow cost and time are the two points mentioned in previous studies regarding the implementation of interventions. Results of the study by Peru et al. showed that nutritional interventions are low cost and may improve growth in the short or midterm . ClombatNevertheless, results of some studies showed that some interventions had low impact. The results of three meta-analysis studies showed that the use of micronutrient as an intervention was effective in improving the development of children, but iron and vitamin supplements did not have a significant effect in the improvement of children’s growth . A studyIn general, it seems that screening in short intervals, the status of children can be monitored regularly and the effectiveness of interventions can be controlled in order to modify them. It should also be noted that health policies, including nutrition, differ greatly from other policies, and interests and objectives of the governors have a direct impact on nutrition policies . Some stThe findings of policy analysis in the current study showed that using evidence, documents, policies, previous experiences, and experiences of other countries, a successful model for preventing child malnutrition can be developed for the country. A model that, in addition to the children’s health and nutrition needs, involves all individuals, organizations and stakeholders, considers the health of mother and child before the childbirth and has the ability to align the activities of all stakeholders. Furthermore, a nationwide program and a comprehensive document in the field of the nutrition for CU5 is needed; to do so, strengthening of the political process is crucial .Some studies presented a model for preventing and reducing malnutrition in children. Using appropriate models to improve the quality of services in different organizations can be considered to develop the final model –52. DiffPolicy options of the final model for malnutrition prevention in CU5 in Iran are presented based on educational, research, structural, economic, health-oriented, hygiene, political and inter-sectorial dimensions. This model was developed based on the current conditions in different regions of the country, factors related to child malnutrition, context effecting policy making, the content of previous policies, the process of policy making in Iran, and stakeholders and actors effective in policy making.The findings showed that in order to prevent malnutrition, first, the context causes should be identified and resolved. Context causes, solutions and interventions may differ in different regions of the country. The interventions can be supportive and health care related. The adopted policies should be strongly based on key stakeholders and actors. One of the most important needs in children nutrition, especially CU5, is formulating a comprehensive national document designed for this age group. The issue of targeted subsidies, helping poor people, job creation and production are also so important and need to be considered. Economic empowerment will ultimately lead to an increase in welfare of the family and the improvement of the nutritional status of children. One of the important strategies for improving economic conditions of the community is the balanced distribution of resources. Also, the correct reinforcing of laws will improve many indicators and ultimately improve the nutritional status of children. In many cases, there are proper and comprehensive rules for resolving issues but they are not correctly implemented. Finally, nutritional culture and literacy need to be considered. Low nutritional literacy will result in inadequate or inappropriate nutrition, malnutrition and other complications in children. Many of the current nutritional problems are due to wrong consumption culture in family that transfers to children. Solving the above-mentioned problems and meeting the desired goals require strong cooperation between the organizations involved in the nutrition of children; and it can be achieved with sufficient expertise and being commitment to the goals.