Alzheimer’s disease (AD) is a neurodegenerative disorder characterized by a progressive memory loss and cognitive decline that has been associated with an accumulation in the brain of intracellular neurofibrillary tangles (NFTs) formed by hyperphosphorylated tau protein, and extracellular senile plaques formed by β-amyloid peptides. Currently, there is no cure for AD and after the failure of anti β-amyloid therapies, active and passive tau immunotherapeutic approaches have been developed in order to prevent, reduce or ideally reverse the disease. Vaccination is one of the most effective approaches to prevent diseases and poxviruses, particularly modified vaccinia virus Ankara (MVA), are one of the most promising viral vectors used as vaccines against several human diseases. Thus, we present here the generation and characterization of the first MVA vectors expressing human tau genes; the full-length 4R2N tau protein or a 3RC tau fragment containing 3 tubulin-binding motifs and the C-terminal region . Both MVA-Tau recombinant viruses efficiently expressed the human tau 4R2N or 3RC proteins in cultured cells, being detected in the cytoplasm of infected cells and co-localized with tubulin. These MVA-Tau vaccines impacted the innate immune responses with a differential recruitment of innate immune cells to the peritoneal cavity of infected mice. However, no tau-specific T cell or humoral immune responses were detected in vaccinated mice. Immunization of transgenic P301S mice, a mouse model for tauopathies, with a DNA-Tau prime/MVA-Tau boost approach showed no significant differences in the hyperphosphorylation of tau, motor capacity and survival rate, when compared to non-vaccinated mice. These findings showed that a well-established and potent protocol of T and B cell activation based on DNA/MVA prime/boost regimens using DNA and MVA vectors expressing tau full-length 4R2N or 3RC proteins is not sufficient to trigger tau-specific T and B cell immune responses and to induce a protective effect against tauopathy in this P301S murine model. In the pursuit of AD vaccines, our results highlight the need for novel optimized tau immunogens and additional modes of presentation of tau protein to the immune system. Alzheimer’s disease (AD) is the most common neurodegenerative disorder and the biggest cause of dementia worldwide, representing one of the major causes of dependence, disability and mortality in elderly people . It is cSenile plaques are formed of abnormally folded β-amyloid peptides and play an important role in the development of the disease, mainly in familial Alzheimer’s disease (FAD) . TherefoNFTs consist of helical filaments of hyperphosphorylated tau protein inside neurons. Although senile plaques of β-amyloid appear earlier than NFT pathology, they don’t correlate with the disease progression as they reach a plateau early in the symptomatic phase of the disease ,15. HoweModified vaccinia virus Ankara (MVA) is a highly attenuated poxvirus vector extensively used in several preclinical and clinical trials as a vaccine candidate against numerous infectious diseases and cancer ,33,34. MHere, we describe the generation and characterization of two novel vaccine candidates against AD based on the MVA vector expressing either the human full-length 4R2N tau isoform protein or a 3RC tau protein containing 3 tubulin-binding motifs and the C-terminal region . Both MVA-Tau vaccine candidates correctly expressed the corresponding tau proteins, were detected in the cytoplasm of infected cells co-localized with tubulin, and the recombinant viruses were highly stable in cell culture upon multiple passages. Both MVA-Tau vaccines impacted on the in vivo recruitment of innate immune cells in the peritoneal cavity of infected mice, although no tau-specific T and B cell immune responses were detected in mice immunized with the MVA-Tau vectors. Vaccination of transgenic P301S mice, a mouse model for tauopathies, with the combination of two vectors, a DNA-Tau as a priming component (DNA-au4R2N or DNA-Tau3RC) and MVA-au as a booster (MVA-Tau4R2N or MVA-Tau3RC) was not able to decrease significantly the levels of hyperphosphorylated tau, a clinical sign of AD, nor have a significant impact on the motor capacity and survival rate. Besides the failure to control the AD progression in this P301S mouse model, these results open the path to generate novel optimized tau immunogens able to induce a better tau antigen presentation to the immune system and to develop novel vaccination protocols that could control disease progression.Female C57BL/6OlaHsd mice (6 to 8 weeks old) were purchased from Envigo Laboratories, stored in the animal facility of the Centro Nacional de Biotecnología (CNB) , and the immunogenicity studies were approved by the Ethical Committee of Animal Experimentation (CEEA) of CNB-CSIC  and by the Division of Animal Protection of the Comunidad de Madrid (PROEX 331/14). The efficacy animal studies in transgenic P301S mice were approved by the CEEA of Centro de Biología Molecular “Severo Ochoa” (CBMSO)  and the Division of Animal Protection of the Comunidad de Madrid (PROEX 62/14). Animal procedures were performed according to international guidelines and to the Spanish law under the Royal Decree (RD 53/2013).® CCL-2) were grown in complete Dulbecco’s modified Eagle’s medium (DMEM)  supplemented with 10% heat-inactivated fetal calf serum (FCS) Gibco-Life Technologies, Carlsbad, CA, USA) for DF-1 and CEF cells or 10% newborn calf serum (NCS)  for HeLa cells. Cell cultures were maintained at 37 °C in a humidified incubator containing 5% CO2. Cell lines were infected with viruses and after 1 h of adsorption, the virus inoculum was removed and DMEM-2% FCS or DMEM-2% NCS was added to the cell cultures.DF-1 cells  cell line, ATCC catalog no. CRL-12203), primary CEF cells , and HeLa cells  [Parental virus used for the generation of recombinant MVA-Tau4R2N and MVA-Tau3RC vaccine candidates is a wild-type (WT) MVA (MVA-WT) modified by inserting the green fluorescent protein (GFP) gene into the vaccinia virus (VACV) thymidine kinase (TK) locus and by deleting the immunomodulatory VACV genes A-Δ-GFP) ,37,38. TA-Δ-GFP) . All MVAIn this study we used the full-length human tau gene , and a tau 3RC fragment containing three tubulin-binding motifs and the C-terminal region . Both taThe plasmid transfer vectors pCyA-Tau4R2N and pCyA-Tau3RC were constructed and used for the generation of recombinant viruses MVA-Tau4R2N and MVA-Tau3RC, respectively, allowing the insertion of the human tau genes in the TK locus of parental MVA-Δ-GFP by homologous recombination, following an infection/transfection procedure, as previously described ,37,38,42MVA-Tau4R2N and MVA-Tau3RC were generated using MVA-Δ-GFP as parental virus and pCyA-Tau4R2N or pCyA-Tau3RC as plasmid transfer vectors, respectively, using an infection/transfection protocol previously described ,37,38,42C6L, K7R and A46R genes in MVA-Tau4R2N and MVA-Tau3RC was confirmed by PCR, as previously described [The correct generation and purity of recombinant viruses MVA-Tau4R2N and MVA-Tau3RC was confirmed by PCR with primers TK-L and TK-R, annealing in the VACV TK locus and allowing the amplification of the full-length human Tau4R2N or the Tau3RC inserts, as previously described ,37,38,42escribed ,44. FurtTo confirm the correct expression of the human Tau4R2N or Tau3RC antigens present in MVA-Tau4R2N or MVA-Tau3RC, DF-1 cells were mock-infected or infected with MVA-Tau4R2N, MVA-Tau3RC, and MVA-Δ-GFP at 5 plaque forming units (PFUs)/cell and at 3, 6 and 24 h postinfection (hpi), cell extracts or supernatants were collected, lysed in Laemmli buffer, fractionated in 10% sodium dodecyl sulfate–polyacrylamide gel electrophoresis (SDS-PAGE), transferred to a nitrocellulose membrane and analyzed by Western blotting with a mouse monoclonal antibody against the microtubule-binding region of the human tau protein  to evaluate the expression of the human Tau4R2N or Tau3RC proteins. As loading controls, a rabbit anti-β-actin antibody , and a rabbit anti-VACV E3 antibody  were used. The membranes were incubated with the antibodies at 4 °C overnight. Anti-mouse horseradish peroxidase (HRP)-conjugated antibody , or anti-rabbit HRP-conjugated antibody , were used as secondary antibodies. The immunocomplexes were detected using an HRP-luminol enhanced-chemiluminescence (ECL) system (ECL Plus) .To determine the virus growth profile of MVA-Tau4R2N and MVA-Tau3RC, in comparison to that of parental MVA-Δ-GFP, DF-1 cells were infected in duplicate at 0.01 PFUs/cell with MVA-Δ-GFP, MVA-Tau4R2N or MVA-Tau3RC, as previously described ,37,38,42The genetic stability of recombinant viruses MVA-Tau4R2N and MVA-Tau3RC was analyzed as previously described ,37,38,42Immunofluorescence studies were done in HeLa cells mock-infected or infected at a MOI of 0.5 PFUs/cell with MVA-Tau4R2N, MVA-Tau3RC or MVA-WT for 24 h, as previously described ,37,38,45n = 5 mice per group) were injected by the intraperitoneal (i.p.) route with 1 x 107 PFUs per mouse of MVA-Tau4R2N, MVA-Tau3RC, MVA-∆-GFP or PBS. At 24 and 48 h post inoculation, peritoneal exudate cells were collected in 6 mL of PBS-2% FCS and the presence of different immune cells was analyzed by flow cytometry, as previously described [Groups of female C57BL/6OlaHsd mice  were immunized with 100 µg of pSG5-Tau4R2N (termed DNA-Tau4R2N) or pSG5-Φ (termed DNA-Φ), by the intramuscular (i.m.) route. Four weeks after the first immunization (week 17), mice received a booster dose with 2 × 107 PFU of MVA-Tau4R2N or MVA-WT by the i.p. route. Additionally, 4 C57BL/6 WT mice were immunized at the same time points (weeks 13 and 17) with PBS. At week 31 mice were sacrificed and their brains were collected to study the tau phosphorylation by immunohistochemistry or Western blot  were immunized with 100 µg of pSG5-Tau3RC (termed DNA-Tau3RC) or DNA-Φ by the i.m. route. Four weeks after the first immunization (week 26), mice received a booster dose with 2 × 107 PFU of MVA-Tau3RC or MVA-WT by the i.p. route. At weeks 34, 41, and 47  the motor capacity of the mice was evaluated by a rotarod test PS19Vle/J), that carries a mutant (P301S) human microtubule-associated protein tau (MAPT) gene encoding T34-tau isoform (1N4R) driven by the mouse prion-protein promoter (Prnp) on a B6C3H/F1 genetic background ,48. Thisblot see . For thetest see . At weektest see .Tau phosphorylation in hippocampal brain samples was analyzed by Western blotting, as previously described . Extract2O2 in PBS for 30 min to quench endogenous peroxidase activity. Subsequently, sections were blocked for 1 h in PBS containing 0.5% Fetal Bovine Serum, 0.3% Triton X-100 and 1% BSA  and incubated overnight at 4 °C in PBS containing 0.3% Triton X-100 and 1% BSA with the corresponding primary antibody: anti-tau TAU-5  or anti-phosphorylated tau AT8 (Thermo Fisher Scientific) antibodies. Finally, brain sections were incubated with anti-mouse secondary antibody and avidin-biotin complex using the Elite Vectastain kit (Vector Laboratories). Chromogen reactions were performed with diaminobenzidine  for 10 min. Mouse sections were mounted on glass slides and coverslipped with Mowiol . Images were captured using an Olympus BX41 microscope with an Olympus camera DP-70 (Olympus Denmark A/S).Tau phosphorylation in dorsal dentate gyrus samples was analyzed by immunohistochemistry, as previously described . SectionMotor coordination and balance of immunized P301S mice were tested by rotarod test by using an accelerating rotarod apparatus . After a pre-trained period of two days at a constant speed (4 rpm the first day over 1 min four times and 8 rpm over 1 min four times the second day), on the third day the rotarod accelerated from 4 to 40 rpm over 5 min and mice were tested three times. The latency to fall was measured during accelerating trials.p ≤ 0.05; ** p ≤ 0.005; *** p ≤ 0.001.The statistical significance of differences between groups in the experiment of cell recruitment was determined by Student’s t test . The statistical analyses of the tau phosphorylation measured by immunohistochemistry and Western blotting in immunized P301S mice were done using one-way ANOVA  with Dunnett’s correction for multiple comparisons. Statistical analysis of rotarod test was performed using Student’s t test. For the survival analysis, we carried out a Kaplan–Meier comparison of the survival curves (Log-rank Mantel-Cox test). Significant differences are described as follows: * C6L, K7R, and A46R [To generate novel vaccines against AD that could impact on tau pathology, we have developed two MVA-based vaccine candidates expressing either the full-length human Tau4R2N isoform or the Tau3RC mutant protein . Human Tau4R2N and Tau3RC genes were inserted in the vector backbone of an optimized parental MVA (termed MVA-Δ-GFP) that contains deletions in the VACV immunomodulatory genes and A46R ,37,38, aand A46R , EbolaviThe correct insertion and purity of recombinant MVA-Tau4R2N and MVA-Tau3RC viruses were analyzed by PCR using primers annealing in the VACV TK-flanking regions that confirmed the presence of the full-length human Tau4R2N and the Tau3RC genes in MVA-Tau4R2N and MVA-Tau3RC, respectively B. MoreovTo show that MVA-Tau4R2N and MVA-Tau3RC constitutively express the full-length human Tau4R2N and Tau3RC proteins, respectively, we carried out a Western blot analysis using the specific antibody 7.51 that binds to the microtubule-binding region to analyze cell extracts from DF-1 cells mock infected or infected at a MOI of 5 PFU/cell with MVA-Tau4R2N, MVA-Tau3RC, or parental MVA-Δ-GFP for 24 h. The results demonstrated that MVA-Tau4R2N and MVA-Tau3RC properly expressed the full-length human Tau4R2N and Tau3RC proteins, respectively C. MoreovNext, to determine whether expression of the full-length human Tau4R2N or Tau3RC proteins affects MVA replication in cell culture, we evaluated the growth kinetics of MVA-Tau4R2N and MVA-Tau3RC in DF-1 cells, in comparison to parental MVA-Δ-GFP. The results showed that the kinetics of viral growth were similar between all the viruses A, indicaThen, to ensure that MVA-Tau4R2N and MVA-Tau3RC are stable and can be maintained in cultured cells without the loss of the full-length human Tau4R2N or Tau3RC genes, both MVA-Tau vaccine candidates were additionally grown in DF-1 cells infected at a MOI of 0.01 PFU/cell during 9 consecutive passages, and expression of the human Tau4R2N or Tau3RC proteins during the different passages was determined by Western blotting. The results showed that both MVA-Tau vaccine candidates efficiently expressed the human tau proteins after successive passages, demonstrating that recombinant MVA-Tau4R2N and MVA-Tau3RC are genetically stable B.The expression and intracellular localization of the human Tau4R2N and Tau3RC proteins expressed by MVA-Tau4R2N and MVA-Tau3RC was studied by confocal immunofluorescence microscopy in HeLa cells infected at a MOI of 0.5 PFU/cell with MVA-Tau4R2N, MVA-Tau3RC or MVA-WT. At 24 hpi cells were permeabilized and stained with antibodies against human tau protein and tubulin . The res7 PFUs/mouse; five mice per group; see Materials and Methods), and determined by flow cytometry the absolute numbers of several immune cell populations present in the peritoneal cavity at 24 and 48 h post inoculation  were immunized by the i.m. route with DNA-Tau4R2N (or empty DNA-Φ as a control) and, 4 weeks later, boosted by the i.p. route with MVA-Tau4R2N or MVA-WT, as described in Materials and Methods and in n = 4 mice; 13 weeks old) inoculated with 2 doses of PBS at weeks 13 and 17 were used as control animals. At week 31, animals were sacrificed and the levels of hyperphosphorylated tau in the brain were determined by Western blotting analysis in hippocampal samples  with DNA-Tau3RC  and 4 weeks later (week 26) infected them with MVA-Tau3RC or MVA-WT, as described in Materials and Methods and Next, we further analyze whether MVA-Tau3RC could control the tau pathology in vaccinated P301S mice by avoiding or diminishing the motor failure, increase survival rate and reduce the hyperphosphorylated tau. Thus, we vaccinated P301S mice  agonists, such as type B CpG oligodeoxynucleotides (ODNs) is an effective and safe method to reduce tau-related pathology in AD mouse model ,57.+ or CD8+ T cells in the spleen of immunized mice at the peak of the response (10 days after the boost and following stimulation with tau peptides), although we could detect VACV-specific CD8+ T cells, reinforcing that the lack of an immune response is tau-specific. Moreover, binding antibodies against human tau protein were not detected in sera obtained from those immunized animals. The absence of tau-specific T-cellular and humoral immunogenicity could be due to a stabilization of the microtubule network induced by the larger human brain tau isoform (Tau4R2N) expressed by MVA-Tau4R2N or by the tau 3RC mutant protein expressed by MVA-Tau3RC, which would lead to an impairment of clonal expansion upon activation, finally resulting in a poor activation of the immune system of the mouse model, and a lack of function of the immune cells. Moreover, the presence of a microtubule-bound human tau protein may also result in the absence of a tau protein bound to the external membrane or the lack of a secreted tau protein, hence failing to be presented to T and B cells.The analysis of the tau-specific immunogenicity in immunocompetent BALB/c and C57BL/6 mice vaccinated with a DNA prime/MVA boost immunization protocol (either DNA-Tau4R2N/MVA-Tau4R2N or DNA-Tau3RC/MVA-Tau3RC) showed no tau-specific CD4Furthermore, the efficacy study in the transgenic P301S mice showed that neither DNA-Tau4R2N/MVA-Tau4R2N nor DNA-Tau3RC/MVA-Tau3RC induced a significant reduction in the levels of hyperphosphorylated tau in the hippocampus and the dorsal dentate gyrus of vaccinated mice. Moreover, no control in the motor failure and mortality of vaccinated transgenic P301S mice was observed. To explain the failure to control the progression of the AD in immunized mice by both DNA-Tau and MVA-Tau vaccine candidates, we suggest that the expression of the human tau proteins (either Tau4R2N or Tau3RC) in immune cells may result in a loss of proliferation due to the microtubule stabilization promoted by these proteins. Thus, the presence of Tau4R2N or Tau3RC may prevent the microtubule depolymerization required for mitosis during cell proliferation. Therefore, future experiments will be performed to evaluate the expression by MVA of other tau isoforms or optimized tau fragments with a diminished microtubule stabilization, such as the C-terminal region of tau that has also been described as the most immunogenic region ,58, or oIn conclusion, we described here for the first time the generation of novel vaccine candidates against AD based on MVA vectors expressing either the full-length human Tau 4R2N isoform (MVA-Tau4R2N) or the human tau 3RC mutant protein (MVA-Tau3RC). Well-established and potent prime/boost immunization protocols using DNA-Tau and MVA-Tau vaccine candidates did not induce tau-specific T-cell and humoral immune responses or a significant protection against AD-like disease in transgenic P301S mice. These results open the path to generate novel MVA-based vectors expressing optimized tau antigens that could elicit a better tau antigen presentation to the immune cells and to develop novel vaccination protocols that could control AD progression.


In criminal trials, evidence often involves a degree of uncertainty and decision-making includes moving from the initial presumption of innocence to inference about guilt based on that evidence. The jurors’ ability to combine evidence and make accurate intuitive probabilistic judgments underpins this process. Previous research has shown that errors in probabilistic reasoning can be explained by a misalignment of the evidence presented with the intuitive causal models that people construct. This has been explored in abstract and context-free situations. However, less is known about how people interpret evidence in context-rich situations such as legal cases. The present study examined participants’ intuitive probabilistic reasoning in legal contexts and assessed how people’s causal models underlie the process of belief updating in the light of new evidence. The study assessed whether participants update beliefs in line with Bayesian norms and if errors in belief updating can be explained by the causal structures underpinning the evidence integration process. The study was based on a recent case in England where a couple was accused of intentionally harming their baby but was eventually exonerated because the child’s symptoms were found to be caused by a rare blood disorder. Participants were presented with a range of evidence, one piece at a time, including physical evidence and reports from experts. Participants made probability judgments about the abuse and disorder as causes of the child’s symptoms. Subjective probability judgments were compared against Bayesian norms. The causal models constructed by participants were also elicited. Results showed that overall participants revised their beliefs appropriately in the right direction based on evidence. However, this revision was done without exact Bayesian computation and errors were observed in estimating the weight of evidence. Errors in probabilistic judgments were partly accounted for, by differences in the causal models representing the evidence. Our findings suggest that understanding causal models that guide people’s judgments may help shed light on errors made in evidence integration and potentially identify ways to address accuracy in judgment. Legal decision making often involves causal reasoning under uncertainty. Jurors who make decisions in criminal cases are tasked with dealing not only with inherent uncertainty of a myriad of facts but also with disentangling the complexity of causal relations. For example, criminal law draws a distinction between factual and legal causes . FactualMany of these aspects of legal cases can be represented using Causal Bayesian Networks (CBN). CBNs  represenThe present study draws on an existing body of literature, according to which probabilistic learning and reasoning approximates Bayesian principles . RottmanResearch by One area of difficulty in quantitative updating concerns the interpretation of competing causes. When two independent causes can explain a common effect, observing that this effect is present, increases the probability of both causes. However, if one then receives evidence that one of the causes has occurred, the probability of the other cause decreases. This pattern of judgment is known as ‘explaining away’ . It suggIn explaining away situations people struggle with both qualitative and quantitative aspects of judgments . From thResearch by Another bias that people exhibit when reasoning about competing causes is the zero-sum fallacy. Zero-sum reasoning broadly represents thinking where gains in one area take place at the expense of another’s losses. In the context of causal reasoning, this is represented by treating evidence in support of a given cause as evidence against an alternative cause. In a recent study by A balanced evaluation of evidence in legal cases involves weighing up evidence against competing hypotheses. These hypotheses are often about the causes that lead to outcomes under examination. Making accurate inferences requires not only correct interpretation of the weight of evidence, but also being able to correctly identify the hypotheses against which evidence is tested. Hypotheses can be considered mutually exclusive and exhaustive only when one (and only one) of the hypotheses can be true, ruling out any other explanation. For example, someone either dies from natural or unnatural causes. However, evidence in reality rarely warrants exclusivity and exhaustiveness of causes. There are usually many unknown possible causes of any piece of evidence. Being able to differentiate hypotheses that are not mutually exclusive and exhaustive is critical to avoiding the zero-sum fallacy, which occurs when hypotheses that are not mutually exclusive and exhaustive are erroneously treated as such.Inferences from causes to effects represent predictive reasoning and moving from effects to causes corresponds to diagnostic reasoning. In a study of diagnostic causal reasoning with verbal probabilistic expressions, such as “frequently,” “rarely,” “likely” and “probably,” Diagnostic reasoning is underpinned not only by probabilistic judgments about cause given effect, but also by causal relations that connect causes to effects . The plaWhile the normative interpretation of the Bayesian formula implies that beliefs about guilt will be updated based on evidence, the judgments may not always match the quantitative Bayesian norms even when the qualitative interpretation is accurate.Previous research suggests that jurors are competent at evaluating scientific evidence but tend to show systematic errors in processing quantitative evidence under certain circumstances . The disThe present study explores how people update beliefs in light of evidence, examining alignment with Bayesian norms from a qualitative (direction of updating) as well as a quantitative (numeric computations) perspective. The study focuses on the following aspects of causal reasoning: (1) predictive inferences from effects to causes; (2) diagnostic inferences from causes to effects; and (3) explaining away inferences with competing independent causes.The present study is based on a summary of a real case where a couple was accused of intentionally harming their baby. In this case, a young child was brought to hospital by his parents because they noticed the child had bleeding in his mouth. The parents had no explanation for the bleeding, and said that the child had not been involved in an accident. In our experiment participants are given two possible causes for the bleeding: abuse and a rare blood disorder.Participants are provided with information about the hospital admission rates for children with this symptom for cases of abuse and rare blood disorder. The story mentioned that figures from previous hospital admissions suggest that 1 in 100 children admitted with bleeding to the mouth have been abused by their parents, and 1 in 1,000 have the rare blood disorder.After presenting background information, further evidence was presented one piece at a time. This involved information about:(1)Doctors noticing bruising on the child(2)The hospital radiologist carrying out an X-ray on the child and reporting that the X-ray showed fractures.(3)The child being tested for the blood disorder and testing positive.(4)An independent expert radiologist employed by the prosecution re-examining the X-ray results and claiming there were no fractures.The causal structure of the case is presented in The main goal of Experiment 1 was to examine whether participants’ beliefs are updated in line with Bayesian norms when dealing with competing causes (abuse and blood disorder) in a sequential inference task. Evidence was presented in stages, one piece of evidence at a time.155 participants were recruited through Amazon Mechanical Turk to take part in the study. In all experiments, participation was restricted to respondents who had at least a 95% approval rating for their previous MTurk work. Participants were English speakers and based in the United States. Participants who were unable to correctly answer the comprehension check questions regarding the underlying causal structure, were excluded from the analysis, leaving 127 participants . The mean age was 33.9 . Out of the 127 participants included in the study, 36.2% had an undergraduate degree, 10% – Masters or Ph.D. degree, 32.3% completed a college education and 22% had no qualification.In the present study base rates are presented in a frequency format as research  suggestsThe following evidence was presented in stages: bruises, a hospital x-ray expert’s report, blood test results and an independent x-ray expert’s report. To examine the possibility of zero-sum reasoning when assessing evidence  we varieParticipants were divided into three groups according to the presentation format for the abuse and causes. The experiment consisted of the following conditions:Condition 1: Abuse and disorder were presented as non-exclusive causes of the child’s bruises and bleeding.Condition 2: Abuse and disorder were presented as non-exclusive and non-exhaustive causes of the child’s bruises and bleeding.Condition 3: Control condition contained no statement about the relationship between abuse and disorder as causes of the child’s bruises and bleeding.The dependent measures included the probabilistic judgments about the abuse and disorder as causes of the child’s symptoms. The probability judgments were recorded after introducing the background information as well as after exposure to each new element of evidence .Information presented to participants specified that bruising was a common consequence of abuse and also of the blood disorder. It further stated that fractures were a common consequence of abuse, but not of the blood disorder.Conditional probabilities elicited from the participants at the end of the experiment were used to construct their models of evidence evaluation based on Bayesian reasoning. Individual Bayesian network models were constructed for each participant. Actual probability judgments were compared to those predicted by these models. The differences between the probabilistic judgments predicted by the subjective models in line with Bayesian norms and the actual probabilistic judgments formed a dependent measure in this experiment. Probability judgments were compared to the individual causal models inferred from conditional probabilities. Subjective priors were compared to base rates supplied in the introduction.The probability judgments in the experiment were recorded on a scale of 0 to 100%. Participants used an on-screen slider with numerical values to indicate their answer. Questions about the probability judgments used the following format: “What are the chances of …?”. On the slider response scale, 0% was labeled as “Very unlikely” and 100% as “Very likely”.The magnitude of updating from one stage of evidence to another was calculated as a difference between the probability estimates at the present and previous evidence stage, at Stage 2 (Bruises), Stage 3 , Stage 4 (Blood test results), and Stage 5 (Independent expert report).1. Participants were given a legal case  = 85.298, p < 0.001, and partial eta squared = 0.408. Pairwise comparisons indicated that there was a statistically significant shift in beliefs about abuse at each stage of evidence, suggesting that participants integrated evidence and revised beliefs following the presentation of evidence. There was no main effect of Condition, F = 1.201, p = 0.304.The effect of Evidence and Condition on the abuse probability judgments in the observed data was examined with a mixed ANOVA with Condition as a between-subject and Evidence Stage as a within-subject variable. Following a Greenhouse-Geisser correction, the main effect of evidence was statistically significant, p < 0.001, partial eta squared = 0.794. There was no effect of Condition, F = 0.127, p = 0.881. Post hoc comparisons using the Bonferroni test indicated no difference between prior beliefs  and revised beliefs after evidence about bruises . All other stages of belief updating, including the first expert’s report , blood test results  and the second expert’s report  showed differences in beliefs compared to the previous stage. Subjective priors were considerably higher  than the objective priors  supplied as part of the case scenario.A mixed ANOVA was carried out to explore the effect of Evidence Stage and Condition on the observed subjective probability judgments for disorder. Similar to the abuse probability judgments, a significant main effect of Evidence was found following a Greenhouse-Geisser correction, F = 479.268, Individual Bayesian belief updating models were obtained using thegRain package in R . DiffereAn example Bayesian belief updating model is presented in Differences between the observed and predicted judgments were found to be significant for all pieces of evidence, including bruising symptoms, hospital radiologist’s report and test results with regards to the abuse and disorder-related probability judgments indicating that the participants’ probabilistic judgments were different from exact Bayes computations. However, judgments were qualitatively in the right direction. There was no significant difference between conditions in belief updating, indicating that making the non-exclusivity and non-exhaustiveness of causes explicit did not affect probability judgments.Experiment 2 focused on testing belief updating when participants were explicitly told, just before each probability judgment, that the causes in the study were non-exclusive and non-exhaustive. The purpose of this experiment was to determine the effect of bringing participants’ attention to the non-exclusivity and non-exhaustiveness of causes on the accuracy of judgments. This would allow us to rule out lack of understanding of the causal structure as a contributing factor to biased judgments observed in Experiment 1. The following statement was included at every stage of subjective probability elicitation: “Note that it is possible that both causes are true: e.g., that a child has been abused and has the disorder; it is also possible that neither are true, and that the symptoms arise due to other causes.” Additionally, participants’ understanding of the case causal structure was tested.93 participants were recruited using the same protocol as in Experiment 1. As in Experiment 1, participation was restricted to respondents who had at least a 95% approval rating for their previous MTurk work. Participants were English speakers and based in the United States. The mean age was 35.08 . Out of the 93 participants  included in the study, 44% had an undergraduate degree, 21.5% – Masters or PhD degree and 23.7% completed a college education.The procedure, instruction, and materials, including the questions were identical to those used in Experiment 1 except there was only one Condition, which corresponded to Condition 2 in Experiment 1. Additionally, at the end of the task we included questions to elicit participants’ causal models, focusing on the links included in the case model . QuestioF = 300.192, p < 0.001, partial eta squared = 0.802], observed and model judgments based on Bayesian predictions , but not between the abuse and disorder probability judgments .A repeated measures ANOVA with a Greenhouse-Geisser correction showed that mean probability estimates differed significantly between the evidence presentation stages [abuse = 0.467, SD = 0.313; Mdisorder = 0.441, SD = 0.325) than the objective priors  supplied as part of the case scenario.Subjective priors were higher , which led to the under-weighting of evidence and belief updating far lower than mandated by Bayesian normative judgments. Another instance of inaccurate quantitative judgment was following the positive blood test results, which showed that participants attributed excessive weight to evidence.The findings from both experiments suggest that in legal decision making people qualitatively update their beliefs in line with Bayesian norms. In our experiments, participants’ belief updating was qualitatively aligned with normative judgments, i.e., probability judgments increased or decreased in the same direction as predicted based on Bayesian norms. This was observed with both predictive inferences , diagnostic inferences  and explaining away inferences . While most judgments fit with qualitative predictions of Bayesian models, an exception is observed at the final stage of evidence presentation where the subjective probability of blood disorder was lowered slightly rather than raised. This can be explained by exposure to conflicting expert reports, which may have decreased the perceived reliability of reports, resulting in a greater skepticism toward the blood test results.Overall the results indicate that people’s qualitative reasoning is mostly accurate and follows qualitative predictions of Bayesian models in predictive, diagnostic and explaining away inferences. These findings reinforce results from previous studies where Bayesian probabilistic reasoning was observed e.g., .Results from both experiments indicate that people tend to ignore the priors provided as part of the background case information and set their own subjective priors. The subjective priors in both experiments were significantly higher than the objective priors offered in the case summary. Prior knowledge and expectations, underlying causal models may have contributed to setting higher priors than suggested by base rates.Previous research about explaining away inferences is not conclusive  and offePrior evidence on people’s ability to make quantitative probabilistic judgments aligned with Bayesian norms is not definitive, with studies indicating either under- or over-estimation of evidence. In the existing body of literature, a unified mechanism for explaining an excessively low and high weight attributed to evidence has not been decisively established. Our findings show both types of departures from quantitative normative judgments: under-weighting of evidence when participants update beliefs based on the evidence of bruises and over-weighting of evidence following the evidence of the positive test result. Both these findings can be explained with zero-sum reasoning, which provides insights into how people integrate evidence when dealing with competing causes .Zero-sum reasoning represents thinking whereby the gains of one person take place at the expense of another’s losses. A zero-sum model of the world presumes a finite and fixed amount of resources in the world, which necessitates a competition for these resources. In the context of competing causes, when two causes equally predict the same evidence, zero-sum thinking treats such evidence as neutral because it tacitly assumes that the causes are exclusive and exhaustive accounts of the evidence. For example, in our experiments the evidence of bruises which was predicted by both the abuse and disorder hypotheses, the evidence was treated as neutral, resulting in only slight increase in the probability of both, which was considerably lower than expected by Bayesian updating.Zero-sum thinking also accounts for the over-weighting of evidence which was observed in the case of the excessive decrease in the probability of disorder following the hospital radiologist report and an excessive increase in the probability of disorder given the positive blood test result with a simultaneous disproportionate lowering of the probability of abuse. Excessive raising or lowering of probabilities points to the zero-sum nature of the reasoning involved in this process. Competing causes were perceived as exclusive, which had a hydraulic effect on the evidence interpretation: increasing the probability of one cause excessively decreased the probability of the other cause.These findings are consistent with the results of Zero-sum reasoning in the context of our experiments suggests that under- and over-estimation of evidence are observed due to underlying assumptions about causes modeled on zero-sum principles. This type of reasoning may result in more accurate judgments when dealing with competing causes that are exclusive and exhaustive.Our study suggests that people are able to make qualitatively accurate causal inferences and update beliefs in the direction predicted by Bayesian norms. However, quantitative computations are not always accurate and show a gap between observed and normative judgments. Instances of underweighting and overweighting of evidence in our experiments can be explained by a zero-sum fallacy. This offers a useful perspective for shedding light on evidence integration in legal cases, where a balanced evaluation of evidence often involves weighing up of the evidence against competing hypotheses. These hypotheses are often about the causes that lead to outcomes under examination. Making accurate inferences requires not only correct interpretation of the weight of evidence, but also being able to identify the hypotheses against which evidence is tested. Being able to differentiate hypotheses that are not mutually exclusive and exhaustive is critical to avoiding a zero-sum fallacy.The datasets generated for this study are available on request to the corresponding author.The studies involving human participants were reviewed and approved by Ethics Committee of the Department of Experimental Psychology, University College London. The patients/participants provided their written informed consent to participate in this study.Both authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The handling editor declared a shared affiliation, though no other collaboration, with the authors at the time of the review.


National health information (HI) systems provide data on population health, the determinants of health and health system performance within countries. The evaluation of these systems has traditionally focused on statistical practices and procedures, and not on data use or reuse for policy and practice. This limits the capacity to assess the impact of HI systems on healthcare provision, management and policy-making. On the other hand, the field of Knowledge Translation (KT) has developed frameworks to guide evidence into practice.A scoping review of the KT literature to identify the essential mechanisms and determinants of KT that could help monitor the impact of HI systems.We examined 79 publications and we identified over 100 different KT frameworks but none of these were focused on HI systems per se. There were specific recommendations on disseminating evidence to stakeholders at the institutional and organizational level, and on sustaining the use of evidence in practice and the broader community setting.We developed a new model, the HI-Impact framework, in which four domains are essential for mapping the impact of national HI systems: (i) HI Evidence Quality, (ii) HI System Responsiveness, (iii) Stakeholder Engagement and (iv) Knowledge Integration. A comprehensive impact assessment of HI systems requires addressing the use of HI in public health decision-making, health service delivery and in other sectors which might have not been considered previously. Monitoring Stakeholder Engagement and Knowledge Integration certifies that the use of HI in all policies is an explicit point of assessment. In 2016, the World Health Organization (WHO) presented an action plan to strengthen the use of evidence for policy-making in the European Region.9,10 The current Joint Action on Health Information, InfAct, which gathers 40 public health institutes from 28 European countries also embodies this will for a more integrated HI strategy.3,4 Whereas the vision is clear, it requires monitoring the impact of national HI systems on reducing the evidence-to-practice gap.11 In this context, we define impact as the demonstrated use of evidence by stakeholders for decision-making and intervention in public health policy and practice.,12National health information (HI) systems compile evidence on population health, the determinants of health and health system performance within countries (i.e. HI).15 Yet, evaluations of these systems have traditionally focused more on statistical data processes and data quality, and less on how data are integrated to practice.1 On the other hand, the field of Knowledge Translation (KT) has developed frameworks that guide ‘the appropriate exchange, synthesis and ethically sound application of knowledge to interventions that strengthen the healthcare system and improve health’.16,17 Therefore, our aim with this study was to draw from the lessons learned in the field of KT in order to derive a new HI system evaluation framework—one based on monitoring the impact of national HI systems in healthcare, policy-making and service delivery, as well as in other sectors which might not have been considered previously.The Performance of Routine Information System Management (PRISM) framework was seminal in identifying the key features of efficient HI systems, including the use of information for decision-making.18–20 HI systems contribute to sound decision-making at all levels of the health system.2 End-users have different roles ranging from providing care, implementing screening programmes, planning interventions and policy-making. Therefore, the scope of our study bridges across different fields of practice which warranted a system-wide approach, and a broad search strategy. We conducted a scoping review of KT frameworks; this methodology answers far-reaching questions, and is more amenable than systematic reviews to explore the conceptual boundaries of a given topic.21 We investigated PubMed, Medline and the Cochrane Database of Systematic Reviews between August and December 2018, using common terms used to define KT in the literature. Straus et al.22 note that ‘the terms implementation science or research utilization are common in Europe, while in the United States, the terms dissemination and diffusion, research use, knowledge transfer, and uptake are often used. In Canada, the terms knowledge transfer and exchange and knowledge translation are employed’. We built our search using the following terms combined and truncated*: knowledge*, translation, health data*, dissemination, implementation, adaptation, evidence informed*, evidence based*, public health, health policy, healthcare, impact, health system*, performance, health information*, tool, framework, monitoring, evaluation .aluation . We furt23 according to process-based (i.e. on the mechanisms and stages of evidence-use), or determinant-based frameworks (i.e. on the barriers and facilitators to evidence-use) operating at organizational and/or institutional level. Per standard scoping review methodology, we included publications even when the risk of bias was not explicitly assessed.21 We excluded, however, study protocols, commentaries, editorials and opinion pieces, or when the full-text article could not be retrieved . From each source, we extracted the following:Characteristics: author and citation; country of origin; publication year; and the number of studies/frameworks included in the review.Content: Type of framework (e.g. KT processes/determinants), socio-ecological level ; KT stage (e.g. dissemination and/or implementation), and field-use .We conducted a thematic content analysis which consisted in examining (i) the mechanisms, and (ii) the determinants underpinning the use of evidence in practice. We defined practice as the act of intervening  in healthcare provision, service planning and delivery, policy-making and evaluation.24In total, we screened 533 publications, and we included 79 publications on the mechanisms and determinants of KT. Reviews came primarily from the USA, Canada, Australia (65%) and the UK (22%). Across publications there were large differences in the terminology that was used—with respect to how evidence is defined , or embedded into practice . Although our search spanned the last 10 years, most reviews were published in the last 5 (67%). There were many more KT publications in healthcare (53 studies), versus policy (16) or community health (8), and two reviews in public health in general. We saw an evolution in the publications, with a greater emphasis on KT in healthcare in the beginning of the study period, and over time an increasing focus on the use of evidence in health policy, and ultimately at the level of health system. This could indicate a growing trend in recognizing the importance of interdisciplinary interventions in KT, as also noted by Vollmar et al.25 and 15926 different KT frameworks, and as shown in 17,22,23,25–65 In general, KT can be summarized as a dynamic and iterative cycle in which data-providers ‘push’9 information into the hands of a target audience—which is known as the dissemination stage.28 In turn, data-users can choose to ‘pull’9 this evidence for intervention in a particular setting —which is known as the implementation stage.28 Feedback from the field guides further practice changes, and informs future data needs thereby completing the process. In our study, there were 19 reviews on the dissemination of evidence to stakeholders, 26 on the implementation of evidence into practice. Another 34 reviews were focused on both aspects of dissemination and implementation highlighting their interdependency in the KT process. Some KT frameworks were applied in specific areas of health, namely: cancer research,48,49,58,66,67 child health and welfare,38,,63,68,69 chronic illness,26,53,70,71 cardiovascular health,72 healthy aging,41,54,73 immunization,74 acute and primary care,27,71,75 mental health,37,38,51,68,73 nursing,76 physiotherapy,61 chiropractic,77 rehabilitation,39 renal replacement therapy53 and genomics.47The studies we included examined between 49,22,28–31,33,34,36,37,45,52,54–56,64,78–80 We identified three main attributes of data quality: statistical accuracy, content relevance and coverage.9,11 Standardized methodological approaches, data innovations and advanced statistical methods can improve the accuracy of evidence provided to stakeholders.67 On the other hand, content relevance is ensured by providing the data needed by stakeholders for planning, care or service delivery .33,81 Adequate coverage in the data provides for more equity in the interventions.9,30,36,82 For instance, disaggregated data by age, gender or socio-economic status are needed to plan actions that target vulnerable segments of the population, diverse communities or subgroups at the highest health risk.67,83,84A common starting point in the KT frameworks we reviewed was the generation of high-quality data.39 note that strategies range from passive methods such as printed educational material to more active multicomponent strategies involving audit and feedback from local opinion leaders, while only relying on printed material is regarded as a ‘reasonably ineffective KT strategy’. Two methods, however, have been recognized by Blessing et al.9,39 as particularly useful: (i) Exchange methods in which data-providers and policy-makers for instance work in partnership, as well as (ii) Integrated methods ‘where a knowledge translation platform is institutionalized in an organization or in the broader health system’. When stakeholders have access to evidence, this can spearhead problem recognition and the development of meaningful solutions in the field.9,24 These include actions whether legal, financial or political that align with national and international health targets.25,38,59,81,88We found that although data production initiates the KT process, dissemination carries it through. Jones et al.33 emphasize that there are three broad types of knowledge use: instrumental (i.e. for direct application in the field), conceptual (i.e. for problem-solving and agenda setting) and symbolic . Donaldson et al. suggest monitoring intermediate shifts in stakeholders’ knowledge, attitude and skills89 as proof that evidence has been accessed and used,90 granted few frameworks provide specific KT outcome or impact measures.91,92During the implementation stage, Davies et al.45,52 For instance in the prominent Re-aim framework, there are five dimensions to consider: reach (R) and effectiveness (E), operate essentially at individual-level; whereas adoption (A), implementation (I) and maintenance (M) are focused on organizational and community-level change.45 Another highly cited model, the Knowledge-to-Action framework is based on two components: (i) the Knowledge creation funnel in which research data is generated, synthesized and contextualized to become more useful, and (ii) the Action cycle which includes a range of actions targeting changes in stakeholders’ level of understanding, behaviours and attitudes.45 Other models explicitly rely on the latest developments in health information technology for the dissemination and implementation stages. For instance, in the Learning Healthcare Systems framework, electronic health records and data mining facilitate exchanges between data-users and data-providers; this model also puts emphasis on shared decision-making between patients and care providers with the aim to bring quality improvements in clinical care.93Although in general, KT frameworks share the aforementioned characteristics, we found notable differences in the intricacy of the data-provider/data-user interaction across publications.9,16,17,80,94,95 We have summarized KT determinants by field of practice  in 11,15,33,35,41,54,59,66,81,87,88,9,41,42,54,56,59,64,67,80,81,96,99–There were 58 reviews which explored the determinants of the use of evidence in healthcare, public health policy-making and service delivery. Among which, six were focused on the tools (i.e. checklists and guidelines) which have been developed to facilitate the access to, and the use of evidence.56 In policy, a setting that has a ‘stable, clear and decentralized decision-making authority’ is said to have a greater capacity to adopt innovations.102 When leaders commission or champion evidence themselves this can facilitate the KT process.9,100 Inter-sectorial structures (i.e. the EU Joint Research Centre) and institutionalized knowledge brokers can also play a role in disseminating evidence in a whole-of-government approach.33 A common understanding between data-providers and data-users, a sense of trust, a shared vision among stakeholders and the provision of resources also facilitate KT . We found that engaging frontline staff and health managers in improving data collection and its use is crucial for team transformation and quality improvement107; capacity building and workforce development also.31,43,61,70,72,81,95,108 A culture of evidence-use can also be supported by health information technology to facilitate a rapid access to evidence .41,68,80,94,109 Local opinion leaders,85 greater social cohesion in medical facilities,41,80,106,110 performance rankings,14 providing resources and financial incentives further contribute to KT at organizational level.9,14,43,59,79,100,106In medical facilities, Reschovsky et al.25,30,31,35,40,42,59,65,66,67,69,85,97,110,111; although Davies et al.33 noted that very few KT strategies targeted the general public. In a study of adaptations in evidence-based interventions, the most cited concern was the need for cultural appropriateness, and acceptability.35 Common adaptations included content (100%), context (95.2%), cultural modifications (73.8%) and delivery (61.9%).35 This relies on the consolidation of partnerships for health, and outreach to key players in the communities where the interventions would take place. Haynes et al.112 note that platforms for cross-sectoral collaborations can increase decision-makers’ capacity to use evidence. This highlights the importance of appraising potential conflicts of interests across stakeholder groups113 when KT is the aim.There were additional determinants related to the sustainability of the KT process. Fifteen reviews highlighted that embedding evidence within social contexts is essential for achieving system level change within organizations and local communitiesFinally, we have synthesized our finding into four domains that are relevant for capturing the mechanisms and determinants that drive KT. The domains of ‘HI Evidence Quality’, ‘HI System Responsiveness’, ‘Stakeholder Engagement’ and ‘Knowledge Integration’ constitute the building blocks of the HI-Impact framework shown in 37 and Straus et al.,22 the concept of KT is an umbrella term and inconsistencies in the terminology made it more difficult to screen eligible publications. There is a lack of guidance in choosing one KT framework over another, many KT frameworks have not been evaluated, and consensual outcome measures of KT are lacking.102,114–116 Nevertheless, a strength of this scoping review is the breadth of our research, we have examined 79 reviews of KT frameworks encompassing 3916 studies. Furthermore, a new HI systems evaluation framework focusing on impact is highly relevant for Europe given that PRISM was originally intended for strengthening systems in low- and middle-income countries,15 and the KT tools from other high-income countries such as the USA or Canada were not intended for the evaluation of national HI systems.Securing evidence for policy development and intervention is not a strictly linear process. KT relies on feedback and exchange mechanisms between data-providers and data-users working in government, clinics, health agencies and in the broader community. To our knowledge, this study is the first to propose an original framework for monitoring the impact of national HI systems based on lessons from the field of KT. However, we experienced several methodological limitations during the scoping review. As stated by Goldner et al.24,117 However, ensuring that routinely collected data are fit-for-use and decision-making can be a challenge. HI systems collect dispersed data from various sources, including systems which were not initially established for public health surveillance purposes . The quality of routinely collected HI can also vary, and there are noted issues with the ascertainment of causes-of-death statistics for instance across countries. Despite these limitations, routinely collected data are often the best evidence available at population level for decision-makers. Therefore, ensuring that robust and timely evidence is generated and accessed by stakeholders is important, and this is why we have included ‘HI Evidence Quality’ as one of our domains .121 Given the importance of dissemination in the KT process, we propose an original application of the concept of ‘Responsiveness’ to HI systems. A responsive HI system would aim to facilitate the access to, and use of HI by lifting the technical, behavioural, organizational and contextual barriers to evidence-use  might also play a role in the evaluation of ‘Responsiveness’. When data are presented in international conferences and high-impact journals, synthesized for decision-makers,124 and also presented in plain language summaries, and in social media,125 this can increase the impact of HI systems. For example, Euro-Peristat, a European maternal and child health information network uses a wide array of dissemination strategies to reach their stakeholders in 31 countries. Euro-Peristat results are published in comprehensive reports for clinicians and policy-makers, presented in international scientific conferences and peer-reviewed journals, on their project website and in a quarterly newsletter.In our framework, monitoring ‘HI Evidence Quality’, and ‘HI System Responsiveness’ would provide insight on the availability and accessibility of robust and credible evidence. This could entail monitoring whether the evidence provided to stakeholders adheres to the FAIR: findable, accessible, interoperable and reusable data principles.Healthy Life Years,13 and the Disability Adjusted Life Years illustrate the need for HI systems to provide data that resonate with the complex problems that policy-makers face in health.126 Yet, the lack of joint understanding can negatively influence the KT process, and the sustainability of evidence-based interventions in the field.59,98 For example in 2018, the French Government pushed for the reduction of the speed limitation from 90 to 80 km/h−1 (50 mph) on two-lane highways. Despite data on the number of avoidable deaths (300–400 per year), and an increasing trend in road mortality since 2014, this measure was met with significant resistance from local authorities. Ultimately, national enforcement was amended to voluntary implementation by the regions. (https://www.theguardian.com/world/2018/jan/10/france-cuts-speed-limit-rise-deaths). Politics and external actors with vested interests also influence policies—this can lead to dissonance when objectives across sectors do not align to improve health.12,127 In Europe, the Joint Research Centre, EVIPnet and the European Health Observatory facilitate the continued use of evidence in policy development.Our review of KT determinants also highlighted the importance of organizational readiness for evidence-based practice. Therefore, monitoring HI system use within healthcare facilities, and institutions is the third domain in our framework  which we128 model of the layers of influence on health, and the more recent focus on ‘Health in all policies’.86 Therefore, we have included ‘Knowledge Integration’ in civil society and across sectors as the fourth domain in the HI-Impact framework. This domains relates to the effective incorporation of knowledge into the decisions, practices and policies of organizations and systems.65 With the Sustainable Development Goals, countries have been encouraged to build alliances for health in all policies; yet these integrated approaches are currently not reflected in the way national HI systems are being evaluated.86 In their review of HI system evaluation frameworks, Eslami et al.1 stressed that the role of context has been largely neglected thus far, and requires more attention.Monitoring the impact of HI systems within the broader social context could provide valuable information on the extent of HI system use. This is because health is largely influenced by determinants outside the health sector as recognized in Dahlgren and Whitehead’s,130 Therefore, increasing scientific and health literacy could constitute another important lever of impact for HI systems.Together, the domains of ‘Stakeholder engagement’ and ‘Knowledge integration’ aim to ensure that HI systems are providing data that are used in coherent interventions on the medical, social and commercial determinants of health. These domains expand the scope of existing HI system evaluation frameworks, by examining the contextual impact of HI systems in areas which might not have been considered before . In a recent study, commissioned by the European Parliament, it was noted that ‘faced with emerging threats relating to the spread of disinformation and pseudo-science (…) fostering scientific literacy can provide people with tools to navigate (…) the vast amounts of information exchanged in public debate.’ The inefficient use of healthcare services and poorer health also depends on the level of health knowledge, socio-demographic characteristics and education.how we might want to evaluate the impact of national HI systems, and why the HI-Impact framework domains could be relevant. However, input from public health professionals and policy-makers will be critical to operationalize what specifically should be evaluated in each domain. To this aim, we have conducted an online DELPHI exercise with experts in 38 countries, and the development of an evaluation tool based on the HI-Impact framework is currently underway. Further piloting in European public health agencies could contribute to a more balanced assessment of HI systems in each country—one that takes into account the quality of the evidence, as well as the health and societal impact of the systems generating these data.In conclusion, the field of KT could serve to secure current and future investments in HI systems, by drawing attention to the mechanisms and determinants of evidence-based public health practice and policy-making. In this study, we have answered M.D. is funded by a Marie Skłodowska-Curie Action Individual Fellowship, GA No 795051; D.I. is funded by HealthPros, a Marie Skłodowska-Curie Innovative Training Network, and GA No 765141.Conflicts of interest: None declared.Creating a culture of accountability for evidence-based decision-making requires close monitoring of health information (HI) use in policy, at the points of care and service delivery, and in the broader community.Current frameworks used to evaluate the performance of HI systems focus on data production; we however, encourage giving increased attention to the dissemination, relevance and use of HI in all policies, and by new actors in civil society.There should be a deliberate strategy to provide HI to stakeholders within and outside the health system for a greater societal impact.ckz164_Supplementary_DataClick here for additional data file.


Raine (2019) reviewed previous research on the neural correlates of antisocial, violent, and psychopathic behavior based on previous studies of neuroscience of morality. The author identified neural circuitries associated with the aforementioned types of antisocial behaviors. However, in the review, Raine acknowledged a limitation in his arguments, the lack of evidence supporting the presence of the neural circuitries. In this correspondence, I intend to show that some of his concerns, particularly those about the insula and cingulate cortex, can be addressed with additional evidence from recent neuroimaging research. In addition, I will propose that the additional evidence can also provide some insights about how to design future neuroimaging studies to examine the functionality of the striatum in the circuitries. Psychiatry Research in 2019 concerning neuromoral theory of antisocial, violent, and psychopathic behavior1. The author proposed a comprehensive model of the neural network of morality and antisociality to explain the neural-level mechanisms of antisocial behavior. The author referred to previous neuroimaging studies and meta-analyses to identity the aforementioned neural networks and proposed that the prefrontal cortex, amygdala, insula, and anterior cingulate cortex are included in both networks, while the striatum is included in the antisociality network. The author stated two limitations regarding the network model that he proposed: first, the involvement of the insula and cingulate cortex regions in the neural networks could not be sufficiently supported by previous neuroimaging studies and meta-analyses; second, evidence that supports the involvement of the striatum in the morality network is insufficient.A review article by Raine was published inAlthough Raine raised the aforementioned two concerns regarding the lack of supporting evidence, I suggest that recent research in the field of social neuroscience can provide some supports to his points. Herein, I introduce two supporting findings, one from online-based large-scale analyses of neuroimaging data, and the other from recent neuroimaging experiments focusing on brain circuitries associated with morality. These recent research findings will be able to provide evidence of the involvement of the insula and cingulate cortex, which are involved in both moral and antisocial brain circuitries. Furthermore, some additional evidence might be able to support the involvement of parts of striatal regions in the circuit of antisociality although the evidence is not sufficient to completely address the issue. At least, the additional evidence might be able to provide ideas about potential research questions and hypotheses focusing on the striatum in morality and antisociality.2. NeuroSynth automatically gathers coordinate information that is reported in published neuroimaging articles and performs meta-analysis of the gathered information. A result from a meta-analysis of 87 studies and 2,806 activation foci that are associated with a keyword “moral” demonstrates that the left insula and anterior and posterior cingulate cortices show significant common activity across moral task conditions . Moreover, a recently developed analysis tool, NeuroQuery3, also reported the consistent result. Unlike NeuroSynth, which only analyzes published papers’ abstracts, NeuroQuery analyzes the full article texts, so it shows better selectivity compared with NeuroSynth3. It employs machine learning to examine the pattern of neural activation associated with a keyword of interest3. When “morality” was explored by NeuroQuery, both the left and right insula and cingulate cortex showed strong association with the keyword across 68 studies . These results provide evidence that supports the involvement of the insula and cingulate cortices in the neural network of morality based on large-scale data. In fact, the three meta-analysis articles that Raine reviewed meta-analyzed relatively fewer numbers of neuroimaging studies (references6), so he could only partially and tentatively propose the involvement of the insula and cingulate cortex regions in the neuromoral network. Hence, I suggest that Raine’s argument about the involvement of insula and cingulate cortex in the circuitries can be at least partially supported by large-scale neuroimaging data and the result from NeuroSynth and NeuroQuery. Moreover, the reported involvement of the left insula may suggest the possibility of laterality effects that Raine mentioned in his review, although more research that directly focuses on the laterality effects should be conducted.First, results from large-scale analysis of previous neuroimaging studies provides additional evidence supporting Raine’s model. Thanks to the development of information technology, performing meta-analysis of large-scale neuroimaging data has become feasible. A web-based analysis tool, NeuroSynth, is one exampleHowever, there are several caveats while interpreting these findings. First, although the involvement of the insula was consistently supported by two large-scale neuroimaging data analyses, some of the introduced meta-analyses did not report such a result. One point related to the large-scale analyses that should be considered is that compared with meta-analyses that were conducted with human-involved literature review, such automated analyses, particularly NeuroSynth, simple analyze texts per se so their selectivity might not be optimal. This point warrants further studies that more carefully explore large-scale database. Second, none of the meta-analyses and NeuroSynth directly demonstrated the association between the striatum with morality tasks. This might be due to that the analyzed previous studies were mainly about morality, not antisociality, so they might not be able to well address the neural correlates of antisociality.9 suggest that the insula and striatum regions showed significant activation and interaction with prefrontal and cingulate regions, which were indicated as core regions in the neural network of morality by Raine, in moral task conditions. The author tentatively proposed that increasing evidence may suggest that the striatum can be included in the morality network as well as the antisociality network. The new neuroimaging studies may provide additional evidence that supports the involvement of the striatum in the morality network. Our neuroimaging study from 20147 reported that the insula, cingulate cortex, and striatum (e.g. caudate and putamen) were significantly activated when participants were solving moral dilemmas .Are arguments sufficiently supported by evidence from the published literature or by new data and results?PartlyIs the conclusion balanced and justified on the basis of the presented arguments?PartlyIs the rationale for commenting on the previous publication clearly described?YesAre any opinions stated well-argued, clear and cogent?YesReviewer Expertise:NAI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. The author sought to provide additional support concerning the neuronal bases of the neuromoral model of antisocial behavior, initially proposed by Raine and Yang (2006) and recently updated by Raine (2019), who reviewed findings on brain mechanisms underlying moral decision-making and antisocial behavior that were published after the initial proposal. Specifically, this comment was intended to provide evidence of the involvement of the insula, cingulate cortex, and striatum in the circuits of the neuromoral model. To that end, the author uses: (a) NeuroSynth (which is not recent) to obtain meta-analytic data from neuroimaging studies of (supposedly) moral decision-making; and (b) results from his own neuroimaging studies focusing on brain circuitries associated with morality.per se, it is highly questionable whether the author provides stronger evidence than the one compiled by Raine in his own review to support his cautious arguments. It is worth mentioning that Raine refers to four meta-analyses, and most support the role of the cingulate in the neuromoral circuit, but he also explains that a recent meta-analysis by Ereset al. (2018) does not.  Although the NeuroSynth-based meta-analysis involved 87 studies, it should be noted that: (a) the NeuroSynth performs a semantic search limited to the abstracts of the papers; (b) many of the said studies do not report the involvement of the insula or the cingulate cortex in moral tasks; (c) the NeuroSynth lacks specificity, as moral decision-making is often studied with other constructs, which may have induced the observed brain activity; and (d) automated meta-analyses prevent the careful examination of relevant publication indices, such as the publication/confirmation bias. For these reasons and taking the NeuroSynth results into account, which do not seem substantial  Similarly, Raine reports meta-analytic studies that implicate the insula in morality, but he also explains that other meta-analyses do not (one of them was conducted by the author of this comment), leading to the conclusion that evidence is not sufficient to consider the insula as playing a role in moral decision-making. Moreover, given the fact that recent meta-analyses of morality have not directly implicated the striatum , Raine judiciously considers its inclusion in the neuromoral model debatable. The studies conducted and reported by the author of this comment do lend additional support to the possible role of the insula and striatum in moral decision-making, but such results do not provide stronger evidence than the meta-analytic findings compiled by Raine.  Summing-up, the arguments and results presented here do not make the findings regarding the implication of the cingulate, insula and striatum in moral decision-making any more consistent, nor any less debatable. Both the claim that “the insula, cingulate cortex, and striatum can be considered as parts of the neural network of morality”, and the conclusion that such claim “can be well supported by the analyses that I [the author] introduce here” sound like overstatements. Raine’s cautiousness is still well justified.  Minor issues: “circutries”, circutiries, and “evience” are typos.Are arguments sufficiently supported by evidence from the published literature or by new data and results?PartlyIs the conclusion balanced and justified on the basis of the presented arguments?PartlyIs the rationale for commenting on the previous publication clearly described?YesAre any opinions stated well-argued, clear and cogent?YesReviewer Expertise:Psychopathy, Antisocial Behavior, EEG/ERP, fMRI, NeuropsychologyI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. 1. Although the NeuroSynth-based meta-analysis involved 87 studies, it should be noted that: (a) the NeuroSynth performs a semantic search limited to the abstracts of the papers; (b) many of the said studies do not report the involvement of the insula or the cingulate cortex in moral tasks; (c) the NeuroSynth lacks specificity, as moral decision-making is often studied with other constructs, which may have induced the observed brain activity; and (d) automated meta-analyses prevent the careful examination of relevant publication indices, such as the publication/confirmation bias. For these reasons and taking the NeuroSynth results into account, which do not seem substantial per se, it is highly questionable whether the author provides stronger evidence than the one compiled by Raine in his own review to support his cautious arguments. It is worth mentioning that Raine refers to four meta-analyses, and most support the role of the cingulate in the neuromoral circuit, but he also explains that a recent meta-analysis by Eres et al. (2018) does not.Response: Thank you very much for your comments regarding the limitations of NeuroSynth and the point related to the involvement of the cingulate cortex presented in the meta-analysis. In the revised manuscript, I added some descriptions about the limitations of NeuroSynth that warrant a cautious interpretation of results. Simultaneously, as a way to partially address the issue, I introduced NeuroQuery that analyzes the full article texts with Machine learning:Moreover, a recently developed analysis tool, NeuroQuery 9, also reported the consistent result. Unlike NeuroSynth, which only analyzes published papers’ abstracts, NeuroQuery analyzes the full article texts, so it shows better selectivity compared with NeuroSynth 9. It employs machine learning to examine the pattern of neural activation associated with a keyword of interest 9. When “morality” was explored by NeuroQuery, both the left and right insula and cingulate cortex showed strong association with the keyword across 68 studies .Furthermore, I added a paragraph that discusses some additional caveats while interpreting the results from the large-scale analyses and meta-analyses:However, there are several caveats while interpreting these findings. First, although the involvement of the insula was consistently supported by two large-scale neuroimaging data analyses, some of the introduced meta-analyses did not report such a result. One point related to the large-scale analyses that should be considered is that compared with meta-analyses that were conducted with human-involved literature review, such automated analyses, particularly NeuroSynth, simple analyze texts per se so their selectivity might not be optimal. This point warrants further studies that more carefully explore large-scale database. Second, none of the meta-analyses and NeuroSynth directly demonstrated the association between the striatum with morality tasks. This might be due to that the analyzed previous studies were mainly about morality, not antisociality, so they might not be able to well address the neural correlates of antisociality.2. Similarly, Raine reports meta-analytic studies that implicate the insula in morality, but he also explains that other meta-analyses do not (one of them was conducted by the author of this comment), leading to the conclusion that evidence is not sufficient to consider the insula as playing a role in moral decision-making. Moreover, given the fact that recent meta-analyses of morality have not directly implicated the striatum , Raine judiciously considers its inclusion in the neuromoral model debatable.The studies conducted and reported by the author of this comment do lend additional support to the possible role of the insula and striatum in moral decision-making, but such results do not provide stronger evidence than the meta-analytic findings compiled by Raine.Response: I sincerely appreciate your comments regarding the involvement of the insula and striatum in the circuitries. I added some points from NeuroQuery while toning down my interpretations in general.In addition to these neuroimaging experiments, the result from NeuroQuery might also provide additional supports. Although the size of the identified region was small, NeuroQuery reported that when “morality” was entered, a part of the right putamen, which constitutes the dorsal striatum, showed a significant association with the keyword. Several previous fMRI studies have shown that the putamen was associated with intuition-related moral judgment 6, evaluation of moral intentionality of an actor in the relation with potential benefits and losses 10, and modulation of moral behavior 11. Hence, future studies may focus on the putamen as a possible candidate region to be included in the morality-antisociality circuitries based on these previous findings. Furthermore, although additional investigations with more focused experimental designs are warranted, such findings might be able to provide ideas about the involvement of striatal regions, particularly the putamen, in the morality-antisociality network.3. Summing-up, the arguments and results presented here do not make the findings regarding the implication of the cingulate, insula and striatum in moral decision-making any more consistent, nor any less debatable. Both the claim that “the insula, cingulate cortex, and striatum can be considered as parts of the neural network of morality”, and the conclusion that such claim “can be well supported by the analyses that I [the author] introduce here” sound like overstatements. Raine’s cautiousness is still well justified.Response: Thank you very much for your comments regarding the overall tone of my commentary. I agree with you that the limitations in the existing literature cannot completely confirm the involvement of the aforementioned regions in the circuitries. Thus, I toned down the introduction and conclusion accordingly. Instead of strongly argue that every single issue can be addressed by additional evidence, I proposed that the evidence is somehow suggestive. In other words, I proposed several directions for future research based on the added evidence. Also, as a way to strength my point, I additionally introduced NeuroQuery as well.These recent research findings will be able to provide evidence of the involvement of the insula and cingulate cortex, which are involved in both moral and antisocial brain circuitries. Furthermore, some additional evidence might be able to support the involvement of parts of striatal regions in the circuit of antisociality although the evidence is not sufficient to completely address the issue. At least, the additional evidence might be able to provide ideas about potential research questions and hypotheses focusing on the striatum in morality and antisociality.Given the aforementioned additional large-scale analyses and neuroimaging studies, we might have additional evidence that can support the point that the insula and cingulate cortex can be considered as parts of the neural network of morality. Particularly, both large-scale analyses, NeuroSynth and NeuroQuery, supported such a point although the meta-analyses introduced in Raine’s article demonstrated mixed results. One remaining issue is that although the association between the moral-antisocial circuitry and the striatum was partially supported by the additional neuroimaging experiments and NeuroQuery, NeuroSynth and the majority of the meta-analyses did not report such a result. Even if that is the case, the aforementioned results related to the functioning of the striatum might be able to provide researchers with some ideas for research question and hypothesis building. Given that the involvement of the striatum, particularly the putamen, was partially supported, future neuroimaging studies that intend to explore the antisocial circuitry may focus the point with more specialized research designs.4. Minor issues: “circutries”, circutiries, and “evience” are typos.Response: I appreciate your comments about the typos. In the revised manuscript, I corrected those points. The author provides a comment on a recently published article by Raine (2019) which is about a neuromodal theory of antisocial, violent, and psychopathic behavior.Major comments:The first claim is about the insula and cingulate cortex involvement in morality. According to the author, there is evidence that the insula and cingulate cortex are involved in morality based on a meta-analysis in Neurosynth with the word “moral”. It is stated that: “A result from a meta-analysis of 87 studies and 2,806 activation foci that are associated with a keyword “moral” demonstrates that the left insula and anterior and posterior cingulate cortices show significant common activity across moral task conditions .” I had a look at the result provided by the link, but I couldn’t see much evidence of insula of cingulate cortex involvement. Maybe there is a voxel or two that overlaps with the left insula and cingulate cortex but there is barely any evidence of this. Also, although I like the program Neurosynth, its data is quite messy because it just automatically scans studies based on a word . The algorithm will often make mistakes that can be overcome by a carefully conducted “manual”  neuroimaging meta-analysis. Therefore, I wouldn’t trust a Neurosynth meta-analysis more than a thoroughly conducted manual neuroimaging meta-analyses on the topic. Given that several manual neuroimaging meta-analyses on the topic of morality couldn’t find any evidence of insula and cingulate cortex involvement, I don’t think that this Neurosynth meta-analysis (that barely shows any evidence of insula or cingulate cortex activity) is now the crucial missing evidence that we have been waiting for.The second claim is that the striatum is involved in the morality network. Here the author uses three neuroimaging studies by his own research group. The three studies seem to be based on the same data of around 16 (sic) participants. I don’t doubt that the author is familiar with their own study and that striatum was activated but again, I don’t think this is the convincing evidence to finally claim that the striatum is consistently involved in morality. If the author wants to make the point about the striatum (or the insula and cingulate cortex), they should do a thorough look through the neuroimaging morality literature to argue their case.   The author makes two main claims: In short, I don’t think the comment adds much to the literature.Minor comments: 1) circutiries => circuitries 2) recently invented => Neurosynth has been around since 2011Are arguments sufficiently supported by evidence from the published literature or by new data and results?NoIs the conclusion balanced and justified on the basis of the presented arguments?NoIs the rationale for commenting on the previous publication clearly described?YesAre any opinions stated well-argued, clear and cogent?YesReviewer Expertise:fMRI, morality, meta-analysesI confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above. The current literature is a well written concise commentary. The title and the abstract precisely summarize the content of the article. The authors provided evidence supporting Raine’s model on the presence of the neural circuitry of antisociality. The concluding paragraph also clearly reflects the author's argument. This commentary would be useful to design or establish further research or hypothetical model regarding the insula, cingulate cortex, and striatum regions as parts of the neural network of morality or antisociality. (there are some typos (circutries) in the manuscript).Are arguments sufficiently supported by evidence from the published literature or by new data and results?YesIs the conclusion balanced and justified on the basis of the presented arguments?YesIs the rationale for commenting on the previous publication clearly described?YesAre any opinions stated well-argued, clear and cogent?YesReviewer Expertise:Psychopathology, Social neuroscience.I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Thank you very much for your kind comments. First of all, I corrected the typos in the manuscript that you mentioned. Second, following your suggestion and other reviewers' points, I added some points related to how my commentary can provide insights about how to develop further research questions and hypotheses.


Technology is a potentially powerful tool to assist patients with transitions of care during and after hospitalization. Patients with low health literacy who are predisposed to poor health outcomes are particularly poised to benefit from such interventions. However, this population may lack the ability to effectively engage with technology. Although prior research studied the role of health literacy in technology access/use among outpatients, hospitalized patient populations have not been investigated in this context. Further, with the rapid uptake of technology, access may no longer be pertinent, and differences in technological capabilities may drive the current digital divide. Thus, characterizing the digital literacy of hospitalized patients across health literacy levels is paramount.We sought to determine the relationship between health literacy level and technological access, use, and capability among hospitalized patients.P value for significance was <.003.Adult inpatients completed a technology survey that asked about technology access/use and online capabilities as part of an ongoing quality of care study. Participants’ health literacy level was assessed utilizing the 3-question Brief Health Literacy Screen. Descriptive statistics, bivariate chi-squared analyses, and multivariate logistic regression analyses  were performed. Using Bonferroni correction for the 18 tests, the threshold P=.007) and had used the Internet previously . Participants with low health literacy were more likely to report needing help performing online tasks . In the multivariate analysis, when adjusting for age, race, gender, and education level, we found that low health literacy was not significantly associated with a lower likelihood of owning smartphones  or using the internet ever . However, low health literacy remained significantly associated with a higher likelihood of needing help performing any online task .Among 502 enrolled participants, the mean age was 51 years, 71.3% (358/502) were African American, half  were female, and half  had at least some college education. Over one-third  of participants had low health literacy. The majority of participants owned devices (owned a smartphone: 116/173, 67.1% low health literacy versus 235/300, 78.3% adequate health literacy, The majority of participants with low health literacy had access to technological devices and had used the internet previously, but they were unable to perform online tasks without assistance. The barriers patients face in using online health information and other health information technology may be more related to online capabilities rather than to technology access. When designing and implementing technological tools for hospitalized patients, it is important to ensure that patients across digital literacy levels can both understand and use them. Technology-based interventions have the potential to improve care transitions; however, they can also exacerbate existing health disparities. For instance, hospitalization represents a time of vulnerable care transitions  when tecPrior research has shown that health literacy is an important contributor to the digital capability divide, with patients with lower health literacy being 7%-47% less likely to access and have the ability to use technology ,11. At tThe relationship between health literacy and technology use may be particularly relevant for hospitalized patients, since many care transition interventions rely on technology. Prior research has also suggested that health literacy is dynamic, with hospitalization representing a period when health literacy may acutely decrease . It is aWe performed a cross-sectional observational study among adult inpatients at the University of Chicago Medicine as part of a large, ongoing study of inpatient quality of care . InclusiWe used a survey that was comprised primarily of national benchmarked Pew Research Center survey questions  that werP<.003 defining statistical significance based on Bonferroni correction.Differences in technology access, use, and capabilities by health literacy level were analyzed using chi-squared tests. Multivariate logistic regression analyses were conducted to determine differences in technology access, use, and capabilities adjusting for health literacy (binary), age (continuous), gender (binary), race , and education (less than any college versus some college or more). All analyses were performed using STATA version 15.0 (StataCorp), with P<.001) or laptop . In contrast, there was no significant difference in ownership of tablets  or smartphones  by health literacy level. Participants with low health literacy were less likely to report using the internet ever , daily internet use , or searching for health information online . Participants with low health literacy were more likely to report needing help for all online tasks queried  were Black (non-Hispanic), 265 (52.8%) were female, and 253 (50.4%) had at least some college education. Out of 502 participants, 191 (38.0%) had low health literacy. Compared to participants with adequate health literacy, participants with low health literacy were less likely to own a desktop , but not with owning a smartphone (P=.50). Low health literacy level was not significantly associated with a lower likelihood of internet use (P=.02) or searching for health information online (P=.10). Low health literacy was associated with a higher likelihood of needing help with online tasks overall. Further results from multivariate logistic regression analyses are provided in In multivariate analysis, we found that health literacy remained significantly associated with some aspects of technology access/use, but not others . For exa use P=.0 or searcWe found that despite most participants having access to smartphones and using the internet, hospitalized patients with low health literacy were significantly more likely to need help with online tasks. This raises the question of whether inpatients with low health literacy are able to effectively utilize technology, even if they have access to do so. Although our study suggests that certain devices, such as smartphones, may have appealing interfaces for patients across literacy levels, it also underlines that access is not synonymous with ability. This is consistent with a recent study showing disparities in hospitalized patients’ interest in patient portals . PatientSome hospitals are measuring health literacy among their patients to identify those who may be at high risk for nonunderstanding and poorer health outcomes . In addiRecently, electronic health  literacy has gained traction as an important aspect of patients’ ability to obtain and use information from online sources. eHealth literacy represents a complex interplay between multiple literacies, including health literacy and digital literacy , which wLimitations of our study include being a single-site study and using self-reported measures of technology access, use, capabilities, and health literacy. Further, our study did not measure eHealth literacy directly. Future studies could focus specifically on the eHealth literacy of hospitalized patients and how it relates to their ability to use health technologies during and after hospitalization.In summary, our results indicated that health literacy is an important contributor to the digital capability divide among hospitalized patients and helped to identify areas of future research. Technology provides both opportunities for improvement during care transitions and potential pitfalls. To mitigate these pitfalls, health literacy and digital literacy should be accounted for when considering how best to implement technology-based interventions across hospital-to-home transitions of care.