JOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Review Health Literacy in Web-Based Health Information Environments: Systematic Review of Concepts, Definitions, and Operationalization for Measurement Anna-Maija Huhta1,2, MA; Noora Hirvonen1,2,3, PhD; Maija-Leena Huotari1,2, PhD 1Department of Information Studies, Faculty of Humanities, University of Oulu, Oulu, Finland 2Medical Research Center Oulu, Oulu University Hospital and University of Oulu, Oulu, Finland 3Department of Information Studies, Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Turku, Finland Corresponding Author: Anna-Maija Huhta, MA Department of Information Studies Faculty of Humanities University of Oulu PO Box 1000 Oulu, FI-90014 Finland Phone: 358 504285175 Email: anna-maija.huhta@oulu.fi Abstract Background: Health literacy research seems to lack a consensus on what aspects to include into literacyin the context of health and on how to operationalize these concepts for measurement purposes. In addition to health literacy, several other concepts, such as electronic health (eHealth) literacy and mental health literacy, have been developed across disciplines. This study examines how these different concepts are used when studying health-related competencies in Web contexts. Objective: This study systematically reviews health literacy concepts and definitions and their operationalization in studies focused on Web-based health information environments. Methods: A systematic literature search was conducted in April 2016 in 6 electronic databases with a limitation to articles in English published between January 2011 and April 2016. Altogether, 1289 unique records were identified and screened according to the predefined inclusion criteria: (1) original, peer-reviewed research articles written in English; (2) the topic of the article concerned literacy in the context of health; (3) informants of the study were lay people, not health professionals or students of the field; and (4) the focus of the study was placed on an Web-based information environment. In total, 180 full texts were screened, of which 68 were included in the review. The studies were analyzed with an emphasis on the used health literacy concepts and measures. Results: On the basis of the included studies, several concepts are in use when studying health-related literacy in Web environments, eHealth literacy and health literacy being the most common ones. The reviewed studies represent a variety of disciplines, but mostly medical sciences. Typically, quantitative research methods are used. On the basis of the definitions for health literacy, 3 thematic categories were identified: general and skill-based, multidimensional, and domain-specific health literacy. Most studies adopted a domain-specific concept, followed by the ones that used a general and skill-based concept. Multidimensional concepts occurred least frequently. The general health literacy concepts were usually operationalized with reading comprehension measures, the domain-specific concepts with self-efficacy measures, and multidimensional concepts with several types of measures. However, inconsistencies in operationalization were identified. Conclusions: The results show that in studies conducted in Web-based information environments, several different health literacy concepts are in use, and there is no clear consensus on the definitions for these concepts. Future studies should place emphasis on the conceptual development of health literacy in Web contexts to gain better results on operationalization for measurement. Researchers are encouraged to provide clear operational definitions for the concepts they use to ensure transparency in reporting. (J Med Internet Res 2018;20(12):e10273) doi: 10.2196/10273 http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 1 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al KEYWORDS health literacy; consumer health information; internet; review, systematic distinction between basic functional health literacy, Introduction communicative or interactive health literacy, and critical health literacy have been debated [14]. Mårtensson and Hensing [9] Background note that the research on health literacy is heterogeneous and The contemporary digital information environment challenges identify 2 perspectives: health literacy as a polarized our understanding of what it means to be literate. The fast and phenomenon focused on the extremes of high and low and health free flow of information on the internet offers multiple ways to literacy as a multidimensional concept that acknowledges the communicate, but it can also challenge with overload of broadness of skills in interaction with social and cultural information and loss of authority and identity [1]. Exercising contexts. These definitions emphasize the interactive and critical critical thinking and employing information and digital literacies skills needed to use information for making appropriate health are ways to reduce the effects of information overload [2]. These decisions [9]. They also consider multiple settings and recognize types of literacies usually refer to a diverse set of competencies, that there are both social and individual components to the skills, and strategies vital for acting in multimodal and concept [3]. transforming information environments. In the context of Web-based health information, these competencies are essential The internet and the new digital tools for seeking, as the amount of health information is rapidly increasing and communicating, and using information have become embedded the possibility to encounter misinformation is apparent. in the social actions of people since the 1990s. Moreover, the growing interest in consumer health and digital solutions to The concept of health literacy has been widely used to address tailor health information for electronic health (eHealth) purposes literacy competencies required in health settings. A recent has increased research and generated new conceptualizations definition [3] describes health literacy as a concept that for health literacy. The concept of eHealth literacy by Norman recognizes people’s different capacities to find, understand, and and Skinner [15] was one of the first attempts to capture the use health information as well as the different life experiences meaning of health literacy in the digital context. The definition that shape peoples’willingness and confidence to do these tasks. draws on Eng’s [16] definition of eHealth as “the use of According to the World Health Organization (WHO) [4], health emerging information and communication technology, especially literacy regards the environmental, political, and social factors the internet, to improve or enable health and health care.” that determine health, and it is gained through comprehensive However, Norman and Skinner [15] add to it by stating that health education at the individual and community levels. Both “[c]onsumer eHealth requires basic reading and writing skills, the concept of health literacy and the means to measure it have working knowledge of computers, a basic understanding of been under development for over three decades. Yet, the science, and an appreciation of the social context that mediates research on the phenomena seems to lack a consensus on what how online health information is produced, transmitted, and aspects to include into literacyin the context of health and on received.” how to operationalize it for measurement purposes [5-8]. The definition of eHealth literacy by Norman and Skinner [15] On the basis of earlier reviews, health literacy is typically has been criticized for not fully describing the competencies understood as individuals’ functional skills, such as reading essential in digital environments [17-19]. Gilstad [18] notes that comprehension and numeracy [9] that are assessed in clinical the concept lacks the notions of contextual and cultural literacy settings [5], and the research is conducted predominantly within and communicative expertise as central literacy competencies. medical sciences [10]. More recently, however, research on There are several new definitions proposed for the concept. For health literacy–associated issues has been conducted in several example, Griebel et al [19] recently proposed a definition of other disciplines and subconcepts and related concepts have eHealth literacy that encompasses aspects of interactivity, the emerged [11,12]. Although the definitions have unique elements, dynamic evolvement of literacy, changing information practices especially the most recent definitions for health literacy overlap of individuals, and the integration of technology aspects. The substantially [3]. The digital context that has changed the ways authors note that there are several models describing eHealth people communicate has been taken into account in the literacy but also that there is a lot of research that deals with definitions of the concept only recently and, thus, needs to be the themes related to eHealth literacy but uses other terms [19]. investigated further. Typically, health literacy is seen as an umbrella concept that The aim of this study was to increase understanding of the health covers other concepts such as eHealth literacy and mental literacy concepts that are used as well as their definitions and literacy. However, the hierarchy is not entirely clear. For operationalization in Web-based information environments. example, health information literacy, a concept used in The purpose was to provide a synthesis of their use in a sample information science, can be seen as a related rather than a of studies published between the years 2011 and 2016. subconcept to health literacy as it combines the concepts of health literacy and information literacy [20]. In this study, we From Health Literacy to Electronic Health Literacy do not focus on the hierarchical relationships of these concepts Contemporary discussion on health literacy reveals that there and use the phrase health literacy concepts to refer to all is no consensus on the definition of the concept [5,10,13,14]. health-related literacy concepts. For instance, the attributes included in the concept [10] and the http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 2 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Measuring Health Literacy Objectives The first health literacy assessment tools were designed to Earlier reviews have focused on: (1) the definitions and measure the functional health literacy of individuals in clinical measures of the concepts of health literacy [5,6,9,11,12,25,26], settings [21]. The basis of these measures is on the definitions eHealth literacy [28,29], and critical health literacy [13,14] and of health literacy that present individuals’ reading their (2) operationalization in a specific demographic group, comprehension and numeracy as central competencies when for example, adolescents [30-32] and older adults [33], or in a dealing with medical texts. Therefore, these measures have been specific context, for example, eHealth service use [34]. criticized for capturing only a narrow spectrum of the conception This systematic review contributes to these earlier reviews by of health literacy [5,11,22]. Another way to assess health literacy synthesizing health literacy research conducted in Web-based is to measure the level of health knowledge of individuals. information environmentsand in different disciplines. It differs Usually, these measures are content- and context-based from the earlier reviews as it reviews not only the definitions knowledge tests that have been developed in and for the use of of different health literacy concepts but also the measures used clinical settings [11]. The more recent measures for health to operationalize these concepts in empirical studies. By literacy consider individuals’ self-reported abilities or elaborating remarks made in previous literature about the self-efficacy as an indicator of health literacy. These measures conceptions of health literacy, the following objectives were usually aim to detect the self-perceived abilities of the individual set: to, for example, collect, communicate, and evaluate health information [23] or to rate the individuals’ability to understand 1. To categorize thematically the definitions of health literacy health-related material [24]. However, the risk of assessing and related concepts used in empirical studies focused on merely self-efficacy or behavior instead of health literacy is Web-based information environments. considered to be a major disadvantage of self-reported health 2. To examine the operationalization of the concepts within literacy measures [11]. these thematic categories. Altin et al [25] reviewed generic health literacy instruments and Methods categorized them by their measurement modes (print, oral, numeracy, and multimodal) and their measurement approaches Data Sources and Search Strategy (objective, subjective, mixed, and multidimensional construct). This systematic review follows the Preferred Reporting Items The review indicated that more than two-thirds of the generic for Systematic Reviews and Meta-Analyses (PRISMA) [35]. health literacy instruments were based on multidimensional The review is interpretive [36] and emphasizes the integration constructs of health literacy. Moreover, it was shown that there of studies across different disciplines to create a synthesis of is a trend toward mixing objective and subjective measurement the data. A search strategy was developed to identify articles approaches. In addition, a third of the reviewed instruments examining health literacy or related concepts in a Web-based were based on existing functional literacy screeners. O’Neill et information environment. Overall, 6 academic databases were al [26] reviewed self-administered health literacy instruments searched on April 14, 2016. The databases were Library and and discovered that the majority of the instruments measured Information Science Abstracts, Applied Social Sciences Index general health literacy, whereas one-third of them measured and Abstracts, Education Resources Information Center, US condition- or context-specific health literacy (see also [22]). National Library of Medicine premier bibliographic database Therefore, it was suggested that for the instruments to progress, (MEDLINE), Library and Information Science and Technology more research should be focused on the investigation and Abstracts, and the Cumulative Index to Nursing and Allied elaboration of the construct of health literacy itself [26,27] Health Literature. The search terms used covered 3 domains, A systematic review on eHealth literacy measures [28] found “web,” “health,” and “literacy,” including related terms. The that all the identified measures were based on self-report and search was limited to title and abstract and to peer-reviewed measured the self-efficacy of individuals. The authors identified articles published in English between years 2011 and 2016. This 3 concept-based eHealth literacy measurement tools and 5 time span was chosen to provide a sample of studies published dual-design tools that comprised individual measures of health during a period within which Web information seeking [37] literacy and digital literacy. The dual-design measurement tools and the use of social media [38] have increased considerably. did not intend to measure eHealth literacy but ended up doing This tight time frame enabled reviewing a manageable sample so by including the main components of the concept [28]. An of studies. A broader time frame would have required a narrower overview of the recent eHealth literacy research [29] indicates search strategy. The search strategy is reported in detail in that although international research has been conducted, the Multimedia Appendix 1. tools to measure eHealth literacy lack acknowledgment of In addition, 1 academic journal (Computers in Human Behavior) different personal backgrounds influencing the measured was searched manually as it was not indexed in the searched competencies, such as social and cultural factors. Griebel et al databases but showed potential to finding relevant articles. [19] criticize the eHealth literacy community for missing an Search from this journal was conducted by searching with the agreement on how to measure eHealth literacy. Accordingly, it phrase “health” AND “literac*” OR “knowledge” from article is stated that the new tools should consider the earlier research titles and abstracts and within the same time frame as the and create a well-founded theoretical basis to place eHealth database search. This search resulted in 4 relevant articles. In literacy into broader context [19]. http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 3 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al total, 1289 articles were identified through the literature search, After the study selection process, 68 articles were included in as presented in Figure 1. the review. The following data were extracted from these articles: Study Selection and Extraction of Data 1. Title The screening process of the articles was 2-phased. In the first 2. Authors phase, the duplicates were removed and the titles and abstracts 3. Publication title of the articles (n=1289) screened independently by the first 4. Year of publication author to identify eligible articles for full-text screening. A 10% 5. Research area or discipline (according to the first authors’ random sample was screened by the second author with an affiliation) interrater agreement rate of 93%. The articles chosen for the 6. Aim or objective of the study full-text screening had to fulfill the following inclusion criteria: 7. Method of data collection (1) original, peer-reviewed full-text article written in English; 8. Method of data analysis (2) the topic of the article concerned literacy in the context of 9. Health literacy concept used health; (3) informants of the study were lay people, not health 10. Definition of the concept professionals or students of the field; and (4) the focus of study 11. Measurement tool and its description. was health literacy in a Web-based information environment. In the second phase of the selection process, 180 full-text articles were screened, 112 of which were excluded. Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of the study selection process. LISA: Library and Information Science Abstracts; LISTA: Library and Information Science and Technology Abstracts; ASSIA: Applied Social Sciences Index and Abstracts; ERIC: Education Resources Information Center; CINAHL: Cumulative Index to Nursing and Allied Health Literature. http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 4 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al A detailed description of the study selection process is presented veterans (n=14), students (n=8), adults (n=8), and parents or in the PRISMA chart (see Figure 1). The characteristics of the caregivers (n=4). Other groups were participants with limited included studies can be found in Multimedia Appendix 2. health literacy or computer literacy (n=2), middle-aged men (n=1), library users (n=1), members of an online support group Results (n=1), and the general public (n=12). The sample sizes ranged from 20 to 4368. Characteristics of the Included Studies Categorization In total, 68 studies were included in the systematic review. The The content analysis focused on the health literacy concepts studies represent a variety of disciplines (based on the first along with their definitions and measures. On the basis of the author’s affiliation), including medicine (n=13), health education definitions of the health literacy concepts identified in the and promotion or health communication (n=8), nursing (n=6), included articles, the studies were grouped into 3 thematic health sciences or public health (n=5), health policy (n=2), categories: health literacy as (1) a general skill, (2) a nutrition science (n=2), pharmacy (n=2), gerontology (n=1), multidimensional concept, and (3) as a domain-specific concept. biomedical informatics (n=1), communication or advertising The categorization is drawn from the data, and it follows (n=9), psychology (n=8), information science and information remarks made on health literacy research in earlier literature studies (n=8), sociology or social work (n=2), and behavioral [9,25]. In Table 1, the identified definitions are presented in sciences (n=1). these categories. A total of 8 different health literacy concepts (Table 1) with 21 If several concepts were cited, the main concept of the included definitions (Multimedia Appendix 3) were identified. The most study was derived from the article title, or if it was not commonly used concepts were health literacy, which was mentioned, from the abstract. A detailed description of all referred to in 38 studies, and eHealth literacy, which was used identified concepts and their definitions is provided in in 37 studies. Other health-related literacy concepts that emerged Multimedia Appendix 3. were mental health literacy (n=3), oral health literacy (n=1), and bad health literacy (n=1). The concepts of health Health Literacy as a General Skill information literacy and everyday health information literacy The definitions that describe health literacy as personal skills were presented in 1 study. Refer to the study by Huhta et al [10] to utilize health information to gain better health were for a detailed description of the concepts and their definitions. categorized as general and skill-based constructs. A general The most common method for data collection was a health literacy concept was adopted as the main concept in 23 questionnaire survey, which was the only data collection method studies. These studies referred to the health literacy definitions in 58 studies. There were 2 studies where interviews or focus by Nutbeam [39], American Medical Association [40], Ratzan groups were the only methods used. In 8 studies, several data and Parker [41], Australian Bureau of Statistics [42], Rootman collection methods were used. The analysis methods were and Gordon-El-Bihbety [43], Berkman et al [44], The Patient predominantly quantitative (n=62). Mixed methods were applied Protection and Affordable Care Act [45], National Network of in 4 studies and qualitative methods in 2 studies. Libraries of Medicine [46], and the health information literacy definition by Shipman et al [20]. The included studies focused on different populations: patients or adults with risk factors for a disease (n=17), older adults or Table 1. Health literacy concepts identified in the included articles. Thematic category Concept Defined by Example of definition General and skill- Health literacy; Health Nutbeam [39], American Medical Associa- Health literacy is “the degree to which individuals have based information literacy tion [40], Ratzan and Parker [41], Australian the capacity to obtain, process, and understand basic Bureau of Statistics [42], Rootman and health information and services needed to make appro- Gordon-El-Bihbety [43], Berkman et al priate health decisions.” Ratzan and Parker [41] [44], The Patient Protection and Affordable Care Act [45], National Network of Li- braries of Medicine [46]; Shipman et al [20] Multidimensional Health literacy Nutbeam [47], Zarcadoolas et al [48], Baker “Health literacy is linked to literacy and entails people’s [49], Nutbeam [50], Sørensen et al [6] knowledge, motivation and competences to access, un- derstand, appraise, and apply health information in order to make judgments and take decisions in everyday life concerning healthcare, disease prevention and health promotion to maintain or improve quality of life during the life course.” Sørensen et al [6] Domain-specific eHealth literacy; Mental Norman and Skinner [15], Bodie and Dutta “eHealth literacy is defined as the ability to seek, find, health literacy; Oral [51], Norman [52]; Jorm et al [53]; US De- understand, and appraise health information from elec- health literacy; Bad partment of Health and Human Services tronic sources and apply the knowledge gained to ad- health literacy [54]; Schultz and Nakamoto [55] dressing or solving a health problem.” Norman and Skinner [15] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 5 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al The definition of health literacy as a capacity that individuals Rapid Estimate in Adult Literacy in Medicine (REALM) [91] have in certain degrees by Ratzan and Parker [41] was cited in used in 3 studies [56,58,63]. Other measurement tools used 24 studies [56-79]. Overall, it is the most often cited definition were The Test for Functional Health Literacy in Adults for health literacy in the included articles. Most of the articles (TOHFLA) [21] (cited in [59]) and its shorter version cited a secondary source for the definition, such as that by the S-TOFHLA [92] (cited in [84]), Short Assessment of Health Healthy People 2010 initiative of the US Department of Health Literacy in Dutch [93] (cited in [81]), and Adult Literacy & and Human Services [54]. The concept is process-oriented, Life Skills Survey [94] (cited in [67]). These tools were focusing on obtaining basic health information and health developed to detect limited health literacy among adult patients services to make health decisions. A rather similar definition, in clinical settings. but one with a wider scope including oral communication skills Self-efficacy measures of health literacy were used in 5 studies by Berkman et al [44] was cited in 5 studies [62,74,80-82]. This [66,68,70,83,96] that adopted a general health literacy concept. was the second most cited definition. Of these studies, 3 used a self-efficacy measure with only few The health literacy definition adopted by the WHO and outlined screening items. Kim [66] states that individuals with higher by Nutbeam [39], stressing on both cognitive and social skills levels of health literacy are expected to search health-related of an individual in the process of building motivation and information from the Web more efficiently, and thus, in the understanding health information, was cited in 4 studies [83-86]. study, health literacy was measured by asking whether the The health literacy definition by the American Medical respondents searched for health information from the Web. Lee Association [40] focused on individuals’skills to perform tasks et al [68] used a 1-item health literacy screener by Chew et al on reading comprehension and numeracy. It was cited in 2 [24], and Mayberry et al [70] used a modified 3-item version studies [87,88]. Other definitions for general health literacy of the screener. It consists of questions about reading problems were cited only once and were rather similar to each other with and confidence in filling out medical forms [24]. only minor differences. For example, the definition by Rootman Other self-efficacy measures used were a reading comprehension and Gordon-El-Bihbety [43] includes the attribute of evaluation screener called Single Item Literacy Screener [97] (cited in [71]) and presents health literacy as an ability that can be improved and the Functional Communicative and Critical Health Literacy across the life-course. scale (FCCHL) [23] (cited in [61]), which is based on Nutbeam’s The concept of health information literacy by the Medical [47] multidimensional definition of health literacy. FCCHL is Library Association [20] presents individuals’skills to recognize a self-efficacy measure containing questions about the frequency an information need, seek information, and use it as key of the patient’s actions, such as how often the patient had competencies needed to make good health choices [20]. It was problems to read and comprehend medical texts (functional cited in 1 study [89]. In this definition, the focus is placed on health literacy); how often they collect information, the process of information seeking, described in more detail communicate about medical conditions, and apply the found compared with the definitions for health literacy. The concept information (communicative health literacy); and how often of health information literacy addresses also the individuals’ they critically evaluate the found information (critical health ability to assess the found information critically and to evaluate literacy) [23]. Furnival et al [89] used the Everyday Health its applicability to a specific situation. This critical attribute is Information Literacy (EHIL) screening tool by Niemelä et al not present in all the definitions for health literacy and related [98] to measure the study participants’ health information concepts [10] and thus distinguishes the concept from other, literacy. The screening tool is based on the concept of health more functional health literacy definitions. information literacy and was developed for studying “laypersons’ general and nonprofessional abilities related to Common for these definitions of health literacy and health health information” [98]. information literacy is the focus on individuals’ abilities to obtain health information to make good health decisions. These In addition, 2 studies [65,80] measured health literacy with a definitions describe health literacy from 2 perspectives. First, knowledge test. Jiang and Beaudoin [65] referred to Ratzan’s health information is seen as general information obtained and Parker’s [41] definition of health literacy in their study and through information seeking. Second, health literacy is seen as operationally defined the concept as “one’s knowledge and a general skill set that an individual has to some degree and that understanding on health-related issues.” The test consists of it can be utilized universally in decision-making situations. self-reported knowledge about medical research (scientific Thus, health literacy is understood as a general, skill-based literacy), beliefs about US tobacco regulation (civic literacy), ability that can be applied to all kinds of situations that are and a numeracy section. The authors suggested that the used related to health. knowledge test aligns with the multidimensional model of health literacy developed by Zarcadoolas et al [48]. Lee et al [80] cited Operationalization of the General Health Literacy the health literacy definition by Berkman et al [44] and stated Concepts that health knowledge is seen as a subdimension or a proxy of In total, 11 studies in this category used 1 or several health literacy. In their study, health knowledge was measured measurement tools with an aim to detect the functional reading by asking respondents to indicate the plausibility of 7 health skills and numeracy of the selected population (see Table 2). statements [44]. Other types of measures identified were a The most often used functional measurement tools were the skill-based health literacy performance test [74] and qualitative Newest Vital Sign [90] used in 4 studies [57,62,73,87] and the assessment of health-related information literacy [86]. http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 6 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Table 2. Operationalization of health literacy concepts in selected studies (N=68). Type of measure Thematic category, n (%) General and skill-based (n=23) Multidimensional (n=6) Domain-specific (n=39) Reading comprehension and numeracy 11 (16.2) 0 (0) 1 (1.5) Self-efficacy 6 (8.8) 2 (2.9) 34 (50.0) Knowledge 2 (2.9) 0 (0) 3 (4.4) Performance tasks 1 (1.5) 0 (0) 0 (0) Qualitative assessment 1 (1.5) 0 (0) 0 (0) Several 2 (2.9) 4 (5.9) 1 (1.5) Moreover, 2 studies [64,99] used several types of measures to include, for example, awareness of possible biased authorities assess general health literacy. In both studies, health literacy is in consumer advertising [48]. defined as a skill-based construct, and it is assessed with reading The health literacy definition by Sørensen et al [6] was cited in comprehension and self-efficacy measures [64] or additionally 3 studies [75,88,101]. Sørensen et al [6] reviewed health literacy also with a knowledge test [99]. For example, in a study by research and created an integrated model with 6 dimensions of Woods et al [99], the study participants completed 11 different health literacy: (1) competence, skills, and abilities; (2) actions; questionnaires that measured health knowledge, health literacy, (3) information and resources; (4) objective; (5) context; and and internet and computer skills. In 1 study [86], a qualitative (6) time. The definition considers individual capabilities, but it assessment of health and information literacy was conducted. also aims to address the public health perspective [6]. Almost all the studies that adopted a general health literacy Baker’s [49] conceptual model of health literacy was cited in 2 concept screen participants’ internet use [56,57,59,61-68,70, studies [102,103]. It presents several domains that affect health 71,74,77,80,84,86,87], usually with a simple yes or no question. literacy. In the model, prior knowledge, such as vocabulary and In 4 studies [60,61,66,68] computer or internet literacy was conceptual knowledge of health together with reading fluency, measured, although in 2 of these, this means screening the is seen as a resource for an individual for facilitating health internet use of the participants. In fewer cases, measures also literacy. Health-related print and oral literacy are seen as included access to internet [59,63,87], skills [56,68,85,100] or dimensions of holistic health literacy that can lead to improved comfort [70] to use internet or a computer, and abilities to health outcomes. In addition, influencing factors, such as culture communicate with peer or health professionals and providers and norms, and barriers, such as limited access to health care, in the Web [57,68]. In 3 studies [58,73,81] internet, computer, can have an effect on health behavior change [49]. or technology-related measures were not included. Nutbeam [47] continued his examination on health literacy by Health Literacy as a Multidimensional Concept broadening the definition into a conceptual model. The model Models that include several attributes, such as the social factors consists of 3 literacy concepts: functional health literacy relates and cultural context into the definitions of health literacy, were to health education and learning of factual information on health categorized as multidimensional health literacy concepts. For risks and on how to use the health system. Interactive health example, the critical appraisal of found information is taken literacy concerns improving personal capacity to act into account more thoroughly in these models. These independently on knowledge. Critical health literacy regards multidimensional health literacy definitions and models by cognitive and skills development outcomes that support effective Nutbeam [47,50], Baker [49], Zarcadoolas et al [48], and social and political action. According to Nutbeam [47], the first Sørensen et al [6] were cited in 9 studies, the last 2 being the 2 literacy dimensions are effective on an individual level, but most used. In total, 6 studies chose the multidimensional the third can also be seen linked to population level benefits. construct as the main health literacy concept. The model is developed to address the challenges for health education, and therefore, it presents health literacy as an The health literacy definition by Zarcadoolas et al [48] was cited outcome of health promotion. In his more recent article, in 3 studies [72,83,88]. The definition includes the notion of Nutbeam [50] suggests that instead of conceptualizing health health literacy as a lifelong learning process and sets the literacy as a risk factor influencing clinical outcomes, it should outcome of acquiring health literacy skills as an improved be seen as an asset that can support individual and population quality of life. This definition presents health and health literacy level health outcomes, when improved through patient as the lifelong projects of people, not individuals. The model education. complementing the definition of health literacy by Zarcadoolas et al [48] is built around 4 central domains of literacy: Operationalization of the Multidimensional Health fundamental, scientific, civic, and cultural. Of these, especially Literacy Concepts the domain of civic literacy represents the sociocultural aspect In total, 6 studies [75,83,88,101,103,108] adopted a of literacy, as it includes “[u]nderstanding the relationship multidimensional health literacy concept as the central concept between one’s actions and the larger social group.” The civic of the study. The operationalization of these concepts varied, literacy domain also stresses critical media literacy skills that and several types of measures were used, as seen in Table 2. http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 7 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Rowsell et al [101] referred to the multidimensional health seems to be addressed as a contrast to nonelectronic information literacy definitions by Sørensen et al [6] and Nutbeam [50] and seeking, although a deeper explanation of those electronic evaluated the level of health literacy with a single-item sources is absent in the definition [15]. self-efficacy measure by Chew et al [24] with the aim to detect Bodie and Dutta [51] present an elaborated definition for eHealth patients’difficulties in understanding written information. On literacy that stresses the significance of the Web context in the other hand, van der Vaart et al [103] adopted Baker’s [49] seeking, evaluating, and using health information. This health literacy definition as their main literacy concept and definition was presented in 1 study [114]. Norman’s [52] measured it with the FCCHL self-efficacy scale that includes definition for eHealth literacy 2.0 was presented in 1 study several literacy domains. [126]. With the definition, Norman attempts to emphasize the In 4 studies, several types of measures were used. In a study by context of social media regarding eHealth literacy screening Tam et al [75], the combination of measures included a reading tool development by presenting social media relevant tasks and comprehension and numeracy measure the Rapid Estimate of skills to the concept [52]. Adult Literacy in Medicine and Dentistry measure Other domain-specific health literacy concepts identified in the (REALMD-20) [107], a 2-item self-efficacy measure by Chew studies were mental health literacy used in 3 studies [127-129], et al [24], and a dental health knowledge test. In this study, oral oral health literacy used in 1 study [130], and badhealth literacy health literacy was measured, although the authors did not used in 1 study [31]. The definition of mental health literacy by provide a clear definition of the concept itself. Instead, the health Jorm et al [53], unlike other health literacy definitions, also literacy definition by Sorensen et al [6] and the concept of addresses beliefs and attitudes toward health issues. The eHealth literacy [15] were discussed. In other studies that definition of oral health literacy by the US Department of Health adopted a multidimensional health literacy concept, reading and Human Services [54] is based on the health literacy comprehension and numeracy [88], self-efficacy [83,88,108], definition by Ratzan and Parker [41] and thus takes a skill-based knowledge [88,108], and performance [83] were measured. approach to the concept. The concept of bad health literacy Computer or internet literacy was not measured in studies that originally introduced by Schulz and Nakamoto [55] refers, adopted a multidimensional concept of health literacy. Instead, according to Allam et al [131], to “the presence of the ability internet access [103,108] and use [103] were screened. to understand medical information turned sour by the Subramania et al [88] included internet-related questions to simultaneous absence of the ability to recognize it as false.” In their overall assessment of health literacy skills of the other words, the information seeker might be literate enough to participants. Moreover, 3 studies [75,83,101] did not include find, understand, and process even low-quality information, any kinds of internet- or computer-related measures to their obtained, for example, from electronic sources but is incapable study. to recognize it as false, irrelevant, or fraudulent [131]. Health Literacy as a Domain-Specific Concept Operationalization of the Domain-Specific Health The health literacy concepts that focus on a specific context or Literacy Concepts target a specific patient group are categorized as domain-specific Within the studies that adopted a domain-specific concept as concepts of health literacy. In total, a domain-specific concept the main health literacy concept (n=39), the operationalization of health literacy was cited in 41 of the included studies. Of is more often done with a self-efficacy measurement tool than these, eHealth literacy by Norman and Skinner [15], Bodie and other types of measures, as seen in Table 2. Dutta’s [51] elaboration of the same concept, and Norman’s Most of the studies that adopted eHealth literacy as the main [52] suggestion of eHealth literacy 2.0 definition are essentially concept used the eHealth Literacy Scale (eHEALS) by Norman targeted to address health literacy in Web contexts. Of these, and Skinner [132] as the main measurement tool. In total, the Norman’s and Skinner’s definition was the most often cited eHEALS is used in 29 of the 39 studies in this category and as definition in included studies. In several studies (n=11), in the only used tool in 25 of them [60,72,76,77,79,82,85, addition to eHealth literacy, also other health literacy concepts 95,96,100,102,104-106,113,115-123,125]. The 8-item eHEALS and definitions were discussed (see Multimedia Appendix 2). scale aims to measure “consumers’ combined knowledge, In total, 39 studies adopted a domain-specific health literacy comfort, and perceived skills at finding, evaluating, and applying concept as the main concept of the study. In most of these electronic health information to health problems.” The scale is studies (n=34), the main concept was eHealth literacy proposed to address the 6 literacy types of the Lily model [15]. [60,69,72,76-79,82,85,95,96,100-102,104-106,109-126]. The In the included studies, the eHEALS is described in different concept of eHealth literacy is accompanied by the Lily model ways. Typically, the scale is described as a measurement that that consists of 6 literacies organized in 2 central types: analytic detects consumers’ perceived information technology or (traditional, media, and information) and context-specific computer skills. In addition, the abilities to seek health (computer, scientific, and health). The analytic literacy types information from the Web are seen as central attributes of the are described as skills that are applicable to a wide range of scale. Other studies that adopted the eHealth literacy as the main information sources [15]. The context-specific types involve concept of the study also used other self-efficacy measures, skills that are applied in specific situations. According to such as EHIL [98] (cited in [110]) and Brief Health Literacy Norman and Skinner [15], all these skills are required when Screening Tool BRIEF [133] (cited in [112]). In addition, 2 engaging with electronic sources. In the definition of eHealth studies [69,114] present a new eHealth literacy measure. Hsu literacy, the electronic element of health information seeking et al [114] discuss eHealth literacy definitions by Norman and http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 8 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Skinner [15] and Bodie and Dutta [51] and present a new there is a tendency to refer to the early definitions of health eHealth literacy measure eHL that seeks to detect individuals’ literacy, which present a functional understanding of the “ability to seek, find, understand, and evaluate health concept. Within the studies that applied the concept of eHealth information from electronic sources and apply this knowledge literacy, a more consistent understanding of the definition was to address or solve a health problem” [114]. The self-efficacy detected as only 2 definitions for the concept were presented. measure eSEARCH, eHealth Literacy Tool used in a study by As earlier reviews indicate, the currently used measures of health Manafò et al [69], was developed to measure eHealth literacy literacy have focused on assessing individuals’ reading skills of older adults. comprehension and understanding of medical texts in clinical Other types of measures used in the included articles that contexts [5,135]. In addition, within the studies conducted in adopted eHealth literacy as the main concept were performance Web-based information environments, general health literacy tests [109,126]; combined measures of reading comprehension, was measured with a widely used and validated functional numeracy, and knowledge [124]; and self-efficacy [78]. In 1 measurement tool, although there are more recent and study [111], eHealth literacy was assessed qualitatively based multidimensional measures available [25]. Pleasant et al [135] on focus group discussions of the participants. argue that the focus on measuring only the functional skills of individuals leaves important factors such as individual Mental health literacy was measured in 3 studies [127-129] and information and communication skills untested. Despite the oral health literacy [130] and badhealth literacy [131] both in trend of understanding health literacy as a multidimensional 1 study. In 2 of the studies that focused on mental health literacy construct including contextual, cultural, and social factors [5], [128,129], the concept was operationalized by measuring the these were not acknowledged in the studies included in this participants’ knowledge about and attitudes toward mental systematic review. health issues. Li et al [127] used several types of measures. The 31-item questionnaire consists of questions about the The concept of eHealth literacy by Norman and Skinner [15] participant’s knowledge and self-efficacy on mental health was clearly the most used concept in the included studies. As issues. In a study by Tse et al [130], oral health literacy was a domain-specific concept, eHealth literacy aims to address measured with REALD-30 [134], a word recognition instrument especially the literacy skills needed in Web contexts. However, that requires participants to read aloud 30 oral health–related in the included studies, the concept was described as the words. Allam et al [131] measured bad health literacy with a technological skills of the study subjects. Yet, it is clear that knowledge test focused on vaccine information. eHealth literacy competencies are more varied than the mere ability to use the internet or a computer efficiently. Addressing Discussion literacy skills or practices through domain-specific concepts offers an opportunity to express domain-specific issues, such Principal Findings as the importance of the technological skills as part of eHealth The aim of this systematic review was to identify health literacy literacy competencies, or oral health knowledge as part of oral concepts and their definitions and operationalization in studies health literacy. However, the development of these concepts focused on Web-based information environments. The concept may be challenging, as the focus of research is fragmented in of eHealth literacy by Norman and Skinner [15] was used most empirical studies and the conceptual development is scarce (See often. However, the concept of health literacy was also used also [8]). and a variety of definitions were presented for it in the selected Measurement of eHealth literacy is more often focused on studies. On the basis of the definitions for health literacy, 3 assessing the self-reported skills of individuals. Unlike in the thematic categories were identified, namely, general and systematic review by Karnoe and Kayser [28], dual-design skill-based, multidimensional, and domain-specific. Most studies eHealth literacy measures are not common in studies conducted adopted a domain-specific concept, followed by the ones that in Web-based information environments, as only few studies used a general and skill-based concept. Multidimensional included internet or digital literacy measures in their health concepts occurred least frequently. literacy screening tools. The general concept of health literacy was typically The trend toward mixing different measuring types, as indicated operationalized by using reading comprehension and numeracy by Altin et al [25], was noted also within the studies conducted measures. In turn, the domain-specific concepts were most often in Web-based information environments. The focus on clinical operationalized by using a self-efficacy measure. Several types settings as a study context was not as clearly indicated as in the of measures were used in studies that adopted multidimensional earlier reviews, and usually, the sample population was a certain constructs of health literacy. Nevertheless, inconsistencies in age instead of patients. the operationalization of the different concepts were identified. Strengths and Limitations Comparison With Prior Work To our knowledge, this study is among the first The lack of consensus in defining health literacy, as presented cross-disciplinary reviews of health literacy concepts, in several reviews [6,11,30], is supported by the results of this definitions, and their operationalization in Web contexts. The systematic review as several different definitions for the concept systematic process of this review enabled thorough investigation were identified in the included studies. The modern health of the health literacy–related academic research focused on the literacy definitions are more often multidimensional than context of Web-based information environments. The main functional [3,9]. However, this systematic review shows that http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 9 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al limitations of this review lie within the search strategy. Only thematic categories of the different definitions were identified: studies written in English were included in the review, which general and skill-based, multidimensional, and domain-specific. excluded relevant studies in other languages. In addition, some Typically, general and skill-based health literacy was measured studies may have been missed due to the restricted search terms with reading comprehension or numeracy tests and and limited time frame. domain-specific health literacy with self-efficacy tests. Multidimensional concepts were used less often and Conclusions operationalized by using several types of measures. Future This systematic review identified health literacy concepts, studies conducted in Web contexts should place emphasis on definitions, and operationalization used in research focusing on the conceptual development of health literacy. Researchers are Web-based information environments. On the basis of the encouraged to provide clear operationalization for the concepts results, several concepts are being used, eHealth literacy and they use to ensure transparency in reporting. health literacy being the most common ones. In addition, 3 Acknowledgments This study is part of the Academy of Finland project Cognitive Authorities in Everyday Health Information Environments of Young People (#2430227313) and funded by the Eudaimonia Doctoral Program in the University of Oulu, Finland. Conflicts of Interest None declared. Multimedia Appendix 1 Search strategy. [PDF File (Adobe PDF File), 197KB-Multimedia Appendix 1] Multimedia Appendix 2 Characteristics of the included studies. [PDF File (Adobe PDF File), 447KB-Multimedia Appendix 2] Multimedia Appendix 3 Health literacy concepts and definitions in the included studies. [PDF File (Adobe PDF File), 384KB-Multimedia Appendix 3] References 1. Bawden D, Robinson L. The dark side of information: overload, anxiety and other paradoxes and pathologies. J Inf Sci 2009 Apr;35(2):180-191. [doi: 10.1177/0165551508095781] 2. Koltay T. The bright side of information: ways of mitigating information overload. J Doc 2017 Jul 10;73(4):767-775. [doi: 10.1108/JD-09-2016-0107] 3. Batterham RW, Beauchamp A, Osborne RH. Health Literacy. In: Quah SR, editor. International Encyclopedia Of Public Health, Second Edition. Kidlington, Oxford: Elsevier; 2017:428-437. 4. World Health Organization. Track 2: Health literacy and health behavior URL: http://www.who.int/healthpromotion/ conferences/7gchp/track2/en/[accessed 2018-02-06] [WebCite Cache ID 6x1poUDyA] 5. Guzys D, Kenny A, Dickson-Swift V, Threlkeld G. A critical review of population health literacy assessment. BMC Public Health 2015 Mar 04;15:215 [FREE Full text] [doi: 10.1186/s12889-015-1551-6] [Medline: 25885742] 6. Sørensen K, Van den Brouck S, Fullam J, Doyle G, Pelikan J, Slonska Z, (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012;12:80 [FREE Full text] [doi: 10.1186/1471-2458-12-80] [Medline: 22276600] 7. McCormack L, Haun J, Sørensen K, Valerio M. Recommendations for advancing health literacy measurement. J Health Commun 2013;18(Suppl 1):9-14 [FREE Full text] [doi: 10.1080/10810730.2013.829892] [Medline: 24093340] 8. Mackert M, Champlin S, Su Z, Guadagno M. The many health literacies: advancing research or fragmentation? Health Commun 2015;30(12):1161-1165. [doi: 10.1080/10410236.2015.1037422] [Medline: 26372026] 9. Mårtensson L, Hensing G. Health literacy – a heterogeneous phenomenon: a literature review. Scand J Caring Sci 2012 Mar;26(1):151-160. [doi: 10.1111/j.1471-6712.2011.00900.x] [Medline: 21627673] 10. Huhta AM, Hirvonen N, Huotari ML. Concepts related to health literacy in online information environments: a systematic review with an emphasis on approach to information. In: Information Literacy in the Workplace. Cham: Springer; 2018 http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 10 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al Presented at: 5th European Conference on Information Literacy; Sep 18-21, 2017; Saint Malo, France p. 460-469. [doi: 10.1007/978-3-319-74334-9_48] 11. Frisch A, Camerini L, Diviani N, Schulz PJ. Defining and measuring health literacy: how can we profit from other literacy domains? Health Promot Int 2012 Mar;27(1):117-126. [doi: 10.1093/heapro/dar043] [Medline: 21724626] 12. Mancuso JM. Assessment and measurement of health literacy: an integrative review of the literature. Nurs Health Sci 2009 Mar;11(1):77-89. [doi: 10.1111/j.1442-2018.2008.00408.x] [Medline: 19298313] 13. Chinn D. Critical health literacy: a review and critical analysis. Soc Sci Med 2011 Jul;73(1):60-67. [doi: 10.1016/j.socscimed.2011.04.004] [Medline: 21640456] 14. Sykes S, Wills J, Rowlands G, Popple K. Understanding critical health literacy: a concept analysis. BMC Public Health 2013 Feb 18;13:150 [FREE Full text] [doi: 10.1186/1471-2458-13-150] [Medline: 23419015] 15. Norman CD, Skinner HA. eHealth literacy: Essential skills for consumer health in a networked world. J Med Internet Res 2006 Jun;8(2):e9 [FREE Full text] [doi: 10.2196/jmir.8.2.e9] [Medline: 16867972] 16. Eng TR. The e-Health Landscape: A Terrain Map of Emerging Information and Communication Technologies in Health and Health Care. Princeton, NJ: The Robert Wood Johnson Foundation; 2001. 17. Bautista J. From solving a health problem to achieving quality of life: redefining eHealth literacy. J Lit and Technol 2015;16(2):33-54 [FREE Full text] 18. Gilstad H. Toward a comprehensive model of eHealth literacy. In: Proceedings of the 2nd European Workshop on Practical Aspects of Health Informatics (PAHI 2014).: CEUR Workshop Proceedings; 2014 Presented at: 2nd European Workshop on Practical Aspects of Health Informatics; May 19, 2014; Trondheim, Norway. 19. Griebel L, Enwald H, Gilstad H, Pohl A, Moreland J, Sedlmayr M. eHealth literacy research-Quo vadis? Inform Health Soc Care 2018 Dec;43(4):427-442. [doi: 10.1080/17538157.2017.1364247] [Medline: 29045164] 20. Shipman JP, Kurtz-Rossi S, Funk CJ. The health information literacy research project. J Med Libr Assoc 2009 Oct;97(4):293-301 [FREE Full text] [doi: 10.3163/1536-5050.97.4.014] [Medline: 19851494] 21. Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med 1995 Oct;10(10):537-541. [Medline: 8576769] 22. Haun JN, Valerio MA, McCormack LA, Sørensen K, Paasche-Orlow MK. Health literacy measurement: an inventory and descriptive summary of 51 instruments. J Health Commun 2014;19(Suppl 2):302-333. [doi: 10.1080/10810730.2014.936571] [Medline: 25315600] 23. Ishikawa H, Takeuchi T, Yano E. Measuring functional, communicative, and critical health literacy among diabetic patients. Diabetes Care 2008 May;31(5):874-879. [doi: 10.2337/dc07-1932] [Medline: 18299446] 24. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004 Sep;36(8):588-594 [FREE Full text] [Medline: 15343421] 25. Altin SV, Finke I, Kautz-Freimuth S, Stock S. The evolution of health literacy assessment tools: a systematic review. BMC Public Health 2014;14:1207 [FREE Full text] [doi: 10.1186/1471-2458-14-1207] [Medline: 25418011] 26. Gonçalves D, Ricci-Cabello I, Ziebland S, Valderas J. An overview of self-administered health literacy instruments. PLoS One 2014;9(12):e109110 [FREE Full text] [doi: 10.1371/journal.pone.0109110] [Medline: 25478813] 27. Nguyen TH, Paasche-Orlow MK, McCormack LA. The state of the science of health literacy measurement. Stud Health Technol Inform 2017;240:17-33. [Medline: 28972507] 28. Karnoe A, Kayser L. How is eHealth literacy measured and what do the measurements tell us? A systematic review. Knowl Manag E-Learn Int J 2015 Dec 26;7(4):576-600 [FREE Full text] 29. Pohl A, Griebel L, Trill R. Contemporary eHealth literacy research. An overview with focus on Germany. In: Proceedings of the 3rd European Workshop on Practical Aspects of Health Informatics (PAHI 2015).: CEUR Workshop Proceedings; 2016 Presented at: 3rd European Workshop on Practical Aspects of Health Informatics; Oct 27; Elgin, Scotland, UK. 30. Ormshaw MJ, Paakkari LT, Kannas LK. Measuring child and adolescent health literacy: a systematic review of literature. Health Education 2013 Aug 23;113(5):433-455. [doi: 10.1108/HE-07-2012-0039] 31. Bröder J, Okan O, Bauer U, Bruland D, Schlupp S, Bollweg TM, et al. Health literacy in childhood and youth: a systematic review of definitions and models. BMC Public Health 2017 Dec 26;17(1):361 [FREE Full text] [doi: 10.1186/s12889-017-4267-y] [Medline: 28441934] 32. Okan O, Lopes E, Bollweg TM, Bröder J, Messer M, Bruland D, et al. Generic health literacy measurement instruments for children and adolescents: a systematic review of the literature. BMC Public Health 2018 Dec 22;18(1):166 [FREE Full text] [doi: 10.1186/s12889-018-5054-0] [Medline: 29357867] 33. Watkins I, Xie B. eHealth literacy interventions for older adults: a systematic review of the literature. J Med Internet Res 2014;16(11):e225 [FREE Full text] [doi: 10.2196/jmir.3318] [Medline: 25386719] 34. Kim H, Xie B. Health literacy in the eHealth era: a systematic review of the literature. Patient Educ Couns 2017 Jun;100(6):1073-1082. [doi: 10.1016/j.pec.2017.01.015] [Medline: 28174067] 35. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009 Oct;62(10):1006-1012. [doi: 10.1016/j.jclinepi.2009.06.005] [Medline: 19631508] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 11 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al 36. Harden A, Thomas J. Mixed methods systematic reviews: examples and emerging issues. In: Tashakkori A, Teddlie C, editors. SAGE handbook of mixed methods in social & behavioral research. 2nd ed. Thousand Oaks, California: SAGE; 2010:749-774. 37. Fox S, Duggan M. Pew Research Center. 2013. Health Online 2013 URL: http://www.pewinternet.org/2013/01/15/ health-online-2013/[accessed 2018-08-28] [WebCite Cache ID 720KD06v1] 38. Pew Research Center. Social Media Fact Sheet 2018 URL: http://www.pewinternet.org/fact-sheet/social-media/[accessed 2018-08-28] [WebCite Cache ID 720MwFtrn] 39. Nutbeam D. Health promotion glossary. Health Promot Int 1998 Jan 01;13(4):349-364. [doi: 10.1093/heapro/13.4.349] 40. Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. J Am Med Assoc 1999 Feb 10;281(6):552-557. [Medline: 10022112] 41. Ratzan S, Parker R. National Library of Medicine Current Bibliographies in Medicine: Health Literacy. Washington, DC: National Institutes of Health, US Department of Health and Human Services; 2000. 42. Australian Bureau of Statistics. 4233.0 - Health Literacy, Australia, 2006. Canberra: Australian Bureau of Statistics; 2006. 43. Rootman I, Gordon-El-Bihbety D. A vision for health literate Canada: report of the expert panel on health literacy. Ottawa, Ontario, Canada: Canadian Public Health Association; 2006. 44. Berkman ND, Davis TC, McCormack L. Health literacy: what is it? J Health Commun 2010;15(Suppl 2):9-19. [doi: 10.1080/10810730.2010.499985] [Medline: 20845189] 45. The Patient Protection Affordable Care Act (PPACA). Congress. 2010. Public Law 111–148 URL: https://www.congress.gov/ 111/plaws/publ148/PLAW-111publ148.pdf[accessed 2018-11-30] [WebCite Cache ID 74JrZuk38] 46. National Network of Libraries of Medicine. 2018. Health Literacy URL: https://nnlm.gov/initiatives/topics/health-literacy [accessed 2018-10-01] [WebCite Cache ID 72qMDhSsf] 47. Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int 2000;15(3):259-267. [doi: 10.1093/heapro/15.3.259] 48. Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Promot Int 2005 Jun;20(2):195-203 [FREE Full text] [doi: 10.1093/heapro/dah609] [Medline: 15788526] 49. Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006 Aug;21(8):878-883 [FREE Full text] [doi: 10.1111/j.1525-1497.2006.00540.x] [Medline: 16881951] 50. Nutbeam D. The evolving concept of health literacy. Soc Sci Med 2008 Dec;67(12):2072-2078. [doi: 10.1016/j.socscimed.2008.09.050] [Medline: 18952344] 51. Bodie GD, Dutta MJ. Understanding health literacy for strategic health marketing: eHealth literacy, health disparities, and the digital divide. Health Mark Q 2008 Jan;25(1-2):175-203. [doi: 10.1080/07359680802126301] [Medline: 18935884] 52. Norman C. eHealth literacy 2.0: problems and opportunities with an evolving concept. J Med Internet Res 2011 Dec;13(4):e125 [FREE Full text] [doi: 10.2196/jmir.2035] [Medline: 22193243] 53. Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B, Pollitt P. “Mental health literacy”: a survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med J Aust 1997 Feb 17;166(4):182-186. [Medline: 9066546] 54. US Department of Health Human Services. Healthy People 2010: Understanding and Improving Health, 2nd ed. Washington, DC: US Government Printing Office; 2010:21-39. 55. Schulz P, Nakamoto K. “Bad” literacy, the internet, and the limits of patient empowerment. In: AAAI Spring Symposium Series: Artificial Intelligence & Health Communication. 2011 Presented at: AAAI Spring Symposium Series: Artificial Intelligence & Health Communication; Mar 21-23, 2011; Palo Alto, CA. 56. Agree EM, King AC, Castro CM, Wiley A, Borzekowski DL. “It's got to be on this page”: age and cognitive style in a study of online health information seeking. J Med Internet Res 2015;17(3):e79 [FREE Full text] [doi: 10.2196/jmir.3352] [Medline: 25831483] 57. Bailey SC, O'Conor R, Bojarski EA, Mullen R, Patzer RE, Vicencio D, et al. Literacy disparities in patient access and health-related use of Internet and mobile technologies. Health Expect 2015 Dec;18(6):3079-3087 [FREE Full text] [doi: 10.1111/hex.12294] [Medline: 25363660] 58. Bickmore TW, Utami D, Matsuyama R, Paasche-Orlow MK. Improving access to online health information with conversational agents: a randomized controlled experiment. J Med Internet Res 2016;18(1):e1 [FREE Full text] [doi: 10.2196/jmir.5239] [Medline: 26728964] 59. Chisolm DJ, Hardin DS, McCoy KS, Johnson LD, McAlearney AS, Gardner W. Health literacy and willingness to use online health information by teens with asthma and diabetes. Telemed J E Health 2011 Nov;17(9):676-682 [FREE Full text] [doi: 10.1089/tmj.2011.0037] [Medline: 21943161] 60. Chung S, Nahm E. Testing reliability and validity of the eHealth Literacy Scale (eHEALS) for older adults recruited online. Comput Inform Nurs 2015 Apr;33(4):150-156. [doi: 10.1097/CIN.0000000000000146] [Medline: 25783223] 61. Cnossen IC, van Uden-Kraan CF, Eerenstein SE, Jansen F, Witte BI, Lacko M, et al. An online self-care education program to support patients after total laryngectomy: feasibility and satisfaction. Support Care Cancer 2016 Mar;24(3):1261-1268 [FREE Full text] [doi: 10.1007/s00520-015-2896-1] [Medline: 26306518] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 12 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al 62. Crook B, Stephens KK, Pastorek AE, Mackert M, Donovan EE. Sharing health information and influencing behavioral intentions: The role of health literacy, information overload, and the Internet in the diffusion of healthy heart information. Health Commun 2016;31(1):60-71. [doi: 10.1080/10410236.2014.936336] [Medline: 25668744] 63. Fagnano M, Halterman JS, Conn KM, Shone LP. Health literacy and sources of health information for caregivers of urban children with asthma. Clin Pediatr (Phila) 2012 Mar;51(3):267-273 [FREE Full text] [doi: 10.1177/0009922811421413] [Medline: 21911409] 64. Ghaddar SF, Valerio MA, Garcia CM, Hansen L. Adolescent health literacy: the importance of credible sources for online health information. J Sch Health 2012 Jan;82(1):28-36. [doi: 10.1111/j.1746-1561.2011.00664.x] [Medline: 22142172] 65. Jiang S, Beaudoin CE. Health literacy and the internet: an exploratory study on the 2013 HINTS survey. Comput Human Behav 2016;58:240-248. [doi: 10.1016/j.chb.2016.01.007] 66. Kim Y. Does online searching cause or enforce health information disparity? J Info Know Mgmt 2013 Dec;12(04):1350032. [doi: 10.1142/S0219649213500329] 67. Kobayashi LC, Wardle J, von Wagner C. Internet use, social engagement and health literacy decline during ageing in a longitudinal cohort of older English adults. J Epidemiol Community Health 2015 Mar;69(3):278-283 [FREE Full text] [doi: 10.1136/jech-2014-204733] [Medline: 25428933] 68. Lee YJ, Boden-Albala B, Jia H, Wilcox A, Bakken S. The association between online health information-seeking behaviors and health behaviors among hispanics in New York city: a community-based cross-sectional study. J Med Internet Res 2015;17(11):e261 [FREE Full text] [doi: 10.2196/jmir.4368] [Medline: 26611438] 69. Manafò E, Wong S. eSEARCH©: a tool to promote the eHealth literacy skills of older adults. J Consum Health Internet 2013 Jul;17(3):255-271. [doi: 10.1080/15398285.2013.812914] 70. Mayberry LS, Kripalani S, Rothman RL, Osborn CY. Bridging the digital divide in diabetes: family support and implications for health literacy. Diabetes Technol Ther 2011 Oct;13(10):1005-1012 [FREE Full text] [doi: 10.1089/dia.2011.0055] [Medline: 21718098] 71. McCleary-Jones V, Scheideman-Miller C, Rev DJ, Johnson B, Overall M, Dwyer K. Health information technology use and health literacy among community-dwelling African Americans. ABNF J 2013;24(1):10-16. [Medline: 23589966] 72. Paek H, Hove T. Social cognitive factors and perceived social influences that improve adolescent eHealth literacy. Health Commun 2012;27(8):727-737. [doi: 10.1080/10410236.2011.616627] [Medline: 22452551] 73. Smith SG, O'Conor R, Aitken W, Curtis LM, Wolf MS, Goel MS. Disparities in registration and use of an online patient portal among older adults: findings from the LitCog cohort. J Am Med Inform Assoc 2015 Jul;22(4):888-895 [FREE Full text] [doi: 10.1093/jamia/ocv025] [Medline: 25914099] 74. Sun X, Chen J, Shi Y, Zeng Q, Wei N, Xie R, et al. Measuring health literacy regarding infectious respiratory diseases: a new skills-based instrument. PLoS One 2013;8(5):e64153 [FREE Full text] [doi: 10.1371/journal.pone.0064153] [Medline: 23724029] 75. Tam A, Yue O, Atchison KA, Richards JK, Holtzman JS. The association of patients' oral health literacy and dental school communication tools: a pilot study. J Dent Educ 2015 May;79(5):530-538 [FREE Full text] [Medline: 25941146] 76. Tennant B, Stellefson M, Dodd V, Chaney B, Chaney D, Paige S, et al. eHealth literacy and Web 2.0 health information seeking behaviors among baby boomers and older adults. J Med Internet Res 2015;17(3):e70 [FREE Full text] [doi: 10.2196/jmir.3992] [Medline: 25783036] 77. Xie B. Older adults, e-health literacy, and collaborative learning: an experimental study. J Am Soc Inf Sci 2011 Mar 14;62(5):933-946. [doi: 10.1002/asi.21507] 78. Xie B. Improving older adults' e-health literacy through computer training using NIH online resources. Libr Inf Sci Res 2012 Jan 01;34(1):63-71 [FREE Full text] [doi: 10.1016/j.lisr.2011.07.006] [Medline: 22639488] 79. Xie B. Effects of an eHealth literacy intervention for older adults. J Med Internet Res 2011 Nov;13(4):e90 [FREE Full text] [doi: 10.2196/jmir.1880] [Medline: 22052161] 80. Lee C, Niederdeppe J, Freres D. Socioeconomic disparities in fatalistic beliefs about cancer prevention and the Internet. J Commun 2012 Dec;62(6):972-990 [FREE Full text] [doi: 10.1111/j.1460-2466.2012.01683.x] [Medline: 25530627] 81. Meppelink CS, Bol N. Exploring the role of health literacy on attention to and recall of text-illustrated health information: an eye-tracking study. Comput Human Behav 2015 Jul;48:87-93. [doi: 10.1016/j.chb.2015.01.027] 82. Wenhong C, Kye-Hyoung L. More than search? Informational and participatory eHealth behaviors. Comput Human Behav 2014 Jan;30:103-109. [doi: 10.1016/j.chb.2013.07.028] 83. Austvoll-Dahlgren A, Bjørndal A, Odgaard-Jensen J, Helseth S. Evaluation of a web portal for improving public access to evidence-based health information and health literacy skills: a pragmatic trial. PLoS One 2012;7(5):e37715 [FREE Full text] [doi: 10.1371/journal.pone.0037715] [Medline: 22701531] 84. Echt KV, Burridge AB. Predictors of reported Internet use in older adults with high and low health literacy: the role of socio-demographics and visual and cognitive function. Phys Occup Ther Geriatr 2011 Mar 02;29(1):23-43. [doi: 10.3109/02703181.2010.547657] 85. Manafò E, Wong S. Assessing the eHealth literacy skills of older adults: a preliminary study. J Consum Health Internet 2012 Oct;16(4):369-381. [doi: 10.1080/15398285.2012.701163] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 13 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al 86. Marshall A, Henwood F, Guy ES. Information and health literacy in the balance: findings from a study exploring the use of ICTs in weight management. Library Trends 2012;60(3):479-496. [doi: 10.1353/lib.2012.0006] 87. Gutierrez N, Kindratt TB, Pagels P, Foster B, Gimpel NE. Health literacy, health information seeking behaviors and internet use among patients attending a private and public clinic in the same geographic area. J Community Health 2014 Feb;39(1):83-89. [doi: 10.1007/s10900-013-9742-5] [Medline: 23900880] 88. Subramaniam M, St. Jean B, Taylor NG, Kodama C, Follman R, Casciotti D. Bit by bit: using design-based research to improve the health literacy of adolescents. JMIR Res Protoc 2015 May 29;4(2):e62 [FREE Full text] [doi: 10.2196/resprot.4058] [Medline: 26025101] 89. Furnival A, Silva-Jerez N. The general public's access and use of health information: a case study from Brazil. In: Proceedings of ISIC, the Information Behaviour Conference Part 1. 2014 Presented at: Information Seeking in Context: The Information Behavior Conference ISIC; Sept 2-5; Leeds, UK p. 2-5 URL: http://InformationR.net/ir/19-4/isic/isic15.html 90. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med 2005;3(6):514-522 [FREE Full text] [doi: 10.1370/afm.405] [Medline: 16338915] 91. Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med 1993 Jun;25(6):391-395. [Medline: 8349060] 92. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns 1999 Sep;38(1):33-42. [Medline: 14528569] 93. Pander MH, Essink-Bot M, Leenaars KE, Fransen MP. A short assessment of health literacy (SAHL) in the Netherlands. BMC Public Health 2014 Sep 23;14:990 [FREE Full text] [doi: 10.1186/1471-2458-14-990] [Medline: 25246170] 94. Thorn W. OECD Education Working Papers, No. 26. Paris: OECD Publisher; 2009. International Adult Literacy and Basic Skills Surveys in the OECD Region URL: https://www.oecd-ilibrary.org/docserver/221351213600. pdf?expires=1542639491&id=id&accname=guest&checksum=F651F2E595F3CEF9FF360A18B5EFAB2B[accessed 2018-11-19] 95. Hu X, Bell RA, Kravitz RL, Orrange S. The prepared patient: information seeking of online support group members before their medical appointments. J Health Commun 2012;17(8):960-978. [doi: 10.1080/10810730.2011.650828] [Medline: 22574697] 96. Lee K, Hoti K, Hughes JD, Emmerton LM. Consumer use of “Dr Google”: a survey on health information-seeking behaviors and navigational needs. J Med Internet Res 2015;17(12):e288 [FREE Full text] [doi: 10.2196/jmir.4345] [Medline: 26715363] 97. Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006 Mar 24;7:21 [FREE Full text] [doi: 10.1186/1471-2296-7-21] [Medline: 16563164] 98. Niemelä R, Ek S, Eriksson-Backa K, Huotari ML. A screening tool for assessing everyday health information literacy. Int J Lib Inf Sci 2012;62(2):125-134. [doi: 10.1515/libri-2012-0009] 99. Woods SP, Iudicello JE, Morgan EE, Cameron MV, Doyle KL, Smith TV, HIV Neurobehavioral Research Program (HNRP) Group. Health-related everyday functioning in the internet age: HIV-associated neurocognitive disorders disrupt online pharmacy and health chart navigation skills. Arch Clin Neuropsychol 2016 Mar;31(2):176-185 [FREE Full text] [doi: 10.1093/arclin/acv090] [Medline: 26743327] 100. Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res 2013 May;15(5):e93 [FREE Full text] [doi: 10.2196/jmir.2645] [Medline: 23639979] 101. Rowsell A, Muller I, Murray E, Little P, Byrne CD, Ganahl K, et al. Views of people with high and low levels of health literacy about a digital intervention to promote physical activity for diabetes: a qualitative study in five countries. J Med Internet Res 2015 Oct 12;17(10):e230 [FREE Full text] [doi: 10.2196/jmir.4999] [Medline: 26459743] 102. van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth literacy scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011;13(4):e86 [FREE Full text] [doi: 10.2196/jmir.1840] [Medline: 22071338] 103. van der Vaart R, Drossaert CH, Taal E, van de Laar MA. Patient preferences for a hospital-based rheumatology interactive health communication application and factors associated with these preferences. Rheumatology (Oxford) 2011 Sep;50(9):1618-1626 [FREE Full text] [doi: 10.1093/rheumatology/ker161] [Medline: 21551221] 104. Blackstock OJ, Cunningham CO, Haughton LJ, Garner RY, Norwood C, Horvath KJ. Higher eHealth literacy is associated with HIV risk behaviors among HIV-infected women who use the internet. J Assoc Nurses AIDS Care 2016;27(1):102-108 [FREE Full text] [doi: 10.1016/j.jana.2015.09.001] [Medline: 26456347] 105. Milne RA, Puts MT, Papadakos J, Le LW, Milne VC, Hope AJ, et al. Predictors of high eHealth literacy in primary lung cancer survivors. J Cancer Educ 2015 Dec;30(4):685-692. [doi: 10.1007/s13187-014-0744-5] [Medline: 25355524] 106. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan;14(1):e19 [FREE Full text] [doi: 10.2196/jmir.1619] [Medline: 22357448] 107. Gironda M, Der-Martirosian C, Messadi D, Holtzman J, Atchison K. A brief 20-item dental/medical health literacy screen (REALMD-20). J Public Health Dent 2013;73(1):50-55 [FREE Full text] [doi: 10.1111/jphd.12005] [Medline: 23293880] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 14 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al 108. Duncan M, Vandelanotte C, Kolt GS, Rosenkranz RR, Caperchione CM, George ES, et al. Effectiveness of a web- and mobile phone-based intervention to promote physical activity and healthy eating in middle-aged males: randomized controlled trial of the ManUp study. J Med Internet Res 2014;16(6):e136 [FREE Full text] [doi: 10.2196/jmir.3107] [Medline: 24927299] 109. Chan CV, Kaufman DR. A framework for characterizing eHealth literacy demands and barriers. J Med Internet Res 2011 Nov;13(4):e94 [FREE Full text] [doi: 10.2196/jmir.1750] [Medline: 22094891] 110. Chang F, Chiu C, Chen P, Miao N, Lee C, Chiang J, et al. Relationship between parental and adolescent eHealth literacy and online health information seeking in Taiwan. Cyberpsychol Behav Soc Netw 2015 Oct;18(10):618-624. [doi: 10.1089/cyber.2015.0110] [Medline: 26375050] 111. Connolly KK, Crosby ME. Examining e-Health literacy and the digital divide in an underserved population in Hawai'i. Hawaii J Med Public Health 2014 Feb;73(2):44-48 [FREE Full text] [Medline: 24567867] 112. Haun JN, Patel NR, Lind JD, Antinori N. Large-scale survey findings inform patients' experiences in using secure messaging to engage in patient-provider communication and self-care management: a quantitative assessment. J Med Internet Res 2015;17(12):e282 [FREE Full text] [doi: 10.2196/jmir.5152] [Medline: 26690761] 113. Hogan TP, Hill JN, Locatelli SM, Weaver FM, Thomas FP, Nazi KM, et al. Health information seeking and technology use among veterans with spinal cord injuries and disorders. PM R 2016 Feb;8(2):123-130. [doi: 10.1016/j.pmrj.2015.06.443] [Medline: 26164352] 114. Hsu W, Chiang C, Yang S. The effect of individual factors on health behaviors among college students: the mediating effects of eHealth literacy. J Med Internet Res 2014;16(12):e287 [FREE Full text] [doi: 10.2196/jmir.3542] [Medline: 25499086] 115. Knapp C, Madden V, Marcu M, Wang H, Curtis C, Sloyer P, et al. Information seeking behaviors of parents whose children have life-threatening illnesses. Pediatr Blood Cancer 2011 May;56(5):805-811. [doi: 10.1002/pbc.22674] [Medline: 21370415] 116. Knapp C, Madden V, Wang H, Sloyer P, Shenkman E. Internet use and eHealth literacy of low-income parents whose children have special health care needs. J Med Internet Res 2011;13(3):e75 [FREE Full text] [doi: 10.2196/jmir.1697] [Medline: 21960017] 117. Koo M, Norman CD, Chang HM. Psychometric evaluation of a Chinese version of the eHealth literacy scale (eHEALS) in school age children. Int Electronic J Health Educ 2012;15:29-36 [FREE Full text] 118. Li N, Orrange S, Kravitz RL, Bell RA. Reasons for and predictors of patients' online health information seeking following a medical appointment. Fam Pract 2014 Oct;31(5):550-556. [doi: 10.1093/fampra/cmu034] [Medline: 24963151] 119. Mitsutake S, Shibata A, Ishii K, Oka K. Association of eHealth literacy with colorectal cancer knowledge and screening practice among internet users in Japan. J Med Internet Res 2012;14(6):e153 [FREE Full text] [doi: 10.2196/jmir.1927] [Medline: 23149453] 120. Nahm E, Resnick B, Orwig D, Magaziner J, Bellantoni M, Sterling R, et al. A theory-based online hip fracture resource center for caregivers: effects on dyads. Nurs Res 2012;61(6):413-422 [FREE Full text] [doi: 10.1097/NNR.0b013e318270712a] [Medline: 23052420] 121. Ossebaard HC, Seydel ER, van Gemert-Pijnen L. Online usability and patients with long-term conditions: a mixed-methods approach. Int J Med Inform 2012 Jun;81(6):374-387. [doi: 10.1016/j.ijmedinf.2011.12.010] [Medline: 22261086] 122. Park H, Cormier E, Gordon G, Baeg JH. Identifying health consumers' eHealth literacy to decrease disparities in accessing eHealth information. Comput Inform Nurs 2016 Feb;34(2):71-99. [doi: 10.1097/CIN.0000000000000205] [Medline: 26657619] 123. Park H, Moon M, Baeg JH. Association of eHealth literacy with cancer information seeking and prior experience with cancer screening. Comput Inform Nurs 2014 Sep;32(9):458-463. [doi: 10.1097/CIN.0000000000000077] [Medline: 25105588] 124. Reininger B, Mecca LP, Stine KM, Schultz K, Ling L, Halpern D. A type 2 diabetes prevention website for african americans, caucasians, and mexican americans: formative evaluation. JMIR Res Protoc 2013;2(2):e24 [FREE Full text] [doi: 10.2196/resprot.2573] [Medline: 23846668] 125. Sheng X, Simpson PM. Seniors, health information, and the internet: motivation, ability, and internet knowledge. Cyberpsychol Behav Soc Netw 2013 Oct;16(10):740-746. [doi: 10.1089/cyber.2012.0642] [Medline: 23679569] 126. van der Vaart R, Drossaert CH, de Heus M, Taal E, van de Laar MA. Measuring actual eHealth literacy among patients with rheumatic diseases: a qualitative analysis of problems encountered using Health 1.0 and Health 2.0 applications. J Med Internet Res 2013 Feb;15(2):e27 [FREE Full text] [doi: 10.2196/jmir.2428] [Medline: 23399720] 127. Li TM, Chau M, Wong PW, Lai ES, Yip PS. The association between online health information-seeking behaviors and health behaviors among hispanics in New York city: a community-based cross-sectional study. J Med Internet Res 2013;15(5):e80 [FREE Full text] [doi: 10.2196/jmir.2316] [Medline: 23676714] 128. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother 2013;20(1):10-27. [doi: 10.1002/cpp.770] [Medline: 21887811] http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 15 XSL FO (page number not for citation purposes) • RenderXJOURNAL OF MEDICAL INTERNET RESEARCH Huhta et al 129. Taylor-Rodgers E, Batterham PJ. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J Affect Disord 2014 Oct 15;168:65-71. [doi: 10.1016/j.jad.2014.06.047] [Medline: 25038293] 130. Tse CK, Bridges SM, Srinivasan DP, Cheng BS. Social media in adolescent health literacy education: a pilot study. JMIR Res Protoc 2015 Mar 09;4(1):e18 [FREE Full text] [doi: 10.2196/resprot.3285] [Medline: 25757670] 131. Allam A, Schulz PJ, Nakamoto K. The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output. J Med Internet Res 2014;16(4):e100 [FREE Full text] [doi: 10.2196/jmir.2642] [Medline: 24694866] 132. Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res 2006 Nov;8(4):e27 [FREE Full text] [doi: 10.2196/jmir.8.4.e27] [Medline: 17213046] 133. Haun J, Noland D, Graham-Pole J, Rienzo B, Donaldson P. Testing a health literacy screening tool: implications for utilization of a BRIEF health literacy indicator. Fed Pract 2009;26(12):24-31. 134. Lee JY, Rozier RG, Lee SD, Bender D, Ruiz RE. Development of a word recognition instrument to test health literacy in dentistry: the REALD-30 - a brief communication. J Public Health Dent 2007;67(2):94-98. [Medline: 17557680] 135. Pleasant A, McKinney J, Rikard RV. Health literacy measurement: a proposed research agenda. J Health Commun 2011;16(Suppl 3):11-21. [doi: 10.1080/10810730.2011.604392] [Medline: 21951240] Abbreviations BRIEF: Brief Health Literacy Screening Tool eSEARCH: eHealth Literacy Tool eHealth: electronic health eHEALS: eHealth Literacy Scale EHIL: Everyday Health Information Literacy FCCHL: Functional Communicative and Critical Health Literacy scale PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses REALM: Rapid Estimate in Adult Literacy in Medicine TOHFLA: Test for Functional Health Literacy in Adults WHO: World Health Organization Edited by G Eysenbach; submitted 02.03.18; peer-reviewed by B Xie, K Sørensen, I Mircheva; comments to author 12.04.18; revised version received 07.09.18; accepted 25.09.18; published 19.12.18 Please cite as: Huhta AM, Hirvonen N, Huotari ML Health Literacy in Web-Based Health Information Environments: Systematic Review of Concepts, Definitions, and Operationalization for Measurement J Med Internet Res 2018;20(12):e10273 URL: http://www.jmir.org/2018/12/e10273/ doi: 10.2196/10273 PMID: 30567690 ©Anna-Maija Huhta, Noora Hirvonen, Maija-Leena Huotari. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 19.12.2018. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. http://www.jmir.org/2018/12/e10273/ J Med Internet Res 2018 | vol. 20 | iss. 12 | e10273 | p. 16 XSL FO (page number not for citation purposes) • RenderX