Examining the Definition and Measurement of Quality in Early Childhood Education: A Review of Studies Using the ECERS R from 2003 to 2010.
The field of early childhood education continues to grapple with the issue of understanding quality in classrooms. The lack of clarity in definition (or conceptualization) and related ability to assess (or operationalize) quality has contributed to a reliance on the Early Childhood Environment Rating Scale (ECERS R) which is often interpreted to be synonymous with the quality of a classroom. Likewise, the ECERS R (although a measurement tool) is often used to define quality. Because of the widespread use of this measure as an evaluation tool, early childhood programs have strived to achieve high ratings on this measure, and subsequently the item content of the ECERS R has often become a focus for quality enhancement initiatives. The present study examines the definitions of quality (i.e. how quality is operationalized) in research studies using the ECERS R over the past 8 years (2003 2010) A content analysis of 76 studies conducted in the United States indicates that studies using the ECERS R to operationalize quality do not use a consistent definition; instead they conceptualize quality in a variety of ways ranging from "quality is ECERs" or "classroom quality" to "environmental quality." In light of these varying definitions, implications for research and policy in early childhood education are discussed. Quality in Early Childhood Education The study of quality in early childhood education (ECE) is increasingly relevant because research has continued to demonstrate consistent associations between various aspects of classroom quality and improved social and academic outcomes for young children (Belsky et al. 2007; Helburn et al. 1995; Howes et al. 2008; Mashburn et al. 2008) In response to such findings, national and state policies in the United States have focused on improving the quality of care and education for young children. Although quality continues to be a major focus of research in ECE, defining this construct and measuring it continue to challenge the field both methodologically in research and practically in quality enhancement initiatives aimed at ensuring optimal experiences for children in early childhood education programs. Overview of ECERS R Use The original version of the Early Childhood Environment Rating Scale Revised .a peer reviewed multilingual journal on the development, care, and education of young children Chinese espanol If you have benefited from free access to ECRP, please consider making a financial contribution to ECRP so that the journal can continue to be available free to everyone. (ECERS R) developed in the United States, was an effort to provide a selfassessment tool for programs and to identify program strengths and weaknesses for continuous quality improvement (Harms, Clifford, Cryer, 1998) However, over time, the ECERS R has become a primary measure used in research to assess quality and to inform policy and programmatic decisions. As a pioneering selfassessment tool in early childhood education, it has been a primary means of obtaining an operationalized view of child care quality in large scale national studies, such as the Cost, Quality, and Child Outcomes in Child Care Centers study (CQCO; Helburn et al. 1995) and the National Child Care Staffing Study (NCCSS; Whitebook, Howes, Phillips, 1989) Accordingly, numerous policies were informed and programs were developed and revised based on results from these studies. More recent large studies of classroom quality such as the NCEDL's Multi State Study of Pre Kindergarten and Study of State Wide Early Education Programs (SWEEP) (e.g. Early et al. 2005) also use the ECERS R as a primary measure or operationalization of quality in early childhood classrooms. Predominant use of the ECERS R in large national studies and the adoption of the ECERS R by many states for quality enhancement initiatives including Tiered Quality Rating and Improvement Systems (TQRISs; Tout, Starr, Soli, Moodie, Kirby, Boller, 2010; Caronongan, Kirby, Malone, Boller, 2011) or regulatory systems have led to the perception that ECERS R scores are virtually synonymous with levels of quality in early childhood education. This perception has further yielded unprecedented reliance on the ECERS R as a defining tool of the multidimensional construct of quality. Because the ECERS R is frequently used in research, the authors of the current study were interested in how studies that used the ECERS R as a measure (i.e. operationalization of quality) were defining the intended construct (i.e. conceptualization) to be measured. The link between conceptualization and operationalization of constructs is essential to ensuring the construct validity of a study (Shadish, Cook, Campbell, 2002) Therefore, an understanding of what is being measured is a critical component for research and determining policy implications from research. Given the widespread use of the ECERS R and the limited availability of alternative measures, it was hypothesized that there would be wide variation in the definition or conceptualization of quality across studies that used the ECERS R. Subsequently, it is critical that definitions used within studies that include the ECERS R are examined in order to better understand the way that researchers are overlaying their conceptualizations (or related characteristics) with measurement by the ECERS R (i.e. the operationalization) Accordingly, the research aims for the current study are (1) to analyze the definitions (i.e. methodological conceptualizations) of ECERS R as a measure of quality over a recent 8year span and (2) to discuss the implications of operationalizing quality using the ECERS R and varied definitions for policy and quality enhancement in ECE. Definitions of Quality Historically, definitions of quality in ECE have included multiple proximal (e.g. curriculum and classroom interactions) and distal (e.g. program and state policies) features of classrooms that promote children's development in various domains (Dunn, 1993) However, because of the multitude of perspectives regarding what indicators of either proximal or distal features are most important, the resulting definitions of quality are often broad or nonspecific (Layzer Goodson, 2006) In response to such broad definitions, some researchers have conceptualized ECE quality in terms of global quality with two primary components "structural" and "process" quality (see Vandell Wolfe, 2000 for a review of the literature) Examples of indicators of structural quality include classroom materials, curriculum, teacher education, and teacher child ratio. These indicators are often the regulated aspects of classrooms and programs. Indicators of process quality focus on the more dynamic aspects of early childhood education, including human interactions occurring in the classrooms such as teacher child and peer to peer interactions (Cassidy et al. 2005a; Hamre Pianta, 2007; Vandell Wolfe, 2000) Several studies have reported correlations between variables of process, structural, and global quality (Burchinal, Cryer, Clifford, Howes, 2002; Phillipsen, Burchinal, Howes, Cryer, 1997; Phillips, Mekos, Scarr, McCartney, Abbot Shim, 2000) Structural quality and process quality, each a component of global quality, provide unique and essential information to understanding early childhood classrooms, yet these terms are often defined differently in research and policy reports (Layzer Goodson, 2006) Furthermore, other components of early childhood education environments important to children's development may be excluded from or diluted in broad conceptualizations and unclear operationalizations of quality. Precision in both conceptualization and operationalization is necessary to capture an intended construct (e.g. within quality) that will allow the field to move forward in its understanding of what is best for children in early childhood education programs. Because of the multidimensional nature of quality, it is likely that such definitions and measurement cannot be captured by one all encompassing or "global" term or limited to measurement by one instrument. For example, outdoor learning environments (Chakravarthi, 2009) and emotional climate (Howes, 2000; Raver, 2004; Raver, Garner, Smith Donald, 2007) are among important features of quality that may influence children's development but are not currently captured in depth in the ECERS R. Measuring Quality and the Reliance on the ECERS R For over 25 years, the ECERS, and subsequently the ECERS R, have been heralded as "the" standard. The ECERS R was designed to rate global quality of environments serving children ages 2 1/2 through 5 and is part of a family of Environment Rating Scales (ERS) Thus, it provides an overall assessment of the environment in which young children spend time. The scale assesses classrooms on 43 items across 7 subscales: Space and Furnishings, Personal Care Routines, Language/Reasoning, Activities, Interaction, Program Structure, and Parents and Staff. Each item is scored from 1 (inadequate) to 7 (excellent) Factor analyses of the ECERS R generally indicate a two factor structure "Language/Interactions" and "Materials/Activities" (Cassidy, Hestenes, Hegde, Hestenes, Mims, 2005b; Clifford et al. 2005; Sakai, Whitebook, Wishard, Howes, 2003) Similarly, results from a content analysis conducted by Cassidy and colleagues (2005a) found that half of the indicators on the ECERS R measured structural quality, materials, and activities; the other half measured elements of process quality, language, and interactions. Based on the identified factors, the ECERS R seems to capture two distinct components of quality structural and process. The breadth of information measured by the ECERS R as well as limited alternative and supplemental measures have contributed to widespread use of the ECERS R as the primary measure of quality in ECE programs and has resulted in its inf