Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> This article provides a detailed examination of the oral proficiency interview and its underlying assumptions of validity and value. An attempt is made, by looking at the interview from the inside, to understand what OPIs are and what the participants in them do. The similarities and differences between interviews and conversations are examined, and some of the major problems of proficiency interviewing are illustrated and discussed. Finally, some specific proposals are made that may lead to more effective oral interviewing procedures. In this article I want to look at oral proficiency interviews (OPIs) as instances of conversation between people. Two questions are central to this discussion: (a) Are OPIs examples of conversational language use? and (b) Is conversational language use the appropriate (or the only, or the best) vehicle to evaluate oral proficiency? The tentative answer to the first question is a qualified no, the qualification being that an OPI can and may be designed to elicit conversational language use but frequently is not. The second question is far more difficult to answer: It addresses the central issue of the construct validity of oral tests in general. However, I propose that conversation is an appropriate vehicle for the all-around display of speaking ability in context. If the latter is a definition of oral proficiency, then I think we can be reasonably sure that conversation is the best vehicle for its evaluation. To address the questions I have posed, one must obtain an "insider's view" of the OPI. To do this, I have relied on three sources of data: (a) the analysis of several OPIs I have taken as an interviewee, (b) the study of transcripts and tapes of a variety of <s> BIB001 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> Language testing is concerned with the measurement of language. Language is the trait, and how we go about measuring it is the method. Trait involves the what , and method the how of testing. This paper will focus on the interaction between the what and the how , where the perspective of the what is the domain of discourse analysis. The paper will also review studies and tests that illustrate the degree to which the domain of discourse analysis has contributed to the theory, research, and development of language testing. <s> BIB002 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> Preface 1. Introduction The aims of the book The climate for language testing Research and development: needs and problems Research and development: an agenda Overview of the book Notes 2. Measurement Introduction Definition of terms: measurement, test, evaluation Essential measurement qualities Properties of measurement scales Characteristics that limit measurement Steps in measurement Summary Notes Further reading Discussion questions 3. Uses of Language Tests Introduction Uses of language tests in educational programs Research uses of language tests Features for classifying different types of language test Summary Further reading Discussion questions 4. Communicative Language Ability Introduction Language proficiency and communicative competence A theoretical framework of communicative language ability Summary Notes Further reading Discussion questions 5. Test Methods Introduction A framework of test method facets Applications of this framework to language testing Summary Notes Further reading Discussion questions 6. Reliability Introduction Factors that affect language test scores Classical true score measurement theory Generalizability theory Standard error of measurement: Interpreting individual test scores within classical true score and generalizability theory Item response theory Reliability of criterion-referenced test scores Factors that affect reliability estimates Systematic measurement error Summary Notes Further reading Discussion questions 7. Validation Introduction Reliability and validity revisited Validity as a unitary concept The evidential basis of validity Test bias The consequential or ethical basis of validity Post mortem: face validity Summary Notes Further reading Discussion questions 8. Some Persistent Problems and Future Directions Introduction Authentic language tests Some future directions A general model for explaining performance on language tests Apologia et prolegomenon Summary Notes Further reading Discussion questions Bibliography Author index Subject index <s> BIB003 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> Abstract Large-scale experiments on the testing of oral proficiency in English, French and German have been carried out over the last 5 years in the Swedish gymnasium. Various kinds of tasks and different grading criteria have been used, and the practical problems of scheduling and of teacher training have been discussed. There are plans for a period of mandatory, full-scale participation, but no decisions have yet been taken. <s> BIB004 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> Preface 1 Testing, Assessing, and Teaching What Is a Test? Assessment and Teaching Informal and Formal Assessment Formative and Summative Assessment Norm-Referenced and Criterion-Referenced Test Approaches to Language Testing: A Brief History Discrete-point and Integrative Testing Communicative Language Testing Performance-Based Assessment Current Issues in Classroom Testing New Views on Intelligence Traditional and "Alternative" Assessment Computer-Based Testing Exercises For Your Further Reading 2 Principles of Language Assessment Practicality Reliability Student-Related Reliability Rater Reliability Test Administration Reliability Test Reliability Validity Content-Related Evidence Criterion-Related Evidence Construct-Related Evidence Consequential Validity Face Validity Authenticity Washback Applying Principles to the Evaluation of Classroom Tests 1. Are the test procedures practical? 2. Is the test reliable? 3. Does the procedure demonstrate content validity? 4. Is the procedure face valid and "biased for best"? 5. Are the test tasks as authentic as possible? 6. Does the test offer beneficial washback to the learner? Exercises For Your Further Reading 3 Designing Classroom Language Tests Test Types Language Aptitude Tests Proficiency Tests Placement Test Diagnostic Tests Achievement Tests Some Practical Steps to Test Construction Assessing Clear, Unambiguous Objectives Drawing Up Test Specifications Devising Test Tasks Designing Multiple-Choice Test Items 1. Design each item to measure a specific objective 2. State Both stem and options as simply and directly as possible 3. Make certain that the intended answer is clearly the only correct one 4. Use item indices to accept, discard, or revise items Scoring, Grading, and Giving Feedback Scoring Grading Giving Feedback Exercises For Your Further Reading 4 Standardized Testing What Is Standardization? Advantages and Disadvantages of Standardized Tests Developing a Standardized Test 1. Determine the purpose and objectives of the test 2. Design test specifications 3. Design, select, and arrange test tasks/items 4. Make appropriate evaluations of different kinds of items 5. Specify scoring procedures and reporting formats 6. Perform ongoing construct validation studies Standardized Language Proficiency Testing Four Standardized Language Proficiency Tests Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) Exercises For Your Further Reading Appendix to Chapter 4: Commercial Proficiency Tests: Sample Items and Tasks Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) 5 Standards-Based Assessment ELD Standards ELD Assessment CASAS and SCANS Teacher Standards The Consequences of Standards-Based and Standardized Testing Test Bias Test-Driven Learning and Teaching Ethical Issues: Critical Language Testing Exercises For Your Further Reading 6 Assessing Listening Observing the Performance of the Four Skills The Importance of Listening Basic Types of Listening Micro- and Macroskills of Listening Designing Assessment Tasks: Intensive Listening Recognizing Phonological and Morphological Elements Paraphrase Recognition Designing Assessment Tasks: Responsive Listening Designing Assessment Tasks: Selective Listening Listening Cloze Information Transfer Sentence Repetition Designing Assessment Tasks: Extensive Listening Dictation Communicative Stimulus-Response Tasks Authentic Listening Tasks Exercises For Your Further Reading 7 Assessing Speaking Basic Types of Speaking Micro- and Macroskills of Speaking Designing Assessment Tasks: Imitative Speaking PhonePass Test Designing Assessment Tasks: Intensive Speaking Directed Response Tasks Read-Aloud Tasks Sentence/Dialogue Completion Tasks and Oral Questionnaires Picture-Cued Tasks Translation (of Limited Stretches of Discourse) Designing Assessment Tasks: Responsive Speaking Question and Answer Giving Instructions and Directions Paraphrasing Test of Spoken English (TSE) Designing Assessment Tasks: Interactive Speaking Interview Role Play Discussions and Conversations Games Oral Proficiency Interview (OPI) Designing Assessment: Extensive Speaking Oral Presentations Picture-Cued Story-Telling Retelling a Story, News Event Translation (of Extended Prose) Exercises For Your Further Reading 8 Assessing Reading Types (Genres) of Reading Microskills, Macroskills, and Strategies for Reading Types of Reading Designing Assessment Tasks: Perceptive Reading Reading Aloud Written Response Multiple-Choice Picture-Cued Items Designing Assessment Tasks: Selective Reading Multiple-Choice (for Form-Focused Criteria) Matching Tasks Editing Tasks Picture-Cued Tasks Gap-Filling Tasks Designing Assessment Tasks: Interactive Reading Cloze Tasks Impromptu Reading Plus Comprehension Questions Short-Answer Tasks Editing (Longer Texts) Scanning Ordering Tasks Information Transfer: Reading Charts, Maps, Graphs, Diagrams Designing Assessment Tasks: Extensive Reading Skimming Tasks Summarizing and Responding Note-Taking and Outlining Exercises For Your Further Reading 9 Assessing Writing Genres of Written Language Types of Writing performance Micro- and Macroskills of Writing Designing Assessment Tasks: Imitative Writing Tasks in [Hand] Writing Letters, Words, and Punctuation Spelling Tasks and Detecting Phoneme-Grapheme Correspondences Designing Assessment Tasks: Intensive (Controlled) Writing Dictation and Dicto-Comp Grammatical Transformation Tasks Picture-Cued Tasks Vocabulary Assessment Tasks Ordering Tasks Short-Answer and Sentence Completion Tasks Issues in Assessing Responsive and Extensive Writing Designing Assessment Tasks: Responsive and Extensive Writing Paraphrasing Guided Question and Answer Paragraph Construction Tasks Strategic Options Test of Written English (TWE(R)) Scoring Methods for Responsive and Extensive Writing Holistic Scoring Primary Trait Scoring Analytic Scoring Beyond Scoring Responding to Extensive Writing Assessing Initial Stages of the Process of Composing Assessing Later Stages of the Process of Composing Exercises For Your Further Reading 10 Beyond Tests: Alternatives in Assessment The Dilemma of Maximizing Both Practicality and Washback Performance-Based Assessment Portfolios Journals Conferences and Interviews Observations Self- and Peer-Assessments Types of Self- and Peer-Assessment Guidelines for Self- and Peer- Assessment A Taxonomy of Self- and Peer-Assessment Tasks Exercises For Your Further Reading 11 Grading and Student Evaluation Philosophy of Grading: What Should Grades Reflect? Guidelines for Selecting Grading Criteria Calculating Grades: Absolute and Relative Grading Teachers' Perceptions of Appropriate Grade Distributions Institutional Expectations and Constraints Cross-Cultural Factors and the Question of Difficulty What Do Letter Grades "Mean"? Alternatives to Letter Grading Some Principles and Guidelines for Grading and Evaluation Exercises For Your Further Reading Bibliography Name Index Subject Index <s> BIB005 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> 0 The TOEFL (Test of English as a Foreign Language) is a high-stakes language test intended to measure proficiency in English in an academic setting. A new TOEFL will become operational in the fall of 2005. In developing this new TOEFL, we decided to modify the test in three important ways (Jamieson, Jones, Kirsch, Mosenthal, & Taylor, 2000). One is to include more constructed-response or performance tasks (including speaking) that require the integration of more than one language skill (e.g., writing or speaking in response to listening and reading materials). A second is to develop test materials that more closely reflect the types of discourse that occur in academic settings. And a third is to provide information that will help test users better understand the meaning of test scores. Educational Testing Service carried out an extensive program of research to achieve these goals and to make sure that a new test would meet professional standards (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). But as is often the case in doing research, addressing one question makes us aware of other unanswered questions. This discussion focuses both on the research Educational Testing Service carried out to support TOEFL's new directions and other issues yet to be explored. <s> BIB006 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> SEARCH RESULTS <s> Abstract: Communicative approaches to teaching language have emphasized the centrality of oral proficiency in the language acquisition process, but research investigating oral proficiency has been surprisingly limited, yielding an incomplete understanding of spoken language development. This study investigated the development of spoken language at the high school level over five consecutive years, involving more than 1,500 students representing 23 school districts. Quantitative Standards-Based Measure of Proficiency speaking scores and student]produced qualitative spoken samples (n>6,000 samples) contributed to an understanding of the development of spoken language. Hierarchical linear modeling (HLM) revealed a consistent growth trajectory of spoken language development, and results indicated that 18.30% of the variance in student outcomes may be attributed to the teacher variable.Key words: classroom]based research, longitudinal study, mixed methods, oral language developmentThe rise of communicative language learning has led to widespread acceptance of communicative competence as a primary goal of language education and, as such, central to good classroom practice (Savignon, 1997). This approach to language instruction emphasizes the ability to communicate in a second language in real]life situations both inside and beyond the classroom. Instead of measuring language learning in terms of seat time, test scores, or number of credit hours, communicative skills are demonstrated through task]based communicative activities. As a result of this emphasis on oral communication, proficiency has emerged as central to communicative language learning and teaching. However, there is a lack of research at the classroom level that reveals what students are able to do with oral language after one, two, three, and four years of language study. There is a paucity of research relating specifically to the development of spoken language at the secondary level (Tschirner & Heilenman, 1998). Although several studies have offered a glimpse of classroom-based proficiency ratings for the high school language learner (for examples, see Glisan & Foltz, 1998; Huebner & Jensen, 1992; Moeller & Reschke, 1993; Steinmeyer, 1984), the data have been strictly quantitative and have been conducted within educational systems with no consideration of related and potentially confounding factors, such as the teacher, during the exploration of oral language production.Due to such limitations, as well as substantial differences in results, particularly at the beginning levels of language learning in secondary classrooms, this study explored students' progress toward proficiency over a period of 5 years using a combination of qualitative methods, including thematic coding and organization, to reveal overarching trends in oral spoken language, and quantitative methods, including the Standards- Based Measurement of Proficiency (STAMP) test, a teacher-independent, computer- mediated measure of oral language proficiency.1 Purposefully integrating mixed methods offers "a very powerful mix" (Miles & Huberman, 1994, p. 42) that develops "a complex" picture of oral language development (Greene & Caracelli, 1997, p. 7). In choosing this integrative data design, the researchers' purpose was one of complementarity, a design element used to measure overlapping, but distinct, facets of a phenomenon under investigation (Caracelli & Greene, 1993). Results from one method-in this case, qualitative data-were used to enhance, illustrate, or clarify results from the other method-in this case, quantitative data (Greene & McClintock, 1985).Quantitative research questions for this study investigated the growth trajectory of spoken Spanish over four consecutive years of high school Spanish learning. Quantitative questions also delved into the variance in spoken production scores that was attributable to teacher differences or individual student differences. … <s> BIB007
As expected, the journal Language Testing included the highest number of hits (58). Language Testing and five other journals -Applied Linguistics, Language Assessment Quarterly, Annual Review of Applied Linguistics, ELT Journal, and TESOL Quarterly -explicitly mention 'assessment' and/or 'testing' in their aims and scope, whereas the remaining nine journals examined do not. In light of this, both Language Assessment Quarterly (8 hits) and TESOL Quarterly (7) had proportionally few hits, whereas Foreign Language Annals (27) had surprisingly many. The search results also reveal that seven of the journals had two or fewer hits, which probably indicate that they do not attract submissions on L2 oral testing. Although the search criteria were adhered to, decisions on which studies to include were sometimes difficult to make, mainly because of mismatches between titles and actual article content, and the number of studies with mixed methodological approaches where discursive or interactional analysis constituted a very minor part of the study as a whole. The remainder of the present paper is structured as follows. The immediately following section situates the review in relation to developments in the field of L2 oral proficiency testing; the section also includes accounts of frequently used test formats. Next follows the review of empirical studies on discourse and social interaction in oral proficiency tests (2004) (2005) (2006) (2007) (2008) (2009) (2010) (2011) (2012) (2013) (2014) , divided into three subsections: candidate-examiner tests, paired tests, and group tests. These subsections, in turn, are organized on the basis of themes emerging from the studies examined. Understanding the context: oral proficiency testing formats L2 oral proficiency can be defined as learners' ability to converse with one or several interlocutors (cf. 'interactional competence' in an L2, Kasper & Ross 2013: 9) . It is beyond the scope of this review to explore the definition in detail, but it aligns with BIB003 theoretical framework of communicative language ability, which encompasses five components: strategic competence, knowledge structures, language competence, psychophysiological mechanisms, and the context of situation. Bachman's work largely builds upon  seminal paper, in which they make an effort to 'determine the feasibility and practicality of measuring (…) "communicative competence" ' (p. 1). In their proposed theoretical framework, Canale and Swain list three main competencies: grammatical competence, sociolinguistic competence, and strategic competence. Further, they claim that the studies of sociolinguistic competence and grammatical competence are equally important to the study of communicative competence. In fact, sociolinguistic competence is said to be crucial in a theory of communicative competence and 'particularly deserving of research with respect to second language teaching and testing ' (p. 17) . With regard to testing, they highlight, among other things, that L2 speakers must be given the chance to participate in meaningful communicative interaction (cf. the use of the term social interaction in this review). With their contribution, Canale and Swain redirected scholarly attention away from a focus on test scores to one on the social construction of the assessment itself. Canale and Swain's work also underpins BIB003 concept of language competence, which builds upon organizational and pragmatic competence, respectively. Further, the concept of L2 oral proficiency as discussed in the present review ref lects the communicative abilities as regards speaking and interaction described in the Common European Framework of Reference for Languages (Council of Europe 2001; see also  as well as the functional language ability described in the ACTFL (American Council on the Teaching of Foreign Languages) Proficiency Guidelines (ACTFL 2012). Following upon the shift initiated in the early 1980s, in which Canale and Swain's work played a central role, a number of papers which in various ways picked up on the social construction of language testing were published. One seminal study was a paper by BIB001 , which focused specifically 'what OPIs are and what the participants in them do' (489). Van Lier's study called attention to the micro-level construction of oral testing discourse and attempted to define the very nature of such social interactions in comparison to non-test conversations (see also  . Van Lier's paper was significant in directing researchers with an interest in the process (rather than the outcome) of oral proficiency testing, toward empirical studies of testing discourse and interaction. As regards research on oral proficiency (OP) in an L2, BIB007 rightly argue that it is limited. As a consequence, research on the testing of L2 OP is even more limited and, therefore, much needed (cf. BIB006 . Research on discourse and social interaction in OP testing contexts is also central to test construction and validation, and Lazaraton (2004: 53) states that 'it seems clear that more attention to and incorporation of discourse analysis in language test validation is needed' (see also  BIB002 ). However, in order to situate the review of empirical studies in the past decade, an overview of the most common formats for testing OP in face-to-face contexts follows next. BIB005 suggest that language tests be classified according to their purpose and mention five test formats: achievement, diagnostic, placement, proficiency, and aptitude tests. This overview concerns placement and proficiency tests. The former attempts to place a learner at a specific level of a language curriculum or school (which may include course material), whereas the latter aims to test overall language competence. Due to its status as a global language, English proficiency tests are very common, such as the Test of English as a Foreign Language (TOEFL) and the Cambridge ESOL (English for Speakers of Other Languages) Examinations, including the International English Language Testing System (IELTS). Upon completion of these tests, test-takers/candidates receive a certificate that indicates the level of competence. Roca-Varela and Palacios (2013) discuss the negative washback of international L2 tests: special L2 programs are designed that teach for the test only ( for an overview, see  . Teaching for the test is very different from the positive washback that can be an effect of L2 OP testing BIB004 . In reference to establishing assessment criteria in L2 tests, then, Roca-Varela and Palacios (2013: 66) suggest that 'oral tests should become more human'. The three proficiency tests listed above all include a module for speaking, but there are also pure L2 speaking tests.