Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> 1. Preface (by Celce-Murcia, Marianne) 2. Language Proficiency Interviews: A Discourse Approach (by He, Agnes Weiyun) 3. 1. Language Proficiency Interviews and Conversation 4. Re-analyzing the OPI: How Much Does It Look Like Natural Conversation? (by Johnson, Marysia) 5. Evaluating Learner Interactional Skills: Conversation at the Micro Level (by Riggenbach, Heidi) 6. What Happens When There's No One to Talk to? Spanish Foreign Language Discourse in Simulated Oral Proficiency Interviews (by Koike, Dale A.) 7. 2. Turns and Sequences in Language Proficiency Interviews 8. Answering Questions in LPIs: A Case Study (by He, Agnes Weiyun) 9. Framing the Language Proficiency Interview as a Speech Event: Native and Non-Native Speaker's Questions (by Moder, Carol Lynn) 10. Miscommunication in Language Proficiency Interviews of First-Year German Students: A Comparison with Natural Conversation (by Egbert, Maria M.) 11. Knowledge and Communication in Language Proficiency Interviews 12. Knowledge Structures in Oral Proficiency Interviews for International Teaching Assistants (by Mohan, Bernard A.) 13. The Use of Communication Strategies in Language Proficiency Interviews (by Yoshida-Morise, Yumiko) 14. Meaning Negociation in the Hungarian Oral Proficiency Examination of English (by Katona, Lucy) 15. 3. Language Proficiency Interviews as Cross-Cultural Encounters 16. Maintaining American Face in the Korean Oral Exam: Reflections on the Power of cross-Cultural Context (by Davies, Catherine Evans) 17. Confirmation Sequences as Intercational Resources in Korean Language Proficiency Interviews (by Kim, Kyu-hyun) 18. Divergent Frame Interpretations in Oral Proficiency Interview Interaction (by Ross, Steven) 19. "Let Them Eat Cake!" or how to Avoid Losing Your Head in Cross-Cultural Conversations (by Young, Richard) 20. Index of Names 21. Index of Topics <s> BIB001 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> Preface 1 Testing, Assessing, and Teaching What Is a Test? Assessment and Teaching Informal and Formal Assessment Formative and Summative Assessment Norm-Referenced and Criterion-Referenced Test Approaches to Language Testing: A Brief History Discrete-point and Integrative Testing Communicative Language Testing Performance-Based Assessment Current Issues in Classroom Testing New Views on Intelligence Traditional and "Alternative" Assessment Computer-Based Testing Exercises For Your Further Reading 2 Principles of Language Assessment Practicality Reliability Student-Related Reliability Rater Reliability Test Administration Reliability Test Reliability Validity Content-Related Evidence Criterion-Related Evidence Construct-Related Evidence Consequential Validity Face Validity Authenticity Washback Applying Principles to the Evaluation of Classroom Tests 1. Are the test procedures practical? 2. Is the test reliable? 3. Does the procedure demonstrate content validity? 4. Is the procedure face valid and "biased for best"? 5. Are the test tasks as authentic as possible? 6. Does the test offer beneficial washback to the learner? Exercises For Your Further Reading 3 Designing Classroom Language Tests Test Types Language Aptitude Tests Proficiency Tests Placement Test Diagnostic Tests Achievement Tests Some Practical Steps to Test Construction Assessing Clear, Unambiguous Objectives Drawing Up Test Specifications Devising Test Tasks Designing Multiple-Choice Test Items 1. Design each item to measure a specific objective 2. State Both stem and options as simply and directly as possible 3. Make certain that the intended answer is clearly the only correct one 4. Use item indices to accept, discard, or revise items Scoring, Grading, and Giving Feedback Scoring Grading Giving Feedback Exercises For Your Further Reading 4 Standardized Testing What Is Standardization? Advantages and Disadvantages of Standardized Tests Developing a Standardized Test 1. Determine the purpose and objectives of the test 2. Design test specifications 3. Design, select, and arrange test tasks/items 4. Make appropriate evaluations of different kinds of items 5. Specify scoring procedures and reporting formats 6. Perform ongoing construct validation studies Standardized Language Proficiency Testing Four Standardized Language Proficiency Tests Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) Exercises For Your Further Reading Appendix to Chapter 4: Commercial Proficiency Tests: Sample Items and Tasks Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) 5 Standards-Based Assessment ELD Standards ELD Assessment CASAS and SCANS Teacher Standards The Consequences of Standards-Based and Standardized Testing Test Bias Test-Driven Learning and Teaching Ethical Issues: Critical Language Testing Exercises For Your Further Reading 6 Assessing Listening Observing the Performance of the Four Skills The Importance of Listening Basic Types of Listening Micro- and Macroskills of Listening Designing Assessment Tasks: Intensive Listening Recognizing Phonological and Morphological Elements Paraphrase Recognition Designing Assessment Tasks: Responsive Listening Designing Assessment Tasks: Selective Listening Listening Cloze Information Transfer Sentence Repetition Designing Assessment Tasks: Extensive Listening Dictation Communicative Stimulus-Response Tasks Authentic Listening Tasks Exercises For Your Further Reading 7 Assessing Speaking Basic Types of Speaking Micro- and Macroskills of Speaking Designing Assessment Tasks: Imitative Speaking PhonePass Test Designing Assessment Tasks: Intensive Speaking Directed Response Tasks Read-Aloud Tasks Sentence/Dialogue Completion Tasks and Oral Questionnaires Picture-Cued Tasks Translation (of Limited Stretches of Discourse) Designing Assessment Tasks: Responsive Speaking Question and Answer Giving Instructions and Directions Paraphrasing Test of Spoken English (TSE) Designing Assessment Tasks: Interactive Speaking Interview Role Play Discussions and Conversations Games Oral Proficiency Interview (OPI) Designing Assessment: Extensive Speaking Oral Presentations Picture-Cued Story-Telling Retelling a Story, News Event Translation (of Extended Prose) Exercises For Your Further Reading 8 Assessing Reading Types (Genres) of Reading Microskills, Macroskills, and Strategies for Reading Types of Reading Designing Assessment Tasks: Perceptive Reading Reading Aloud Written Response Multiple-Choice Picture-Cued Items Designing Assessment Tasks: Selective Reading Multiple-Choice (for Form-Focused Criteria) Matching Tasks Editing Tasks Picture-Cued Tasks Gap-Filling Tasks Designing Assessment Tasks: Interactive Reading Cloze Tasks Impromptu Reading Plus Comprehension Questions Short-Answer Tasks Editing (Longer Texts) Scanning Ordering Tasks Information Transfer: Reading Charts, Maps, Graphs, Diagrams Designing Assessment Tasks: Extensive Reading Skimming Tasks Summarizing and Responding Note-Taking and Outlining Exercises For Your Further Reading 9 Assessing Writing Genres of Written Language Types of Writing performance Micro- and Macroskills of Writing Designing Assessment Tasks: Imitative Writing Tasks in [Hand] Writing Letters, Words, and Punctuation Spelling Tasks and Detecting Phoneme-Grapheme Correspondences Designing Assessment Tasks: Intensive (Controlled) Writing Dictation and Dicto-Comp Grammatical Transformation Tasks Picture-Cued Tasks Vocabulary Assessment Tasks Ordering Tasks Short-Answer and Sentence Completion Tasks Issues in Assessing Responsive and Extensive Writing Designing Assessment Tasks: Responsive and Extensive Writing Paraphrasing Guided Question and Answer Paragraph Construction Tasks Strategic Options Test of Written English (TWE(R)) Scoring Methods for Responsive and Extensive Writing Holistic Scoring Primary Trait Scoring Analytic Scoring Beyond Scoring Responding to Extensive Writing Assessing Initial Stages of the Process of Composing Assessing Later Stages of the Process of Composing Exercises For Your Further Reading 10 Beyond Tests: Alternatives in Assessment The Dilemma of Maximizing Both Practicality and Washback Performance-Based Assessment Portfolios Journals Conferences and Interviews Observations Self- and Peer-Assessments Types of Self- and Peer-Assessment Guidelines for Self- and Peer- Assessment A Taxonomy of Self- and Peer-Assessment Tasks Exercises For Your Further Reading 11 Grading and Student Evaluation Philosophy of Grading: What Should Grades Reflect? Guidelines for Selecting Grading Criteria Calculating Grades: Absolute and Relative Grading Teachers' Perceptions of Appropriate Grade Distributions Institutional Expectations and Constraints Cross-Cultural Factors and the Question of Difficulty What Do Letter Grades "Mean"? Alternatives to Letter Grading Some Principles and Guidelines for Grading and Evaluation Exercises For Your Further Reading Bibliography Name Index Subject Index <s> BIB002 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> This book provides a clear and accessible introduction to the main methods of analysing samples of learner language. It examines the theoretical and research bases for the different methods, and thereby serves as an introduction to the field of second language acquisition research. The book is intended for students on postgraduate courses in TESOL or Applied Linguistics, or for teachers wishing to do action research to find out more about how learners learn language. <s> BIB003 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> This article responds to a critique of Ordinate Corporation's Versant for English test (formerly PhonePass and SET-10). The critique (Chun, 2006) purports to apply the language task framework of Bachman and Palmer (1996) and the Test Fairness framework of Kunnan (2004) as guides for an analysis. However, the analysis fails to apply either framework appropriately, leading to a misguided conclusion. Here, we present readers with a more balanced analysis by clarifying the issues that the critic has confused, particularly with regard to the authenticity and construct of the test tasks. Finally, we suggest that a correct application of the usefulness framework of Bachman and Palmer (1996) shows Versant for English to be a useful test in many contexts. <s> BIB004 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> Pragmatics is a key domain in language assessment. For more than two decades, advances have been made in conceptualizing the domain, developing assessment instruments, and applying current methods of data treatment to the analysis of test performance. This book, the first edited volume on the topic, brings together empirical studies on a range of well-established and innovative strategies for the assessment of pragmatic ability in a second language. In this introductory chapter, we will first offer an overview of key concepts, situate theoretical models of pragmatic competence within the larger frameworks of communicative language ability and interactional competence, and consider the relationship between pragmatics and language testing. We will then introduce the chapters, organized into two Parts. The chapters assembled in Part I investigate assessment instruments and practices for a variety of assessment constructs, purposes, and contexts, guided by different theoretical outlooks on pragmatics. Part II comprises studies of interaction in different forms of oral proficiency interview, conducted from the perspective of conversation analysis. <s> BIB005 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> Introduction <s> This study investigates whether raters' knowledge of test takers' first language (L1) affects how the raters orient themselves to the task of rating oral speech. The authors qualitatively investigated the effects of accent familiarity on raters' score assignment processes. Twenty-six trained raters with a second language of Mandarin Chinese, Korean, or Spanish rated English speech samples from 72 English language learners from three L1 backgrounds (Mandarin Chinese, Korean, and Spanish). Afterward, raters watched videos of themselves rating and discussed their rating processes. Analyses of data, assisted by the qualitative analysis software package QSR NVivo 8, revealed that some raters attended to and were influenced by test-taker accent and L1. Raters who were heritage language learners discussed how their personal heritage status influenced how they thought about the speech samples from speakers whose L1 matched their heritage language. Although sensitivity to test-taker accents seemed to occur naturally in the rating process, findings suggest that when raters have learned or know, to varying degrees, the test takers' L1, they tend to orient themselves to the speech in a biased way, compromising test reliability. <s> BIB006
Tests are consequential for language learners. Testing procedures are also of central interest to educational institutions, authorities, and individual teachers, as tests must be designed so that they ref lect the particular aspects of a learner's language proficiency that match requirements for various placements, acceptance into education, professional positions, or national curricula. Given the increased importance given to educational testing in general, it is not surprising that research on language testing has proliferated. This review article examines empirical studies on second-language oral proficiency testing from 2004 through 2014, offering a thorough account of recent trends and developments in research on discourse and social interaction in secondlanguage oral proficiency tests. With the 'communicative movement' (McNamara & Roever 2006: 43) in second-language research, the focus shifted toward communicative language testing, which also entailed an interest in the joint construction of testing interaction by participants in situ. Such approaches take into account the 'situationally and sequentially grounded and culturally shared understandings' of speaking tests as 'particular types of speech situations' (He & Young 1998: 8) . In contrast to everyday social interactions, tests are institutional interactions. Interactions governed by institutional constraints are, to varying degrees, goal oriented and involve particular tasks, roles, and objectives of the interaction (cf. Drew & Heritage 1992) . As for test interactions, goals and tasks may be, for example, the production of assessable talk and the completion of various tasks, such as interview questions and responses, descriptive or narrative tasks. Institutional roles at play in oral proficiency tests, enacted in participants' conduct, can, for example, be 'examiner', 'teacher', or 'test-taker'. The ways in which test participants manage such specific constraints in their talk and social action, and the possible relationship between particular patterns of conduct and assessment, spurred scholarly interest in the testing context per se, and a number of important volumes and papers with an explicit aim to offer new tools for validating oral language tests and assessment, or for understanding the nature of social interaction in testing contexts, followed around the millennium (e.g.,     BIB001 . McNamara (2011: 435) argues that the 'distinctive character of language testing lies in its combination of two primary fields of expertise: applied linguistics and measurement'. The authors of this review enter the field of language testing from applied linguistics. In line with others (e.g., BIB003  ), the term second language (L2) is used in reference to any language -second, third, foreign, etc. -other than the learner's first language. The focus is on L2 oral proficiency testing, in particular on the social context of testing, that is, on discourse and social interaction in tests. Also, this review takes a specific interest in developments in the field from 2004 and onwards and aims to provide a systematic overview of recent work. Two questions guided the review: • What is the state of the field in research on discourse and social interaction in L2 oral proficiency tests published between 2004 and 2014? • What aspects of the testing context remain under-researched? Although the testing of L2 oral proficiency is closely interrelated with its assessment, the latter aspect is treated peripherally ( for a recent overview of L2 assessment, see BIB005 ; here, tests are viewed as a subset of assessment, which is in line with, for example, BIB002 . As the present review is focused on empirical studies on discourse and social interaction in L2 speaking tests, empirical studies on semi-direct tests (such as TOEFL Speaking, see, e.g., BIB006 or machine-controlled oral proficiency tests (such as the Versant automated oral proficiency test, see, e.g.,  BIB004 are beyond the scope of the review, even though such studies constitute a fairly large portion of current L2 oral proficiency testing. Conversely, several well-known tests of language proficiency do include face-to-face components and thus fall within the scope of the review, for example, the Cambridge ESOL tests (including the IELTS Speaking Test) and the Cambridge First Certificate of English (FCE), as well as various national tests of a second or foreign language. In sum, only studies with authentic face-to-face interaction as research data are included in this review. SEARCH CRITERIA AND DELIMITATIONS Initial searches revealed that empirical studies of oral language testing are spread across a number of publication venues, including second-language acquisition (SLA) journals, testing-focused publications, and discourse-oriented journals and edited books. Therefore, in the systematic search for empirical studies in the 2004-2014 time frame, the following three-step strategy was adopted: (1) Considering the fact that studies of L2 oral proficiency testing are rooted in different fields, such as discourse/social interaction and applied linguistics, all issues of 15 top journals, representative of the scholarly fields concerned, were selected and trawled through. Any paper involving L2 oral proficiency testing was examined and included in the review on a relevance basis bearing the focus on discourse and social interaction in oral proficiency tests in mind (see Appendix 1 for a list of initial hits in the selected journals). (2) For additional hits in other peer-reviewed scholarly journals, the databases Modern Language Association/MLA, Education Resources Information Center/ERIC, and Linguistics and Language Behavior Abstracts/LLBA were searched. Search strings applied were combinations of the following keywords: 'oral proficiency' + 'test*', 'speaking test', 'test* interaction', 'test conversation', 'OPI', and 'oral proficiency interview'. Only works published in English language peer-reviewed journals were included. (3) In addition, library catalogues and Google Scholar were searched for monographs and edited collections on language testing. Relevant research reports from language testing organizations were also scanned for, as well as The Language Testing and Evaluation series. However, the review makes no claims as to having captured all publications on L2 oral proficiency testing in the given period. In sum, the review includes original research papers published in selected peer-reviewed journals, empirical studies published in other peer-reviewed journals located through database searches, empirical studies published in edited books, and research reports that include empirical studies published in well-known book series or in research reports from various language testing organizations.