Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> This article provides a detailed examination of the oral proficiency interview and its underlying assumptions of validity and value. An attempt is made, by looking at the interview from the inside, to understand what OPIs are and what the participants in them do. The similarities and differences between interviews and conversations are examined, and some of the major problems of proficiency interviewing are illustrated and discussed. Finally, some specific proposals are made that may lead to more effective oral interviewing procedures. In this article I want to look at oral proficiency interviews (OPIs) as instances of conversation between people. Two questions are central to this discussion: (a) Are OPIs examples of conversational language use? and (b) Is conversational language use the appropriate (or the only, or the best) vehicle to evaluate oral proficiency? The tentative answer to the first question is a qualified no, the qualification being that an OPI can and may be designed to elicit conversational language use but frequently is not. The second question is far more difficult to answer: It addresses the central issue of the construct validity of oral tests in general. However, I propose that conversation is an appropriate vehicle for the all-around display of speaking ability in context. If the latter is a definition of oral proficiency, then I think we can be reasonably sure that conversation is the best vehicle for its evaluation. To address the questions I have posed, one must obtain an "insider's view" of the OPI. To do this, I have relied on three sources of data: (a) the analysis of several OPIs I have taken as an interviewee, (b) the study of transcripts and tapes of a variety of <s> BIB001 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract The simulated oral proficiency interview (SOPI) is a tape-mediated test designed to be a surrogate for the oral proficiency interview (OPI) in situations where a face-to-face interview is not possible or desirable. This article reviews research that sheds light on the comparability of the two tests. It begins with a brief description of the SOPI and continues to discuss the results of research on the reliability of the two tests, the agreement of scores obtained on the two tests, the comparability of the approach to testing, and the qualitative content of speech samples obtained via the two approaches. This article concludes with suggestions for further research. <s> BIB002 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> This article reports the results of a study of examinee attitudinal reactions to taking different formats of oral proficiency assessments across three languages: Spanish, Arabic, and Chinese. All 55 students in the study were administered both the tape-mediated Simulated Oral Proficiency Interview (SOPI) and a new Computerized Oral Proficiency Instrument (COPI). In addition, the 24 students participating in the Spanish study were administered the face-to-face A me ric an Counc il on the T ea ching of For e ign L angua ge s (ACTFL) Oral Proficiency Interview (OPI). Participants were graduate and undergraduate students taking language courses at their universities. The order of test administration was based on self- and teacher-assessed proficiency levels and experience in learning the target language. After each test, the learners completed a Likert questionnaire on six aspects of their attitudes towards and perceptions of that test. After finishing all tests and questionnaires, they were requested to compare the two technologymediated tests directly on those six aspects. This article presents the examinees' responses on those questionnaires, focusing primarily on differences between the two technology-mediated tests. It was found that the adaptive nature of the COPI allowed the difficulty level of the assessment task to be matched more appropriately to the proficiency level of the examinee. This led examinees, particularly those at the lower proficiency levels, to feel the COPI was less difficult than the SOPI. In most other aspects, the two tests were rated similarly. For the Spanish students, the two technology-mediated tests were rated similarly to the OPI, though the OPI appeared to them to be a better measure of real-life speaking skills. How do examinees react to an adaptive, computer-administered oral proficiency assessment? This article reports on examinee perspectives towards a prototype of the Computerized Oral Proficiency Instrument (COPI). The COPI was developed by the Center for Applied Linguistics (CAL; http://www.cal.org) under a research grant from the International Research and Studies Program of the United States Department of Education. Developed for Spanish, Arabic, and Chinese, the COPI is an adaptation of the tape-mediated Simulated Oral Proficiency Interview (SOPI), originally developed at CAL in 1985 (Clark, 1988). The primary goal of the COPI research project was to examine the effects of using multime dia c omputer tec hnology to e nha nc e the SO PI by giving examine es more contr ol over va rious a spe cts of the testing situa tion a nd by inc r ea sing ra ter s' ef fic ie nc y in scoring the test. The projec t c ompa r ed e xamine e per for ma nc e a nd af fe c t, a nd ra ter e f ficie nc y a nd a f fe ct, betw ee n the CO PI a nd the A me r ic an Counc il on the T ea ching of For e ign L angua ge s ( ACTFL ; http://www.actfl.org) O ra l Prof ic ie ncy I nte rview ( O PI ) f or Spa nish, a nd the COPI a nd SO PI for Spa nish, Chinese , and A ra bic . T his a rtic le compa re s e xa minee r ea ctions to the COPI , the SOPI , and the OPI. <s> BIB003 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> The empirical study reported here investigated predictors of oral proficiency gain to the Advanced level resulting from immersion study abroad. The data consisted of 22 preimmersion Oral Proficiency Interviews (OPIs) of American students who participated in semester immersion programs in Moscow and St. Petersburg, and received a score of Intermediate High on the preimmersion OPI. On the basis of the students’ postprogram proficiency scores, the participants were grouped as learners who crossed the Advanced-level threshold (gainers) and as those who did not cross the threshold (nullgainers). This study used linguistic predictor variables identified in previous research (Brecht, Davidson, & Ginsberg, 1993) and introduced a new set of metalinguistic variables. An empirically based model constructed for the study included five variables that proved to be the best predictors for the Advanced-level threshold gain: linguistic (grammar, vocabulary, accuracy) and metalinguistic (self-corrected errors and sentence repair). The resulting findings yielded a robust model for predicting language outcomes of study abroad. <s> BIB004 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract: This study investigates the role of social interaction in language gain among study abroad students in France. Using the ACTFL Oral Proficiency Interview (OPI), the Can-Do self-assessment scale (Clark, 1981), a revised version of the Language Contact Profile (LCP; Freed, Dewey, Segalowitz, & Halter, 2001), and preand postdeparture questionnaires, we examine gains in oral proficiency as related to language contact in the study abroad environment. This research shows that language gain is possible during a semester-long study abroad program. It does not uphold the common belief that living situation and contact with authentic media differentiate students who improve from those who do not. Looking at the background of students (age, gender, grade point average, etc.), it reveals that only prior coursework in French correlates strongly with gains in proficiency once abroad. In its most surprising finding, this study suggests that speaking French with Americans may impede proficiency development. <s> BIB005 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract: This study measures and analyzes improvements in students' oral proficiency during a study abroad semester in Germany. Oral proficiency interviews were conducted with participants before and after the program during three consecutive years. All interviews were assigned official ratings. Usage of two specific grammar structures associated with the Advanced level was also tracked. Before the program, most students were at Intermediate Low, even those who had completed 5th-semester German courses. Although 80% completed the program with improved proficiency ratings, very few reached the Advanced level. Internal analysis of speech samples did reveal that the post-program Intermediate Mid performances were much closer to the Advanced level than pre-program performances with the same rating. The study concludes with an assessment of curricular implications for post-program courses. Key words: German, grammar and proficiency levels, OPI as research tool, oral proficiency development, study abroad program assessment Introduction Language educators and program administrators agree that studying abroad is beneficial, and perhaps even essential, for students who want to improve their oral proficiency in a foreign language. This judgment is often based on intuition or subjective observation, such as informal conversations with students before and after their study abroad experiences, or comparisons of student performance in courses. Still, reliable studies that actually seek to measure and analyze the suspected linguistic gains made during study abroad semesters are relatively rare. Those that exist provide inconsistent results. Attaining a specific proficiency level, as defined by the ACTFL Proficiency Guidelines for Speaking (Breiner-Sanders, Lowe, Miles, & Swender, 2000), can be crucial for the future careers of language learners. The question of whether students actually reach a measurably higher level of proficiency as a result of studying abroad for a semester is therefore an important one. For instance, the National Council for the Accreditation of Teacher Education Programs (NCATE) Standards expect teacher candidates to demonstrate oral proficiency at the Advanced Low level. The teacher certification boards of 16 states currently require their candidates to prove that they have reached an Advanced Low or Intermediate High rating by taking an oral proficiency interview (OPI) conducted by the ACTFL testing office, Language Testing International (LTI). Other states permit individual institutions to decide how they will determine that their teaching candidates have met the required standard.1 Any attempt to assess the role of a study abroad semester in the development of oral proficiency will need to answer the following questions: * At what proficiency level do students enter the program? * Are students returning from their semesters abroad at a higher level of proficiency? * Do students' initial proficiency levels influence the improvement rate during the semester abroad? * What are reasonable expectations for improvement? ACTFL Guidelines The ACTFL proficiency guidelines and the OPI have practical implications for language curricula: They are nationally recognized measures of functional ability in a language. The ACTFL rating scale comprises four major levels (Novice, Intermediate, Advanced, and Superior) that are delineated according to a hierarchy of global tasks. The Novice, Intermediate, and Advanced levels are further divided into sublevels: Threshold performances are indicated as Low and High; solid performances, in both quality and quantity, are indicated as Mid. The ACTFL rating scale therefore measures 10 different proficiency categories. It is often depicted as an inverted pyramid to represent the exponential increase of facility with language throughout and within the major levels (see Figure 1). It is crucial to point out that the ACTFL guidelines define the High sublevel as a "fall from the next higher level rather than a strong ability demonstrated at the general level. … <s> BIB006 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students' performance in an oral proficiency test and the social practice of conducting 'test talk'. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that 'doing-being a successful task manager' is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation. <s> BIB007 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract: Within second language education, concern over teachers' content knowledge has typically manifested itself as concern over the teacher's target language proficiency. In increasing numbers, teacher preparation programs are turning to ACTFL's Oral Proficiency Interview (OPI) for measurement of this proficiency and using OPI ratings as high-stakes tests of readiness for teaching. This study reports the results of a survey of 734 teacher candidates on how they prepare for this assessment, both individually and in their training programs. The results of the survey reveal significant differences in study strategies between candidates who score Advanced or above and those who score below that targeted range. The more successful candidates use more discourse- and culturally rich sources for language input and spend considerably more time using the target language outside of course requirements. Candidates provide useful guidance to teacher educators on what helps and what hurts their efforts to be successful. Key words: assessment, oral proficiency, preservice teacher preparation, standards, survey research Background Teacher quality and teachers' content knowledge have remained top priorities over the past several decades in the discussions of educational policy and reform (e.g., Carnegie Corporation, 1986; Holmes Group, 1986; Smith, Desimone, & Ueno, 2005; Wenglinsky, 2002). Most generally agree that a key, if not the key to better learning for students is better teaching (Darling-Hammond, 2000), and clearly teachers' content knowledge, while elusive in its categorization and its measurement, is integral to teaching. The federal No Child Left Behind Act of 2001 (NCLB) calls for a "highly qualified" teacher in every classroom, which is defined as a teacher with full certification, a college degree, and demonstrated content knowledge in the subject being taught. The meaning of this final attribute, "content knowledge," is a source of considerable debate. It is surprisingly unclear what rank content knowledge plays in the making of a good teacher, yet all can agree that the French teacher who cannot speak French will not be a successful teacher of French. Grossman, Wilson, and Shulman (1989) defined content knowledge as "factual information, organizing principles, central concepts" and therefore as a subset of the broader construct of subject matter knowledge (p. 27). For the world language teacher, content knowledge may include analyses of works of art, themes of literature and film, grammatical structures, and cultural events. However, within second language (L2) education, concern over teachers' content knowledge has typically surfaced as concern over the teacher's target language proficiency. For better or for worse, testing of teachers' knowledge on a variety of fronts is now a common feature of the educational landscape. Fortunately, foreign language educators have been leaders in the arena of assessment of complex content knowledge by virtue of their early work on the Oral Proficiency Interview (OPI), an integrative test of target language speaking. The first oral proficiency guidelines were published in 1982, and the OPI's proficiency scales were developed at that time out of the Foreign Service Institute levels of oral proficiency (Clark & Clifford, 1988) and revised in 1999 (Breiner-Sanders, Lowe, Miles, & Swender, 2000). The ACTFL's current OPI is a tape-recorded interview conducted by a trained examiner that consists of five stages: a warm-up, level checks, probes, role-play, and wind-down and is designed to efficiently elicit a ratable speech sample.1 Test takers take the OPI one-on-one either in person or by telephone, and Swender (2003) found that scores did not vary significantly between those two formats. ACTFL joined forces with the National Council for Accreditation of Teacher Education (NCATE) in 2002 to participate in accreditation reviews of teacher education programs. … <s> BIB008 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract: The renewed national focus on teacher quality and effectiveness has resulted in more rigorous standards that describe the knowledge and skills required of teacher candidates across all disciplines. In the area of foreign languages, three sets of professional standards address the oral proficiency of teachers in the target languages they teach across the career continuum. For teacher candidates, the ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers (2002) establish minimum oral proficiency levels based on the ACTFL Proficiency Guidelines-Speaking (2012). Utilizing ACTFL Oral Proficiency Interview (OPI) data, this study examines to what extent candidates are attaining the ACTFL/NCATE Oral Proficiency Standard of Advanced Low in most languages or Intermediate High in Arabic, Chinese, Japanese, and Korean. Findings indicate that 54.8% of candidates attained the required standard between 2006 and 2012 and that significant differences emerged for language, year tested, and university program results. Further research that takes into account additional contextual information about candidates and programs will inform continuing professional dialogue about the oral proficiency of teacher candidates entering the profession.Key words: Advanced Low, NCATE, proficiency, Standards, teacher preparationIntroductionIn the field of education, the past decade has seen an increasing emphasis on teacher accountability for student learning, at least partially in response to the national de- mand for "highly qualified" teachers pro- posed through the No Child Left Behind Act (NCLB) (2001). The need to identify teachers of high quality has resulted in more rigorous standards related to the preparation of teachers across disciplines, particularly with respect to the develop- ment of their content knowledge and skills (see, for example, NCATE, 2008). This national endeavor to develop teacher stand- ards has prompted widespread discussion and research exploring the type of content- specific knowledge teachers should pos- sess. This discussion has been particularly prevalent in the field of mathematics (Ball, Thames, & Phelps, 2008; Kleickmann et al., 2013), but it has also occurred in most other fields, including science (Lederman, 1999) and physical education (Siedentop, 2002).Although dialogue concerning the content knowledge and skills of foreign language teachers dates back to the late 1980s (ACTFL, 1988), more recently three sets of teacher standards have sparked a renewed discussion regarding the content knowledge and skills that foreign language teachers should possess as they move across the career continuum from teacher candidates to beginning teachers to accom- plished teachers. In 2002, NCATE ap- proved the ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers (ACTFL, 2002) (here- after referred to as the ACTFL/NCATE Program Standards), which define the expectations for teacher candidates who complete a teacher preparation program and earn teacher certification. NCATE, which is recognized by the U.S. Department of Education and the Council for Higher Education as an accrediting agency for schools, colleges, and departments of education, determines which colleges of education meet rigorous national standards for teacher preparation. Foreign language teacher preparation programs that undergo NCATE review and seek national recogni- tion by ACTFL/NCATE must provide evidence that their programs are preparing candidates who can demonstrate the per- formance described in the standards. In conjunction with the development of the ACTFL/NCATE Program Standards, in 2002, the Interstate New Teacher Assess- ment and Support Consortium (INTASC1), with support from ACTFL, designed its Model Licensing Standards for Beginning Foreign Language Teachers, which may also be used by participating states to license teachers and induct them into the profession. To complete the career contin- uum, in 2001, the National Board for Professional Teaching Standards (NBPTS) created the World Languages Other Than English Standards, which describe the knowledge and skills necessary for accom- plished language teachers who elect to earn National Board certification; these stand- ards were updated in 2010 and renamed the World Languages Standards (NBPTS, 2001, 2010). … <s> BIB009 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> This study investigates whether raters' knowledge of test takers' first language (L1) affects how the raters orient themselves to the task of rating oral speech. The authors qualitatively investigated the effects of accent familiarity on raters' score assignment processes. Twenty-six trained raters with a second language of Mandarin Chinese, Korean, or Spanish rated English speech samples from 72 English language learners from three L1 backgrounds (Mandarin Chinese, Korean, and Spanish). Afterward, raters watched videos of themselves rating and discussed their rating processes. Analyses of data, assisted by the qualitative analysis software package QSR NVivo 8, revealed that some raters attended to and were influenced by test-taker accent and L1. Raters who were heritage language learners discussed how their personal heritage status influenced how they thought about the speech samples from speakers whose L1 matched their heritage language. Although sensitivity to test-taker accents seemed to occur naturally in the rating process, findings suggest that when raters have learned or know, to varying degrees, the test takers' L1, they tend to orient themselves to the speech in a biased way, compromising test reliability. <s> BIB010 </s> Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing <s> CANDIDATE-EXAMINER TESTS <s> Abstract: The Oral Proficiency Interview (OPI) has been increasingly used in academia. However, while multiple studies have documented the growth in OPI implementation across the United States and the proficiency rates of its completers, few have focused specifically on foreign language teacher candidates, and even fewer have investigated the impact that this proficiency assessment may have on language teacher training programs. To better understand the impact of the OPI on foreign language teacher education programs and help guide programmatic decision making, a case study was conducted of one such program that recently implemented the OPI as part of its licensure requirements. The results confirmed earlier research with respect to expected proficiency outcomes of foreign language teacher candidates. The results also suggested that the OPI requirement did not negatively affect program enrollment, nor did teacher trainees negatively perceive the OPI requirement. Finally, the study provided evidence of the positive impact the OPI may have on a foreign language teacher education program. Recommended practices for implementing the OPI in teacher training programs and ways to support foreign language teacher candidates who must complete the assessment are discussed.Key words: case study, foreign/second language teacher preparation, language profi- ciency, native speaker/nonnative speaker status, teacher characteristicsIntroductionThe Oral Proficiency Interview (OPI) is a nationally recognized, standardized test that assesses an individual's oral communication skills in a foreign or target language. The OPI was developed in the 1980s by ACTFL and is administered through ACTFL'sofficial testing office, Language Testing International (LTI). The OPI is offered as an in-person or telephonic interview between the test taker and either a certified ACTFL tester who has un- dergone rigorous training in the evaluation process or a computer with which interview- ees interact with a digital avatar (Oral Profi- ciency Interview by Computer; OPIc). Regardless of the means of testing, a record- ing of the interview is analyzed by two cer- tified ACTFL raters who independently assess the candidate's speaking proficiency based on the descriptors in the ACTFL Pro- ficiency Guidelines (ACTFL, 2012), result- ing in 1 of 11 profi ciency level ratings ranging from Novice Low to Distinguished. While approximately 3,000 individuals completed the OPI in 1996, 27,875 OPIs or OPIcs were administered in 2011, and in 2012, that number surpassed 30,000.1Although individuals take the OPI or OPIc for a variety of different purposes, it is being used in increasing numbers in aca- demia for teacher certification purposes (Hammadou Sullivan, 2011; Malone & Montee, 2010). Currently, 23 states require aspiring foreign language teachers to com- plete an OPI. Sixteen of these states have adopted the ACTFL standard of Advanced Low or higher for teacher licensure in more commonly taught languages such as Spanish, French, and German, and the remaining seven have set the minimum standard at Intermediate High. With respect to less com- monly taught languages,2 all 23 states require a minimum score of Intermediate High.In addition to foreign language teacher certification programs in these 23 states, pro- grams seeking national recognition by ACTFL and the Council for the Accreditation of Educator Preparation3 must also demon- strate that their teacher candidates achieve a minimum passing score on the OPI of Ad- vanced Low for commonly taught languages and Intermediate High for less commonly taught languages (Pearson, Fonseca-Greber, & Foell, 2006). Programs whose teacher can- didates do not meet this minimum score as outlined in Standard 1a of the ACTFL Pro- gram Standards for the Preparation of Foreign Language Teachers (ACTFL, 2002) must of- fer planned remediation experiences in order to help those candidates enhance their target language speaking skills. … <s> BIB011
L2 speaking/OP tests date back to the early 20th century, but it was not until during the second world war that such tests shifted from having had a focus on dictation and pronunciation to a focus on the ability to perform/interact  . The three-part test format used then, picture description, sustained speech, and conversation ( for details, see  , was a precursor of the first published speaking test, the Foreign Service Institute (FSI) Oral Proficiency Interview, known as the OPI  . The three-part design of OP tests is still frequently used -lower-level abilities being tested first -such as moving from monologue via dialogue to interaction in paired tests (cf. BIB007 . The OPI (also referred to as Language Proficiency Interview, LPI, see Young & He 1998) is a standardized, global assessment of OP and involves an examiner and a test-taker (audio-recorded). The candidate's output is assessed by the examiner and a second rater against pre-set criteria for 10 levels, described in the aforementioned ACTFL guidelines (ACTFL 2012). Similar to the English tests mentioned above, the OPI is often required for various reasons, for example, for teacher candidates  BIB009 BIB011 ) and study-abroad programs (see, e.g.,   BIB004 BIB006 BIB005  . There are also so-called simulated OPIs (SOPIs) (see, e.g.,  , in which the test-taker is given computer-or tape-mediated prompts along with a printed test booklet and responds by speaking into a microphone BIB003 ; for a review of research comparing OPIs and SOPIs, see BIB002 . Last, the OPI has been criticized for being behavioristic  , for the way the examiner is implicated in the construction of test-takers' output  and for not resembling natural conversation due to the asymmetric relationship between the examiner and the test-taker  BIB001 . In addition, findings in BIB010 suggest that when examiners/raters are familiar with the test-takers' first language, they tend to orient to the learners' talk in a biased way, which compromises test reliability; moreover, different kinds of preparation for OPI affect test scores BIB008 . Others claim the OPI yields a valuable and independent measure of OP  ) resembling real-life speaking skills BIB003 .