The Art of Non-conversation: A Reexamination of the Validity of the Oral Proficiency Interview

by Marysia Johnson

Overview

The Oral Proficiency Interview (OPI) is a widely accepted instrument for assessing second and foreign language ability. It is used by the Foreign Language Institute, the Defense Language Institute, Educational Testing Service, the American Council on the Teaching of Foreign Languages, and at many universities in the United States. The Art of Non-Conversation examines the components of speaking ability and asks whether the OPI is a valid instrument for assessing them.

Marysia Johnson applies the latest insights from discourse and conversational analysis to determine the nature of the OPI’s communicative speech event and investigate its construct validity within Messick’s definition of validity. She discusses models of speaking ability—several communicative competence models, an interactional competence model, and a model of spoken interaction based on Vygotsky’s sociocultural theory of learning. Finally she proposes a new model to test language proficiency drawn from sociocultural theory, one that considers language ability to be reflective of the sociocultural and institutional contexts in which the language has been acquired.

Product Details

ISBN-13:	9780300090024
Publisher:	Yale University Press
Publication date:	09/10/2001
Edition description:	New Edition
Pages:	256
Product dimensions:	5.50(w) x 8.25(h) x (d)

About the Author

Marysia Johnson is assistant professor in the department of English, Linguistics/TESL Program, Arizona State University.

Read an Excerpt

Chapter One

OVERVIEW OF THE BOOK

The purpose of this book is to provide answers to two fundamental questions. The first is a practical one, and it represents the main focus of the book: Is the Oral Proficiency Interview a valid instrument for assessing language speaking proficiency? The second is a theoretical one: What is speaking ability? (that is, speaking ability that exists independently of testing instruments).

    Currently, an interview is a common way of assessing speaking ability in a second/foreign language in the United States and around the world. In particular, the Oral Proficiency Interview (OPI) is a very popular instrument for assessing second/foreign language speaking proficiency in such U.S. government institutions as the Foreign Language Institute and the Defense Language Institute. It is also used by nongovernmental institutions like the Educational Testing Service (ETS) and the American Council on the Teaching of Foreign Languages (ACTFL).

    In the OPI, which is based on the ACTFL, the ETS, and the Interagency Language Roundtable (ILR) scale level descriptions, the examinee converses face-to-face with one or two trained testers on a variety of topics for ten to thirty minutes. The elicited sample is then rated on a scale ranging from 0 (no functional ability) to 5 (proficiency equivalent to that of a well-educated native speaker). Plus ratings are also assigned when the examinee's speaking proficiency exhibits features of the next higher level. Overall, the examinee's level of speaking proficiency is measured on an 11-point rangescale.

    It is estimated that several thousand OPIs are administered each year. Professional careers, future job assignments, pay increases, and entrance to or exit from college language programs frequently depend on the rating obtained in an OPI.

    The question remains, however, whether the interview format represents the most appropriate and desirable method of assessing speaking ability in a second language. The proponents of the OPI claim that "a well-structured oral proficiency interview tests speaking ability in the real-life context—a conversation. It is almost by definition a valid measure of speaking ability" (ETS 1982: 13). However, one of the central characteristics of naturally occurring conversation—as Gumperz (1982), Sacks et al. (1974), and Wolfson (1976), among others, have pointed out—is that language users are largely unaware of how conversation is typically structured and managed. Much of everyday conversation is so deceptively familiar that people studying and testing language often overlook its fundamental characteristics. It is precisely on these grounds that van Lier (1989) has challenged the ETS's claim that it measures speaking ability in the context of a conversation. Van Lier (1984: 494) simply asks, "Is it really a conversation?"

    The answer to the questions, What kind of speech event is the OPI? and Is it really a conversation? is fundamental to our understanding of the construct validity of the OPI. Construct validity is essential for making inferences about the individual's ability on the basis of his/her test score. Establishing construct validity is the focus of major debate and concern among testing experts at present.

    Findings in discourse analysis (Schiffrin 1994) offer the testing community new and unique ways of investigating the construct validity of a test. That is, the application of various discourse techniques allows one to determine what it is that is measured by a test and not what is assumed or claimed to be measured. This methodology has been applied to the OPI data to determine the OPI's major discourse and linguistic features and to answer van Lier's question, "Is it really a conversation?"

    Discourse analysis provides theoretical rationales for conducting empirical investigation, specifically, for gathering empirical evidence as to the nature of the OPI communicative speech event. The empirical evidence in this book is based on both quantitative and qualitative findings from two studies: a discourse analysis study (chapter 5) and a semantic differential study of natives' perceptions of the OPI communicative speech event (chapter 6). These combined findings have led to the development of a prototypical model of the OPI communicative speech event (chapter 7).

    Chapters 8 and 9 investigate the issue, What is speaking ability? (that is, speaking ability that exists independently of testing instruments). For that purpose, several models of speaking ability are described and discussed: the communicative competence models proposed by Hymes (1972), Canale and Swain (1980), and Bachman (1990); the interactional competence mode! (Young 1998, 1999); and a model of spoken interaction based on Vygotsky's sociocultural theory.

    Although Vygotsky's sociocultural theory is a theory of learning and not performance, its foci on social interaction and on the individual's potential development offer a challenge to current thinking about testing speaking ability. That is, Vygotsky's emphasis on social interaction represents a direct challenge to the current testing view of interaction as cognitive and psycholinguistic. Hence, if a Vygotskian framework were integrated with the OPI, the shift of attention from the individual's current development to his/her potential development would require some revisions of the existing rating scales and of subcomponents included in various rating scales.

    In sum, the book deals with the issue of the construct validity of the OPI. The book presents some practical information as to the ways discourse analysis techniques can be utilized in determining construct validity. The OPI's validity is investigated within the framework of Messick's (1989: 13) definition of validity, which is viewed as "an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores."

    The book also proposes a new model of language ability—the Practical Oral Language Ability (POLA) based on Vygotsky's sociocultural theory that considers language ability to be reflective of sociocultural and institutional contexts in which the language has been acquired.

---

Excerpted from The Art of Nonconversation by Marysia Johnson. Copyright © 2001 by Yale University. Excerpted by permission.

Table of Contents

Acknowledgments	ix
Abbreviations	xi
Chapter 1. Overview of the Book	1
Chapter 2. The Genesis and Evolution of the OPI System	6
Chapter 3. A Critical Appraisal of the OPI	20
Chapter 4	Theoretical Bases for Investigating the OPI
Speech Event	41
Chapter 5. A Discourse Analysis Study of the OPI	76
Chapter 6	Native Speakers' Perceptions of the OPI
Speech Event	121
Chapter 7	A Prototypical Model of the OPI
Communicative Speech Event	140
Chapter 8	Communicative Competence versus
Interactional Competence	152
Chapter 9	The Practical Oral Language Ability: The
Application of Vygotsky's Sociocultural Theory to Language	181
Appendixes	207
Bibliography	213
Index	227