Understanding Language Testing

by Dan Douglas

Overview

Understanding Language Testing presents an introduction to language tests and the process of test development that starts at the very beginning. Assuming no knowledge of the field, the book promotes a practical understanding of language testing using examples from a variety of languages.

While grounded on solid theoretical principles, the book focuses on fostering a true understanding of the various uses of language tests and the process of test development, scoring test performance, analyzing and interpreting test results, and above all, using tests as ethically and fairly as possible, and feel positive about their language learning.

Each chapter includes a summary, suggestions for further reading, and exercises. As such this is the ideal book for both beginning students of linguistics and language education, or anyone in a related discipline looking for a first introduction to language testing.

Product Details

ISBN-13:	9780340983430
Publisher:	Taylor & Francis
Publication date:	12/25/2009
Series:	Understanding Language
Edition description:	New Edition
Pages:	168
Product dimensions:	6.10(w) x 9.10(h) x 0.40(d)

About the Author

Dan Douglas is Professor Emeritus of linguistics, language testing, and second language acquisition, Iowa State University.

Table of Contents

Acknowledgements x

Preface xi

1 Introduction: the rubber ruler 1

1.1 Why test language learning? 1

1.2 What is a language test? 2

1.2.1 What are the properties of measuring devices? 2

1.3 The rubber ruler 3

1.4 Tests, measurement and evaluation 5

1.4.1 Evaluation without measurement 5

1.4.2 Measurement without a test 5

1.4.3 Tests 6

1.5 Ethical uses of language tests 9

1.5.1 Reliability 10

1.5.2 Validity 10

Further reading 13

Exercises 13

2 Measuring language ability and making decisions 17

2.1 Measuring language ability 17

2.2 Making decisions about learners, teachers, programmes, and policies 19

2.3 Contexts of language use 20

2.3.1 Contextual features 21

2.3.2 Authenticity 24

2.4 Making valid interpretations of test performance 26

2.4.1 Consistency of measurement 26

2.4.2 Validity: evidence for interpretations of test performance 29

2.5 Conclusion: bias for best 34

Further reading 35

Exercises 35

3 Test development 38

3.1 What do I need to test? Needs analysis 38

3.1.1 Define the purpose of the test 40

3.1.2 Conduct a preliminary investigation 40

3.1.3 Collect primary data 40

3.1.4 Collect secondary data 42

3.1.5 Analyse target language use task and language characteristics 44

3.2 How am I going to test language abilities? Turning target language use tasks into test tasks 48

3.2.1 Developing a test task 48

3.2.2 Developing a blueprint for the test 49

3.2.3 Options for test tasks 49

3.3 How am I going to give the test? Test administration 54

3.3.1 Test environment 54

3.3.2 Personnel 54

3.3.3 Procedures 55

3.3.4 Scoring 55

3.4 How can my computer assist me in test development? Computer-based tools 56

3.4.1 Hot Potatoes 57

3.4.2 Moodle 60

3.4.3 WebCT 60

3.5 Conclusion 63

Further reading 64

Exercises 64

4 Alternatives in assessment 67

4.1 Norm-referenced and criterion-referenced tests 67

4.2 Communicative language tests 69

4.3 'General' and 'specific purpose' language tests 70

4.4 Discrete-point and integrative tests 70

4.5 Formative and summative assessment 72

4.6 Alternative approaches to assessment 73

4.6.1 Conference assessments 74

4.6.2 Portfolio assessment 74

4.6.3 Self- and peer-assessments 75

4.6.4 Task-based and performance assessment 76

4.6.5 Dynamic assessment 79

4.6.6 Summary 80

4.7 Conclusion 80

Further reading 81

Exercises 81

5 By the numbers: a statistics mini-course 85

5.1 Introduction 85

5.2 Normal distribution 87

5.3 The average or mean 88

5.4 Standard deviation 90

5.4.1 Standard deviation as a unit of measurement 92

5.5 Correlation 93

5.6 Probability and statistical significance 97

5.7 The t-test of the difference between two averages 99

5.8 Analysis of variance 101

5.9 Reliability 104

5.9.1 Split-half method 105

5.9.2 Internal consistency method 106

5.9.3 Standard error of measurement 108

5.10 The reliability of human raters 110

5.11 Conclusion 111

Further reading 112

Exercises 113

6 Technology and language testing 115

6.1 Introduction 115

6.2 Issues in technology and language testing 116

6.2.1 Technology and test taker attitudes 117

6.2.2 Language performance and different media 117

6.2.3 Technology and the construct to be measured 118

6.2.4 Technology and assessment tasks 118

6.2.5 The limits of automated scoring 119

6.3 Technology and language task types 119

6.3.1 Listening tasks 119

6.3.2 Integrated listening and speaking tasks 122

6.3.3 Writing tasks 124

6.3.4 Reading tasks 125

6.4 The promise and threats of automated scoring 127

6.4.1 Examples of current automated scoring programs 127

6.4.2 Concerns about automated scoring 129

6.5 Test feedback and reporting 131

6.6 Online and computer-based resources for statistics 135

6.6.1 Microsoft® Excel 135

6.6.2 Online resources 137

6.7 Conclusion 139

Further reading 139

Exercises 140

Afterword: the rubber ruler revisited 144

References 146

Index 153

Editorial Reviews

'This is a crisp, focused, readable and stimulating book. Douglas covers a wide range of territory in concise yet challenging ways'.

/Fred Davidson, Professor of Linguistics, University of Illinois at Urbana-Champaign, USA

From the Publisher