Read an Excerpt
CHAPTER 1
Beyond Hype, Hyperbole, Myths, and Paradoxes: Scaling Up Participatory Learning and Assessment in a Big Open Online Course
DANIEL T. HICKEY AND SURAJ L. UTTAMCHANDANI
Most readers of this volume are likely familiar with the distinctive history of massive open online courses (MOOCs). Their rapid expansion contrasts with the more steady expansion of higher-education technologies in prior decades, punctuated by small bursts around the advent of computers, personal computers, multimedia computers, and the Internet. The pace of change quickened around the turn of the century with the open education movement that laid some groundwork for the modern MOOC. The acronym itself was coined in 2008 for an open course on "connectivist" learning offered by George Siemens and Stephen Downes. MOOCs exploded in 2011–12 with Udacity, Coursera, edX, and others, suddenly enrolling tens of thousands of students around the world in free courses designed around the instruction of prominent academics. This outpouring of attention, investment, and learners was unprecedented in higher education.
Even as the New York Times dubbed 2012 "The Year of the MOOC" (Pappano 2012), the backlash against MOOCs was already underway. Many observed that the streaming videos and quizzes that dominated the newer MOOCs represented relatively shallow ways of interacting with content (e.g., Kays 2012; Marks 2012; Pope 2012). The acronym "xMOOC" (variously for eXtended or eXtension) was introduced to distinguish these newer offerings from the earlier networked and interactive courses advanced by Siemens and Downes, which quickly came to be called "cMOOCs" in response. Some observers had already commented on the difficulty of connecting with other learners in the cMOOCs (Mackness, Mack, and Williams 2010). It turned out that supporting social interaction in the xMOOCs was proving much harder. An effort to include more interaction and group projects in a Coursera course on online learning was widely cited for going "laughably awry" (Oremus 2013, 1). A study found that engagement in Coursera discussion forums declined significantly over time among completers, and that instructor involvement actually worsened participation (Brinton et al. 2014). While the "hype and hyperbole" over MOOCs continued to pour forth (Billsberry 2013, 739), John Daniel, an influential leader in the open-learning movement, captured the widespread concerns by summarizing the "myths and paradoxes" of xMOOCs (Daniel 2012). These included number of students taught (but single-digit completion rates), value (dubious certificates of completion), purpose (disregard for outcomes and focus on posturing and profits rather than spreading learning), pedagogy (essentially behaviorism), access (mostly serving elites), and risks (MOOCs as degree mills).
Yet the rapid expansion of MOOCs also prompted significant scholarly consideration of "learning at scale." In 2013, the National Science Foundation organized workshops on the topic (Fisher and Fox 2013), while the Bill and Melinda Gates Foundation established the MOOC Research Initiative that supported in-depth investigations for over twenty courses (Gasevic et al. 2014). In 2014, the Association for Computing Machinery initiated the annual Learning@Scale conference, and the number of empirical studies of MOOCs grew rapidly (e.g., Ebben and Murphy 2014; Papamitsiou and Economides 2014). As nicely detailed in the other eighteen chapters in this volume, innovators began pushing the boundaries of the xMOOC platforms and created new cMOOC formats, resulting in many promising formats to support more ambitious forms of learning at scale.
This chapter summarizes one ongoing effort that is intended to inform the entire range of efforts to scale open learning. Like others in this volume, we aimed to move beyond the current rhetoric of MOOCs. More specifically, this effort is part of a broader program of research that is attempting to transcend a forty-year-old debate over instructionist versus constructivist approaches to instruction. Instructionist approaches are rooted in a more "associationist" perspective on learning (e.g., Anderson 1990; Gagné 1985) which assumes that higher order knowledge can and should be broken down into smaller elements that can be individually learned, mastered, and assessed. This perspective is explicitly manifested in artificially intelligent tutors, like those associated with Carnegie Mellon University (Koedinger and Corbett 2006) and competency-based education (Bramante and Colby 2012) and more implicitly manifested in the xMOOCs. The obvious advantage of associationist approaches is that they make little or no demand on instructors, and scale readily. In contrast, constructivist approaches are rooted in more "rationalist" views of learning (e.g., Glaser 1984) that emphasize the construction of higher-level conceptual schema that learners create to make sense of the world (rather than by assembling numerous smaller associations). While there are many variants, constructivist approaches are more open-ended and inquiry-oriented than instructionist approaches. This means that these approaches can support plenty of engagement around more fundamental disciplinary concepts and meaningful social engagement around those concepts. But doing so requires patient instructors with sufficient understanding of both the particular discipline and how knowledge develops in the discipline — the so-called PCK, or pedagogical content knowledge, popularized by Lee Shulman in the 1980s (Shulman 1986).
Constructivist approaches such as problem-based learning have been explored extensively in conventional online contexts (e.g., Kanuka and Anderson 2007). However, as exemplified by the case of the Coursera meltdown mentioned above, constructivist approaches can be very difficult to scale. This is because both the people who design a course and then the instructors and facilitators who teach that course need a lot of TPCK (technological pedagogical content knowledge; Koehler and Mishra 2008) concerning the way that disciplinary knowledge can optimally unfold within the particular technology. In many settings this knowledge will be in very short supply, very expensive, or both. The challenge of scaling constructivist learning becomes particularly apparent when it comes to assessing student learning; even knowledgeable instructors are hard pressed to evaluate learner-generated artifacts or student performance efficiently and reliably, and most constructivist assessment practices are exceedingly difficult to automate with computers. Furthermore, assessing constructivist learning with the multiple-choice and short-answer formats associated with instructionist approaches are likely to miss the most important outcomes. For these reasons, the tensions that follow from antithetical assumptions behind constructivist and instructionist approaches are certainly exacerbated in most efforts to scale learning.
The new course described in this chapter attempted to transcend these tensions and produce new practices for scaling learning by drawing on two related sets of contemporary insights. The first are newer "participatory" approaches that can harness the advantages and minimize the disadvantages of both instructionist and constructivist approaches. The new course described in this chapter emerged from prior efforts to create this kind of synthesis in conventional online courses focusing on conventional course content. The second set of insights comes from learning theorists including Siemens, who set out to exploit the unique nature of digital knowledge networks and interest-driven social networking.
Participatory Learning and Assessment (PLA)
The design of this new course was rooted in an extended program of design-based research of educational multimedia (e.g., Hickey, Taasoobshirazi, and Cross 2012; Hickey and Zuiker 2012), educational video games (e.g., Barab et. al. 2007; Hickey, Ingram-Goble, and Jameson 2009), and secondary language arts instruction (Hickey, McWilliams, and Honeyford 2011). These studies used newer "situative" theories of knowing and learning (Brown, Collins, and Duguid 1989; Lave and Wenger 1991) to uncover new solutions to enduring challenges concerning assessment, feedback, grading, and accountability in technology-rich learning environments. What distinguishes situative theories from the prior theories is that they assume that knowledge primarily resides in the social and cultural practices of knowledgeable humans. This means that learning occurs when humans participate meaningfully in those practices (hence the label "participatory"). Contrary to some characterizations (e.g., Anderson, Reder, and Simon 1996), situative theories do not deny individual knowledge or ignore individual learning. Rather, situative theories assume that individual knowing and learning are "special cases" (i.e., secondary representations) of, primarily, social learning, and that the social, cultural, and technological contexts where knowledge is learned and used is a fundamental aspect of the individual knowledge (Greeno 1998). These assumptions lead to a much broader view of "learning" than the individually oriented instructionist or constructivist perspectives. In addition to the more familiar acquisition of knowledge and skills by individuals, situative theories see learning in moment-to-moment interactions (between learners, materials, and other learners), evolving practices within a cohort of learners (such as a new pattern of interaction in a particular discussion forum), and even broader long-term cultural shifts (such as the way that universities are learning to accommodate MOOCs).
Rather than attempting to prove or demonstrate the situated nature of learning, the prior design studies insistently searched for new approaches to assessing learning given these core situative assumptions about learning. For example, these studies embraced a much broader view of "assessment" that saw assessment taking place wherever learning (broadly construed) was occurring. This in turn pushed aside the conventional distinction between formative assessment (in support of new learning) and summative assessment (of prior learning), leading to focus on the actual functions of assessment (intended and unintended) rather than just the intended summative or formative purposes. The assessment framework that emerged from these studies was used to "align" learning across (a) informal socially interactive activities, (b) semiformal classroom assessments, and (c) formal achievement tests. As illustrated below, this alignment is accomplished by "balancing" the formative and summative functions of assessment within each of these "levels." These several multiyear design studies resulted in a number of instruction-assessment "ecosystems" that supported remarkable levels of individual and social engagement with course knowledge. Most importantly, the inclusion of high-quality classroom assessments and rigorous achievement tests showed that such engagement could consistently influence the knowledge of individuals and the achievement of groups without ever "teaching to the test."
The prior program of research referenced above expanded into conventional online courses taught by the first author and others around 2010. The need to organize the findings from the prior research to accomplish this expansion resulted in a general set of design principles. Reflecting a continued focus on participation in sociocultural practices, the larger framework that these principles formed was deemed Participatory Learning and Assessment (PLA). Course features for enacting these principles in the Sakai learning management system were refined in two online graduate-level education courses over several years (Hickey and Rehak 2013). The strategies for organizing the way that students interacted with course content, one another, and the instructor drew significant inspiration from three strands of research that extended situative theories of learning into the era of digital knowledge networks. Two of these were Henry Jenkins's (2009) notion of online "participatory culture" and studies by Ito et al. (2009) of the way young people "geek out" in interest-driven social networks. While both courses involved conventional textbooks, external open educational resources were gradually incorporated over time. Drawing from the notions of connectivist learning advanced by Siemens (2005) and by Downes (2006), the strategies that emerged for supporting disciplinary interactions with those resources emphasized acknowledging a diversity of opinions, making connections within and between disciplines and networks, and identifying current learning resources.
What distinguishes the PLA framework that emerged from these two courses were online strategies for delivering useful evidence that could be used to enhance participation, individual knowledge, and group achievement, without undermining any of them, and without compromising that evidence for making claims about the resulting knowledge or achievement. The next section describes how this framework was used to scale up one of these two courses, Assessment in Schools. The approach to this scaling reflected the concern that the rapid scaling of the xMOOCs had "locked in" the narrow instructional approach with which they started. Hence the new course was capped at five hundred learners to allow the gradual development of more interactive features. This big (rather than massive) course was deemed a "BOOC" and the course was called the Assessment BOOC.
Research and Development Context
After developing the Course Builder platform for a course titled Power Searching with Google, Google released an open-source version and began promoting it for wide usage. The course and consequently the platform featured streaming videos and quizzes, much like the xMOOCs. In contrast to more comprehensive learning management systems, Course Builder was designed to support individual courses and to be easily modifiable. Presumably reflecting concerns about limited interactivity in most MOOCs, Google offered grants to faculty for developing MOOCs that were "more interactive than typical MOOCs." The first author was awarded one of these grants. With university consent, a small team was assembled to develop and promote the Assessment BOOC.
This new course was first delivered as a twelve-week open course offered to the first cohort of students in fall 2013. It was promoted widely using Google and Facebook and 460 people eventually registered. The first assignment was completed by 160 participants, and 60 ultimately completed the course, including 8 students enrolled in a three-credit, graduate-level section. The Assessment BOOC was taught a second time in summer 2014, when some of the new features were automated, and streaming videos and open-ended self-assessments were added. Because most of the energy in 2014 was committed to recording videos and automating features, the course was not as widely promoted, and the instructor and teaching assistant had very limited interaction with individual students. Of the 187 registrants, 76 completed the first assignment and 22 completed the course, including 12 credential students. The course was then refined to allow a self-paced version to be offered in 2015, with little or no direct instructor involvement for the open students.
The following discussion of the BOOC course features is organized around the five PLA design principles. The extensive evidence of student engagement and learning left behind in both courses is currently being analyzed. Some of this evidence of engagement and learning in both courses, as well as details of how each feature was scaled, has already emerged from the peer review process (Hickey, Quick, and Shen 2015). This chapter focuses on how these features embodied the five PLA principles and introduces new theoretical refinements regarding the different kinds of interaction associated with each.
PLA Design Principles, Course Design, and Course Features
The five PLA design principles coordinate activity across different kinds of interactions that support different kinds of learning. Drawing on Hall and Rubin's (1989) study of situated learning in mathematics classrooms, the principles distinguish between interactions that are public (presented to every member of the class and potentially beyond), local (in public but between specific peers and/or the instructor), or private (between individuals). A fourth kind of interaction, discreet (i.e., unobtrusive), was added to highlight the core PLA assumption that conventional achievement tests should be used judiciously and inconspicuously.
The PLA principles and features draw inspiration from Engle and Conant's (2002) notions of productive disciplinary engagement (PDE). Engle and Conant pointed out that engagement that is disciplinary involves both the declarative knowledge of the discipline as well as the social and cultural practices in which disciplinary experts engage. They further argued that disciplinary engagement that is productive generates numerous connections between that declarative knowledge and the learner's experiences engaging in disciplinary practices.
(Continues…)
Excerpted from "MOOCs And Their Afterlives"
by .
Copyright © 2017 The University of Chicago.
Excerpted by permission of The University of Chicago Press.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
