A Selection of Research on the Use of MOOC Software and Computer-Supported Collaborative Learning strategies for Reflection and Peer Assessment in K-12 PBL Classrooms
Introduction
Project Based Learning has become a focus of my learning and teaching. I am particularly interested in the elements of reflective writing and peer assessment within the process of PBL, and how technology can be utilised to scaffold and maximise students’ cognition. After completing a Coursera course, I was intrigued by their peer assessment system and was surprised by its accuracy. I am interested as to how Coursera do their peer assessments and if I can use their strategies in the PBL investigations and research my Year 12 and 13 students will undertake this year. The research articles I have reviewed have been focussed on PBL and the use of CSCL to enhance individual and group knowledge.
The epistemological theories underlying PBL are cognitivism and constructivism, but also experiential learning and constructionism. In a group context of peer review/feedback and reciprocal teaching, social constructivist principles play a part in the learning of the individual and group; what are the successful interactions or barriers afforded by technology in the communication between the student author and their peer reviewers?
Vygotsky views practical activity and social interaction with peers as an effective way of developing skills and strategies; less competent children develop with help from more skilful peers - within the zone of proximal development. By providing appropriate assistance (scaffolding), students can achieve a task. The concept of peer review and reciprocal teaching is supported by Vygotsky’s theories.
Conclusion of annotations
It will be possible to use a Coursera style peer review to enhance student learning and improve self reflection. Due to the numbers of students involved in MOOCs, it is not possible to provide teacher feedback or have interventions, however the research suggests that teacher intervention is necessary for students to acquire the higher levels of knowledge construction when they work as a group in CSCL environments. Since traditional classrooms have manageable numbers of students, the blended learning made possible by CSCL environments should permit the teacher to intervene with support.
Any design of a Coursera style peer review for secondary school students should allow the reviewee to engage in discussion with the reviewer even if they are anonymous - this communication would be an extension of Coursera’s design - discussion as to the reviewer’s points to provide clarification will be necessary for the reviewee to improve.
Bibliographic Entries
Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT TPCK: Advances in technological pedagogical content knowledge (TPCK). Computers & Education, 52, 1.
The authors, from the Department of Education at the University of Cyprus, discuss a model of self, peer and expert assessment for the development of ICT-TPCK skills in pre-service teachers. The article describes the use of BlackBoard 6 to facilitate the process of assessment and discussion between the students and instructor. The students highlight the need for lectures before the development of their lessons, demonstrating the epistemological belief that new knowledge is constructed upon previous knowledge and understanding. Student lesson products became more sophisticated on further cycles of peer assessment and lesson development; an empirical investigation showed significant improved between the first and second design tasks after the repetitive assessment procedure.
The strategies used in this article reflect the suggestions highlighted by Brown, Collins & Duguid (1989) that cognitive apprenticeship and social interaction enhances learning.
It can be inferred from this article that teachers who have not completed a course in ICT-TPCK will struggle to take on the role of technology coach in the classroom.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated Cognition and the Culture of Learning. Educational Researcher, 18, 1, 32-42.
The authors, based at the Institute for Research on Learning, argue that knowledge is situated, being part a product of the activity being undertaken, and the culture in which it is developed and used. Referring to research from English and Mathematics classes, the authors provide examples of how the culture within which an activity is performed has an affect on the learning of the concepts. When activities are transferred to the classroom they become classroom tasks and part of the school culture and ultimately become ersatz - an inferior substitute. The authors go on to describe Cognitive Apprenticeship in the classroom and students must enter a community and its culture; learning is “a process of enculturation”. Through situated modelling, coaching and scaffolding strategies, teachers can promote learning by making their implied knowledge explicit, enabling students to discover or learn concepts for themselves.
In the context of PBL, this research highlights the need to generate truly authentic scenarios in which the students can learn. The application of CSCL and technology needs to reflect how scientists, lawyers, etc. actually use the technology for their work.
Glance, D. G., Forsey, M., & Riley, M. (2013). The pedagogical foundations of massive open online courses. First Monday, 18, 5.
The authors, Associate Professors at the University of Western Australia, discuss the various attributes of MOOCs and how pedagogy relates to these attributes. For Self and Peer-Assessment the authors refer to research by Lewin (2012) that indicated a high degree of correlation between five peer-assessments for a final exam and that of the teaching staff in a sociology course on Coursera. The authors further refer to research findings that refute and support the benefits of peer-assessment in learning but point out that there is more evidence in support. Peer-assessment is said to enhance a student’s self-learning and those students who correct their work improve dramatically.
Of most interest, the authors conclude that MOOCs may have value in campus settings as a form of flipped classroom or blended learning.
Gunersel, A. B., Simpson, N. J., Aufderheide, K. J., & Wang, L. (2008). Effectiveness of Calibrated Peer Review[TM] for Improving Writing and Critical Thinking Skills in Biology Undergraduate Students. Journal of the Scholarship of Teaching and Learning, 8(2), 25-37.
This study focuses on student development with Calibrated Peer Review (CPR), a web-based tool created to promote writing and critical thinking skills. Research questions focus on whether or not students showed improvement in writing and reviewing competency with repeated use of CPR in a senior-level biology course and whether the difference between higher performing and lower performing students decreased over time. Repeated analyses indicated that students showed improvement in writing skills and reviewer competency with repeated use of CPR. The difference between higher and lower performing students decreased over time in both writing skills and reviewer competency.
The observed improvements match the comments by Glance, Forsey & Riley (2013) in MOOCs.
Häkkinen, P. (2002). Internet-based learning environments for project-enhanced science learning. Journal of Computer Assisted Learning, 18(2), 233-234.
The author indicates a need for greater understanding of the role technology plays in the project-oriented curricula and student learning. The two person sample size of this study makes the reliability of the author’s conclusions weak, however the author does refer to several other studies as a comparison.
Huei-Tse, H., Kuo-En, C., & Yao-Ting, S. (2007). An Analysis of Peer Assessment Online Discussions within a Course that uses Project-based Learning. Interactive Learning Environments, 15, 3.
The authors, from the National Taiwan Normal University, Taipai, Taiwan, describe the process of students writing PBL projects, posting them to a forum and the peer assessment lacking teacher intervention that followed. Gunawardena, Lowe and Anderson’s (1997) Interaction Analysis Model (IAM) coding scheme was used to analyse the online discussions of 45 students. The authors compared their results with Jeong (2003) who also used the IAM model in peer assessed online discussions; both results found that students could not reach the in-depth knowledge construction of the higher levels of the IAM model P3 (negotiation of meaning), P4 (testing the proposed new knowledge against existing schema), or P5 (summarising or showing new knowledge construction). The 45 student sample size and comparison with other author’s data improves the reliability of the author’s conclusions.
The authors conclude that active teacher feedback is necessary to prompt students to engage in the higher levels of the IAM model, P3,4 and 5. Student attitudes to peer comments and the length of time for assessment may play a role in the depth of learning in the online discussion peer assessment.
This research highlights the need for teacher input within a technology mediated environment to support students in their ability to assess. The use of MOOC software without teacher input may not produce the higher level of knowledge construction required.
Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., & Koller, D. (2013). Tuned models of peer assessment in MOOCs. Retreived from: http://arxiv.org/abs/1307.2579v1
The authors, from Stanford University and Coursera, discuss their creation of algorithms that correct for grader biases and reliabilities in order to improve grading accuracy. The Coursera assessment model has students assess five anonymous assignments, one of which is a “ground truth” assignment that creates an assessment calibration. The results of the research suggest that the algorithms improve the percentage of accurate “true scores” to 95%.
While this research is not likely to be replicable by a teacher in a classroom, the concept of using calibration assessments is similar to scaffolding in preparation for peer assessment activities. The use of the grader’s accuracy in grading and subsequently assigning them to assess work with large variance in the grades assigned could be applied in the classroom.
Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21, 2, 79-90.
Gerry Stahl, a professor at Drexel University, describes how the paradigm of CSCL is based on anthropology, sociology, linguistics and communication science. Stahl refers to articles from Koschmann (2002), Hutchins (1996), Lave and Wenger (1991), and Suchman (1987) as he points out that the understanding of shared knowledge and group cognition is ambiguous. Stahl lists various options of what shared knowledge and group cognition might be understood by along the spectrum of individual (cognitivist) versus group (socio-cultural), and from Resnick’s (1991) shared cognition to Salomon’s (1993) distributed cognition. Stahl goes on to to describe how various theories may be attached to group knowledge construction, linking back to Vygotsky’s ZPD.
Stahl describes a study of K-12 math students where the study showed that group knowledge can be constructed in discourse and how analysis of that discourse can make the knowledge visible to the researchers. Stahl believes that focusing on discourse is a solution to the confusion between individual and group knowledge, and to the conceptual conflict about how there can be such a thing as group knowledge distinct from what is in the minds of individual group members.
Stahl does not present the data from his research in this paper; the paper is an argument for his ideas where he references other authors and articles of his own work.
With regard to the use of CSCL to enhance reflection, students would need to be engaging in actively solving their domain problems outside of the classroom and within the virtual learning environment.
Tsuei, M. (2011). Development of a peer-assisted learning strategy in computer-supported collaborative learning environments for elementary school students. British Journal of Educational Technology, 42, 2, 214-232.
Mengping Tsuei is a professor at the Graduate School of Educational Communications and Technology, National Taipei University of Education, Taipai. The author provides a literature review of the theories and research associated with CSCL, referring to Lave and Wenger, Hutchins, and Topping as Stahl does. The article reviews the methods and analyses used in the study of elementary children learning to write chinese characters, how they interact online as tutor and tutee, and what they write. The study highlights a significant improvement in the self-concept of passive participants; the authors suggest the students avoid the embarrassment of revealing a lack of skill in person to their peers. The author further suggests that their results support the use of peer-assisted learning strategies combined with CSCL in elementary schools.
The author thoroughly compares his findings with other published research. It should be noted that the students in the study were provided with scaffolded strategies by the teacher to support their interactions; scaffolding is highlighted by the other authors (Hakkinen (2002), Brown, Collins & Duguid (1989)) in this list as being necessary for success in group work. This study further highlights the benefits of peer review in CSCL.