As you have seen from your reading, there are four types of formative evaluation. You are required to write a brief plan for how you will carry out three types.

4A. Expert Review

The current Head of ICT at SCIS, Mr Ed Monds, will act as SME and expert reviewer. Ed has 25 years teaching experience in the ICT field and holds a MSc ICT in Education. Ed has experience in developing instruction for the use of Adobe Dreamweaver and Flash for the various age groups in the the secondary school; his knowledge of teaching students how to build websites makes him the best choice as reviewer.

Ed will be requested complete his review within 1 week starting January 6th, 2014. His feedback will be used to enhance the instruction in preparation for the one-to-one trials commencing on January 20th. The small group trial will commence on January 27th with the completed evaluation on February 3rd. The Field Test will begin on February 10th with the final evaluation being completed on March 10th.

Using a 5 point Likert scale and comments, the SME will provide feedback on whether:

• the instruction meets the Goals/objectives,
• the design and functionality of the SCISport learning context is sufficiently similar to the performance context of the AQA Enrichment diary,
• the pre-test Student IT use survey provides the best analysis of the students’ experience,
• the instruction is chunked appropriately,
• the organisation of the instruction flows appropriately in order for the students to build their portfolio,
• the layout and presentation of the GSite instruction meets Multimedia Design Principles (Clark, R. C., & Mayer, R. E. (2008)),
• the pace of the YouTube demos is appropriate and uses an appropriately “conversational” style,
• the opportunities for self and peer assessment are appropriate.

4B. One-to-One Evaluation

Four Year 10 students, two boys and two girls, will be selected from the cohort. These students will be chosen based on their ability and confidence to express themselves to their teachers, and their skills of evaluation. The experience of these students when using the instruction will be used to identify any overarching problems within the instruction.

In reference to Kirkpatrick’s Four-Level Evaluation model on page 11 of Streamlined ID (Larson & Lockee, 2013), a Level 1 Learner Reaction evaluation will seek to find out how the students react to the delivery method of the learning experience. Since the instruction is designed to be completed in a peer assisted and assessed model, the One-to-One Evaluation cannot directly assess these elements of the instruction, but should give an indication as to how the instruction “feels” to the students.

Questions will be built on a 5 point Likert scale, allowing the students to gauge their reaction to:

• the layout of the instruction,
• the ease of reading,
• the quality of the video and sound in the YouTube videos,
• how easy the videos were to follow.

A Level 2 Learning Evaluation will seek to identify if the students completed the tasks they were trying to achieve and how well the instruction guided them in this process.

Questions will be built on a 5 point Likert scale, allowing the students to assess how well they were able to independently i.e. without instructor assistance:

• edit the Google Site,
• set up their Blog and set the correct privacy policy,
• upload an image to GDrive,
• upload a video to YouTube and set the correct privacy policy.

Further Level 2 Likert scale questions will seek to identify how well the instruction assisted the students in completing the tasks. Students will be asked to comment on their reasons for selecting the grading they chose in order for the designer to improve any weak elements or replicate any particularly strong elements.

4C. Small Group Evaluation

In order to test the revisions made as a result of the Level 1 and 2 evaluations in the One-to-One evaluation stage, three group evaluations will be performed with volunteer students from a single Registration Class. These students will be friends and would likely work together in a group if given any group task to complete; they are comfortable working with each other and giving each other peer assessed comments. The groups will be made up of a pair, a group of three and a group of four: as an instructor, I wish to see which dynamic works best.

The same Level 1 and 2 questions from the One-to-One evaluation will be asked of the students in order to evaluate if the revisions made have had a positive effect.

Further Level 2 questions will be designed on a 5 point Likert scale to identify if the students were capable of setting the sharing settings for their Google Site, GDrive, YouTube videos and Blog to restrict or allow their peers to view their work.

As part of a Level 3 evaluation, the students will be asked to evidence elements of their work within the Google Site ePortfolio, GDrive and YouTube and write a reflective blog entry within seven days. The students will then complete a second evaluation to identify how well they remembered the processes to complete the tasks or if they needed to review the videos again. Questions will be designed to identify if the students had any trouble using the system and if they met with any motivational issues while evidencing their work and writing their reflective blog post. Comments and suggestions for improvement will be sought in order to improve and enhance the system.

4D. Field trial (Implementation)

A field trial of a full tutor group will be conducted in order to evaluate the changes made from the small group evaluation and to identify any problems that may occur during the administration of the instruction by Form Tutors. If any technical issues arise, this evaluation should identify them.

This class of students will be given the same Level 1 and 2 questions to complete, along with the same Level 3 evaluation questions to complete after one week. After four weeks of gathering evidence and having the Form Tutor review the collections and blog entries, the students will be given the same questions in order to review their motivation toward the process and whether they value the process or not.

To evaluate if the instruction has had an affect on the behaviour and motivation of the cohort as a whole, their AQA Enrichment Diary scores in Year 12 will be compared to the previous cohorts. If the instruction has been successful, a null hypothesis that states that the instruction will have no significant effect can be rejected assuming that a significant improvement is found and that it is due to the treatment caused by this project.