Lindsay Jordan
Leading and Managing Educational Innovation MA unit submission
September 2009
An analysis of the progress, strengths and limitations of an attempt to manage educational innovation at the University of Bath.
Introduction
This assignment draws critically on selected literature on the management of change to analyse my own experience of managing educational innovation. The innovation in question is the redevelopment of the six core modules of a distance learning Masters programme in International Construction Management, primarily through the introduction of online collaborative learning and assessment activities. The assignment analyses the need for and approach to change, and the successes and limitations of the process.
Prior to the start of redevelopment, the modules were in the form of paper workbooks and files comprising content organised in sections, individual reflective exercises and essay-style assignment tasks that, along with written examinations, were the means of summative assessment for each module. In 2006 the programme management team, in partnership with two other distance learning programmes in the faculty, secured funding to formulate and implement a ‘blended’ (ie online and traditional distance learning) strategy. In 2007, a significant proportion of this funding was allocated to a full-time e-learning developer post - which I was subsequently appointed to - with a remit to lead the project.
The need for change
Although by 2006 the programme had been attracting applicants and producing graduates for several years, attrition rates were high, with up to 50% of students withdrawing or deferring during the first 6 months of the 3-year programme.
High attrition rates have historically been common among distance learning programmes (Diaz 2002, Parker 1999). A large number of studies (e.g. Minich 1996, Powell, Conway & Ross 1990) have concluded that factors external to the institution, such as family responsibilities, are largely responsible for withdrawal from distance learning courses. However, when our students were asked through focus groups and surveys what they found most challenging or demotivating about distance learning, they didn’t focus on external factors, but spoke instead about feeling isolated, wanting more formative feedback and feeling insufficiently prepared for academic work; factors that are very much within the control of the programme management team.
Structured interviews with tutors supported the students’ responses. These interviews focused on the primary reasons for student failure or poor achievement, and the existing challenges of tutoring on the programme. They highlighted a need for more contact between students and staff, more feedback and greater utilisation of the students’ broad range of existing knowledge and experience.
The findings from our informal study echoed those of Brown (1996), who presents extensive evidence in support of internal factors (such as insufficient support and difficulties in contacting tutors) having a more significant role to play in non-completion of distance learning programmes. The programme management team resolved to focus on addressing the demands for change that had been identified by the students and tutors, and to utilise available technologies to enable the provision of a more supportive and learning environment for all students on the programme. The ultimate aim was to improve student engagement, achievement and retention.
Approach to change
The approach taken to redeveloping the core modules has been presented in Appendix A with an adaptation of Everard, Morris and Wilson’s (2004) ‘systematic approach to change’ model. Everard et al’s model has been chosen as it represents most the major inputs and outputs for this particular change process and indicates the relationship and aims of each activity within the process. There is, however, an apparent emphasis within Everard et al’s model on diagnostic and planning activities over the evaluation and institutionalisation of the change process, which I see as a limitation of the model.
Extent & complexity of change: Single innovation or multiple innovations?
According to Everard, Morris and Wilson (2004), few individuals in organisations appreciate the multidimensionality of change. It makes sense that stakeholders may focus only on the change(s) that affect them directly. However, it has been shown through evaluation surveys and unit reports that most tutors and the majority of the students on the ICM programme recognise the complexity of the changes that have been implemented over the last two years. There has been an evolving profile of innovation, with different aspects of the innovation at different stages throughout the implementation phase.
One could take the view that innovation(s) sit at some point along a scale of complexity and/or dynamism. Much of the research on complex and multiple innovations (e.g. Wallace 2003, Wallace & Pocklington 2002) is set in the context of schools, where dynamically complex change is often the result of a continually evolving profile of government reforms. In comparison, the innovation discussed here has a far narrower scope, and I have found research-informed theoretical principles and perspectives on single innovations of more use when looking back on the change process, particularly in terms of making sense of the schedule of change and the different stages of the process. One concept that I feel is relevant and helpful, and will use to structure the following analysis, is Fullan’s (2007) perspective on the three phases of innovation; initiation, implementation and institutionalisation.
The initiation phase
As the e-learning developer had initially been appointed on a 12-month fixed-term contract, the pressure was on to start implementing change, with only four months available for further initiation and planning. It was accepted among the programme team that a fair amount of 'evolutionary planning' would take place; i.e. adapting the original plans as the implementation proceeded.
Much of the activity presented in Appendix A took place in the initiation phase, although certain elements were repeated in further development cycles. The initial student survey and tutor interviews indicated the nature and potency of the need for change, and helped to inform the desired future state alongside the e-learning developer’s existing knowledge of research-informed principles of learning.
The future state the programme team wanted to move to was one where there was:
· Regular and educationally effective communication between students and staff
· Reciprocity and co-operation among the student body
· Active learning
· Prompt feedback to students.
The programme management team were planning to achieve this through the incorporation of collaborative learning and assessment activities into the modules. These activities, which would be based within the VLE and utilise tools such as discussion forums and wikis, would be designed in collaboration with the module tutors. They would be moderated by the tutors with initial support and assistance from the e-learning developer. This would entail a significant increase in the amount of time tutors would be spending working with the programme. Aside from presenting at residential sessions and setting and marking assignments and examinations, tutors had previously only been paid a small fixed annual sum to answer occasional queries from individual students. During the initiation phase, a new fee tariff was put into place to allow module tutors to be remunerated for time spent on learning to use the VLE, collaborating on activity development and moderating the online activities.
As well as addressing the nature of feedback that was given to students, we particularly wanted to improve the quality of the feedback we received from students about the programme, thereby enhancing the students’ role in future development. As indicated in Appendix A, the existing demand/response system of paper-based, postal-distribution student feedback questionnaires had been yielding few responses with little detail on specific areas for development. It was hoped that, by opening up channels for productive dialogue between students and staff, the students would be willing and able to take a more significant role in the evaluation of the programme. It was expected that difficult issues would be brought into the open; however this was accepted as an essential step in moving towards a more student-centred learning experience.
Determining the work to be done to move from the programme’s existing state to the desired future state was a key and complex part of the initiation phase. On reflection it is this part of the process that would have benefited most from additional time, and more experience on the part of the e-learning developer. Despite also being grounded in the context of schools, I have found House’s (1981) three perspectives on educational change - logistical, political, and cultural - useful in highlighting aspects of the initiation phase that may have benefited from more attention.
As the e-learning developer, my own perspective at the start of the process was very much focused on the technological; I felt that the majority of the work lay in course and activity design, formatting of the learning environment, estimating how much time the module tutors would need to spend moderating the activities, and producing the resources they would need in order to become effective online tutors. On reflection, this initial emphasis on the logistics of the project was influenced by the programme management team’s initial expectations of the project in general and of me in particular, as well as my lack of experience in leading educational change. As I gained experience, I became more aware of disparities in the team’s goals and values, and of how such disparities inhibit the sustainability of educational change, as emphasised by Owston (2006) and Hipp (2008). My perspective moved significantly towards the cultural as I subsequently tried to address barriers arising from these differences in goals and core values.
To provide one example; the tutor for the introductory module was initially reluctant to provide marking criteria or examples of work. After several months, it finally emerged that he felt it wasn’t his responsibility to teach students how to construct an academic essay, and that, rather than increasing the amount of support available to new students, he felt that we should be more stringent with the academic entry requirements for the programme. The programme management team, in contrast, felt very strongly that we should offer an ‘access route’ whereby students could undertake the first two modules regardless of their academic qualifications, and receive appropriate support. Once this difference in core values was made explicit, it became possible to address it appropriately. A deeper investigation into core educational values at the beginning of the project might have enabled these issues to be addressed sooner.
Taking a political perspective from the start of the change process may (according to House), have promoted a more co-operative approach to change, based on the interests and core values of the stakeholders. A concerted effort was made in this case to explore the tutors’ individual interests and core values in the early stages. However, it is likely that the tutors’ true feelings were obscured as they attempted to make a positive impression on the newly appointed e-learning developer. My optimistic nature and inexperience led me to take the tutors’ apparent readiness and capability for change at face value. Initially management, tutors and students seemed in agreement about the desired state we should be aiming for, and therefore Fullan’s (2007) ‘leadership dilemma’ - whether to seek majority agreement or assertively push for implementation - did not arise. The resistance experienced later on, during the implementation phase, and the need for assertive implementation, was therefore both surprising and frustrating.
According to Marris (1986): “every attempt to pre-empt conflict, argument, protest by rational planning, can only be abortive: however reasonable the proposed changes, the process of implementing them must still allow the impulse of rejection to play itself out.” (p.155)
However rational this sounds, rejection of the proposed changes to practice in this case - as described in the next section – was disruptive to the student experience. What might have been done differently in the initiation phase to seek out differences in core values? A group face-to-face orientation event might have provided a better stage for the tutors to explore their (and each others’) ideas about formative assessment and collaborative learning, and to engage in collaborative goal-setting. In this case, geographical dispersion of the tutors meant that such an event would have required a considerable financial cost, and, as differences in goals and values were initially obscured by the tutors’ apparent enthusiasm for change, the programme management team did not feel that the benefits would justify the expense.
Initiation of the change process overlapped considerably with the implementation phase as a rolling schedule of changes to the six modules was to take place over 18 months. One benefit of this rolling schedule was that it enabled the team to apply lessons learned in the implementation of changes in the first two modules to the later modules. Another was the availability of a rich source of exemplar material and case studies for use with tutors working with the later modules; one of the ‘domino effects’ mentioned in Appendix A. A third major benefit was that it should have resulted in a consistent experience for all students. The rolling implementation did, however, bring challenges for the Module 1 and 2 tutors, who not only had to change their practice without a model or example to work from, but also had to cope with continuous change as lessons learned were fed back into their modules for subsequent cohorts. The module 1 and 2 tutors, and the students in the first cohort to experience the implemented changes, were essentially ‘guinea pigs’ for the new tools and learning activities.
The implementation phase
A particularly useful perspective on implementation is that of Fullan (2007), who presents four factors that affect success; the stakeholders’ perceptions of need, clarity, complexity and feasibility. As these sets of factors are presented as characteristics of single innovations, they can be useful in analysing why some aspects of the change process were adopted more readily than others during the implementation phase. Owston (2007) cites Rogers’ (1995) alternative set of factors that are perhaps more relevant to technological innovations; relative advantage compared to existing practices, compatibility with existing values, complexity, trialability (the degree of experimentation or piloting possible) and observability of the benefits.
One aspect of the redevelopment was that written assignments were to be submitted and marked electronically through the VLE. Tutors had to learn how to access the assignments online, insert comments on the students’ work electronically and upload the annotated assignment to the VLE as a response file, alongside some general feedback. Although tutors found this laborious at first, there was surprisingly little resistance. Fullan’s and Rogers’ perspectives serve to explain this. There was an obvious need for (or relative advantage to) such a system because students and tutors generally work at considerable geographical distance from the University, and with the existing postal system it was virtually impossible for the programme team to get feedback to the students within the University’s recommended time period of three weeks. The requirement was made as clear and simple as possible through the provision of training resources in a variety of formats (text & images, video). Aside from an initial time outlay as everyone got used to the system, it proved to be just as efficient from the tutor’s point of view as paper marking - if not more so - and it was therefore perceived as feasible. The decrease in time taken to receive assignments and return feedback was clearly observable.
The trialability of the different aspects of the whole innovation process described here is an interesting point to reflect upon. All of the new tools and activities that were introduced were certainly trialable, and ‘dry runs’ could have been carried out with all staff using dummy students, provided the staff had the time to give to the trial, which in most cases they did not. One could also take the view that, as we were applying the principle of evolutionary planning, everything we did for the first time could be seen as a ‘trial’ or ‘pilot’. There was one incidence where a collaborative assessment task was explicitly introduced to both students and staff as a pilot (Jordan 2009), and this has since developed into an exceptionally successful element of the programme in terms of student participation, interaction and learning. One might take this as evidence in support of Rogers’ view that trialability is an important factor in the successful implementation of innovations. However, as Owston (2007) points out, when tutors are involved in the development of an innovation, or when they have observed fellow educators implementing something similar, the trialability factor (and also clarity and complexity) may not be as influential.
In contrast to the implementation of online assignment marking, a more sporadic uptake was observed with the requirement for tutors to take an active part in online discussions. Clarity, complexity and feasibility did not, in most cases, appear to be major issues; however, it took considerable time for some tutors to believe that there was an educational need for them to engage in this type of activity. Referring to Rogers’ factors, one could say that this particular innovation was not compatible with tutors’ values and experience. Only until tutor engagement was sufficient to result in an observable positive impact on student participation did the tutors’ input became more consistent. It was fortunate that the two newly-appointed module tutors (for the third and fifth modules) moderated their forums diligently and skilfully from the start, as this provided evidence of the educational worth of such an activity. However, this was not evident until 9 months and 15 months into the implementation stage. The lack of regular tutor input in the discussion activities in other modules prior to this meant that students' expectations were not met; this was made explicit in the students’ responses to the module evaluation surveys. However, the quality of the student feedback that the new evaluation process yielded provided us with further leverage with which to persuade the tutors that their input was not only appreciated, but considered necessary for the learning experience. Our experience supports that of Owston (2007), who talks about “the essential role students play in motivating teachers to sustain an innovation” (p.71).
A system for gathering detailed evaluative data from students had been in place from the start of the implementation phase. As the months passed, the involvement of the tutors in these formative evaluations increased. Initially the e-learning developer would collate the student feedback and use it to identify issues that needed to be resolved and any areas for further development, which were then acted upon, primarily by the e-learning developer. Subsequently, this process evolved so that the recommendations were passed to the tutors, who were asked for comments, and a decision on which of the suggested further developments they would like to explore. Most recently, the evaluation process has been centred around a biannual face to face interview between the module tutor, the e-learning developer and the Director of Studies, which is preceded by collation of student feedback by the e-learning developer and an analysis of assessment results by the module tutor. The meeting is focused on deciding upon actions to address feedback points and poor student achievement, and is presented as a conversation between equal parties with a view to collaboratively deciding upon the way forward. This particular aspect of the change management process resonates with three of Louis and Miles' (1990, cited in Fullan 2007) key themes; empowerment, problem coping and evolutionary planning.
It was during the first set of evaluative interviews that persistent differences in core values were finally made explicit, and addressed by reaching an agreement that was focused on actions rather than principles, and accepting that these actions may represent a compromise of our respective values and beliefs. This strategy is supported by Wallace’s (2003) views on orchestrating complex educational change. Wallace suggests that educational leaders can transform staff culture through emphasising professional dialogue, team working and mutual support, and by articulating an altruistic vision that reaches beyond individuals’ self-interest.
Fullan (2007) suggests that practices are easier to change than values and beliefs; a view that resonates with personal experience. A major challenge has been the difficulty in gaining a full understanding of tutors’ core values. With staff based at a distance from the University, there is little opportunity for the type of casual or serendipitous conversation that promotes the kind of relationship where core values are shared and discussed, and may evolve naturally.
Fullan (2007) also suggests that a change in values is likely to follow a change in practice if the experience is deemed to have been a positive one. We were certainly hoping for this outcome in the case of requiring tutors to take an active part in the online discussion forums, and it has been observed to some extent in all cases; for example, the tutor who previously saw no need for example answers and detailed marking schemes has since found them very useful to refer to when giving summative feedback.
Drawing on the themes of successful implementation suggested by Louis and Miles (1990), Fullan (2007) advises that empowerment and choice for users must follow the type of assertive implementation we chose to employ with some tutors. The deeper involvement of the Module 3 and 5 tutors in the design of activities corresponded with a much less painful implementation. Although it was clear that these tutors’ core values were most closely aligned with the goals of the project, we felt that if the all tutors were making active choices about the design and development of learning and assessment activities they would be more enthusiastic about managing them; this viewpoint strengthened the case for increasing the tutors’ involvement in module development through the biannual review.
A by-product and an essential factor in the implementation of this particular change process has been the establishment of a professional learning community encompassing the staff and students connected with the programme.
Building a learning community
‘It is one thing to facilitate collaborative learning in a traditional, face-to-face classroom setting, but quite another to do so over thousands of miles that span several time zones and cultures. In this context, the learning community exercises some special qualities.’
Irwin & Berge (2006)
The philosophy of learning communities or ‘communities of practice’ has been promoted by many, most notably by Lave and Wenger (1991). Through the re-development of these modules, we wanted to build a community of learners which would not only act as a support network but also facilitate the sharing of the breadth of knowledge and experience across many industry sectors and national contexts represented within the student body.
To support the development of a trusting environment necessary for the growth of successful learning relationships, as proposed by McConnell (2002), the redeveloped modules each included at least one ‘socialisation’ activity. Salmon (2002) presents three stages to online socialisation: Establishing a successful group, introducing the knowledge domain and the learning approaches to be used and introducing how the learning environment is to be used. For the ICM students, we designed an online induction area within the VLE where students gain familiarity with the VLE and its various tools (e.g. forums, shared databases, wikis) alongside other students new to the programme. The individual module areas focus on establishing the group, the knowledge domain and the particular learning approaches for that module.
We found that the most successful module introduction activities are those that:
a) Indicate real-time presence from the tutor; this encourages students to participate as they are keen to make themselves known to the person who will be overseeing their activity and assessing their contributions.
b) Introduce the tutor from both a professional and personal perspective, and demonstrate the tutor’s empathy with the challenges of part-time, off-campus study.
c) Gives students the opportunity to introduce themselves.
An example of an introductory activity that we have used in one of the modules is shown in Appendix B.
Once students have introduced themselves and/or contributed to the introductory task, they are prompted to take part in a number of collaborative learning activities. The exact nature of these varies between modules but in the main they take the form of online discussion. There is widespread agreement in the literature (Salmon 2002, Black 2005, Irwin & Berge 2006) on the value of discussion in interactive online learning environments. Effective online discussions promote reflection and critical thinking through dialogue, and enable the participants to collaborate on the construction of meaning (Black 2005); we have also found them to be a key element in the development of a learning community.
The launch of a virtual Tutors’ Area was the result of a decision to apply similar techniques used to build effective student learning communities to the development of a tutor community. As mentioned previously, the freelance and off-campus nature of the tutoring staff has been a significant factor in the challenges the team encountered while implementing change. The degree to which tutors wanted to be involved in the design of the redeveloped modules was variable. Lack of familiarity with the VLE appeared to cause problems with some tutors and not others; some, despite never having used the VLE previously, took to it immediately, while others continued to report feelings of confusion and incomprehension after a number of online and face-to-face orientation sessions. In addition to links to examples of what we considered to be good practice, tutors were invited to share their ideas on, for example, what constitutes a good learning experience, and given space to freely air views on challenges and benefits of change. It was hoped that this would encourage the development of positive working relationships between tutors and a sense of supportive community.
The community area saw a lower amount of active participation then the programme management team had hoped for, although it was apparent that many tutors had accessed the area and it was successful in raising awareness among the tutors of practices in other modules. One or two tutors also used the forums to post up specific concerns, which gave us the opportunity to respond constructively and transparently to them. With hindsight, the development of a sense of community between module tutors prior to module redevelopment may have made it easier to overcome some of the challenges we experienced in effecting pedagogical change.
Kozma (2003, cited in Owston 2007) describes three contextual levels that affect and mediate change; the micro level (e.g. personal characteristics of tutors and students and the organisation of the learning environment), the meso level (institutional administration and policy) and the macro level (e.g. international trends). Kozma states that the success of practices depends on the ways the factors at various levels “fit together and reinforce each other” (p.63). In our case, it is apparent that these levels are more closely aligned now than they were at the start of the project; although it took several months for the relevant characteristics of tutors and students to be revealed, and for the necessary amendments to administration and policy to be recognised and put into place.
Institutionalisation
After two years, the implemented changes are on the cusp of institutionalisation, and the sustainability of the innovation is to be tested as the e-learning developer is shortly to move on to other projects.
Wallace (2003) suggests that “forcing through a change by overcoming resistance -
as opposed to trying to head off negative reactions by nurturing a culture of acceptance - can never bring more than minimal compliance” (p.25). In order for the innovation to sustain itself, we needed to achieve more than minimal compliance from every member of teaching staff. It is worthwhile to note that Wallace’s research was grounded in the context of schools, where cultures of acceptance and resistance are perhaps more pervasive. The lack of community among our remote tutoring staff at the outset of the project may have caused a few challenges, but it may also have been beneficial in allowing us to address resistance on an individual basis. To put this more bluntly; the resistance from some members of staff did not affect the enthusiasm of others.
Owston (2007) states that for improvements to be sustainable, they must ‘focus on creating environments that foster diverse and creative approaches’; this will ‘develop long-term capacity for improvement’ (p.65). Recent evidence suggests that the tutors’ increased involvement in module evaluation and the selection of areas for further development has coincided with an increased sense of ownership and a more creative mindset.
One example of such evidence is the increasing willingness of certain members of staff to engage in positive and encouraging dialogue about assignment feedback. The facility for students to respond formally to their assignment feedback was first introduced in January 2008. Initially, all the tutors felt that it was a good idea to encourage students to reflect on their feedback and how they would use it when approaching future assignments, but many said that they did not wish to be ‘drawn into conversation’ about the grade awarded. Within the last two months, a significant change in approach has been observed by two of the tutors, who have willingly entered into positive and encouraging dialogue with students about their feedback.
Differences in participation between module tutors are still apparent, but are decreasing. This levelling-out has been augmented by the dissemination of good practice across the core modules, and the provision of differentiated support; what Wallace (2003) describes as “carefully planned but also responsive provision of whatever different people need, when they need it.” (p.25). Wallace stresses the importance of ongoing needs identification procedures, which the continuation of our biannual module review process will provide a formal channel for. It is also hoped that the ongoing development of relationships between tutors will facilitate more frequent, informal support.
Conclusion
Although we, the programme management team, had a clear idea of our goals for change and wanted the tutors to subscribe to these, we didn’t want to resort to managerialism to get there. We knew that enforcing tight control could stimulate even stronger resistance to change and also believed in the professional right of the tutors to choose what they do and how they do it, but we still wanted them to choose our vision and our goals. This presented us with a dilemma, and in hindsight, I do feel that we were guilty of the following:
“When those who have power to manipulate changes act as if they have only to explain, and when their explanations are not at once accepted, shrug off opposition as ignorance or prejudice, they express a profound contempt for the meaning of lives other than their own.”
Marris, 1986 (p.155)
On reflecting upon my first experience of change management, I have found Peter Marris’s book a valuable reminder of the need to respect the sense that others’ have made of their practice and the validity of their existing conceptions.
It has already been suggested that the tutors for the first two modules were disadvantaged as none of us knew what to expect in terms of student activity and participation, and what kind of moderation or intervention would be required. In hindsight, bearing in mind the continuous cycles of re-development and varying input from the module tutors, one might question whether we managed to achieve a sufficient degree of consistency in the student experience to justify the challenge such a schedule created for the Module 1 and 2 tutors, who were, co-incidentally, less able or willing to support the change than the other tutors. It is reassuring that a focus group with students from the first cohort to complete all six of the redeveloped core modules concluded that, although they would not have described their experience as consistent, they felt that it had been consistently improving.
On reflection, it is uncertain whether the path of implementation would have been particularly different whatever approach we had taken in the first instance. None of the staff involved on the programme had enough experience in the early stages to predict with any accuracy what would work and what the outcomes would be. Ongoing formative evaluation and the subsequent adaptation of plans (identified as key themes of successful innovation) would have been necessary regardless.
An alternative research, development and dissemination (RDD) style approach; i.e. disseminating good practice and research to staff and inviting them to implement the ideas on their own terms, would have been very unlikely to result in any significant change. Such an approach is only likely to result in change if the ideas being promoted are in line with existing values and needs. Earlier on this year, an attempt was made to stimulate innovation in feedback practices on the programme using an RDD approach. Although no changes in practice have resulted as yet, one tutor has expressed an interest in using some of the proposed ideas. This is encouraging, and could be taken as further evidence of a shift in core values.
The most important factor in the successful implementation of this change has, and continues to be, the growth of a productive learning community among the students and tutors on the programme. It is expected that the active learning community of students and tutors has now reached the critical mass necessary to sustain previous and ongoing development.
References
Black, A. (2005) The use of asynchronous discussion: Creating a text of talk. Contemporary Issues in Technology and Teacher Education, 5 (1).
Brown, K. M. (1996) The role of internal and external factors in the discontinuation of off-campus students. Distance Education, 17 (1), 44-71.
Diaz, D. P. (2002) Online Drop Rates Revisited. The Techology Source. Available from http://technologysource.org/article/online_drop_rates_revisited/ [Accessed 01.06.09]
Everard, K., Morris, G. & Wilson, I. (2004) Effective School Management. London: Paul Chapman
Fullan, M. (2007) The New Meaning of Educational Change [4th Edition]. London: Routledge
Hipp, K. K. (2008) Sustaining professional learning communities: Case studies. Journal of Educational Change, 9, 173-195.
House, E. R. (1981) Three Perspectives on Innovation [in Lehming, R. and Kane, M. (1991) Improving Schools: Using What We Know. London: Sage]
Irwin, C., and Berge, Z., 2006. Socialization in the Online Classroom. e-Journal of Instructional Science and Technology, 9 (1). Available from http://www.usq.edu.au/electpub/e-jist/docs/vol9_no1/papers/full_papers/irwin_berge.htm [Accessed 01.06.09]
Jordan, L. (2009). Using online role-play to assess distance learning students in Construction Law; Case Studies of the HEA Centre for Education in the Built Environment. Available from: http://www.cebe.heacademy.ac.uk/learning/casestudies/list.php.
Lave, J. & Wenger, E. (1991) Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.
Marris, P. (1986) Loss and Change [2nd Edition]. Trowbridge: Redwood Burn
McConnell, D. (2002) The experience of collaborative assessment in e-learning. Studies in Continuing Education, 24 (1), 73-92.
Minich, E. L. (1996) Using student feedback to improve distance education. Educational Resources Information Center. Available from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/14/a1/cf.pdf [Accessed 28.05.09]
Owston, R. (2007) Contextual factors that sustain innovative pedagogical practice using technology: an international study. Journal of Educational Change, 8, 61-77.
Powell, R., Conway, C., Ross, L. (1990) Effects of student predisposing characteristics on student success. Journal of Distance Education, 5 (1), 20-37.
Salmon, G. (2002) E-Moderating: The key to teaching & learning online. Taylor & Francis, London.
Wallace, M. (2003) Managing the Unmanageable: Coping with Complex Educational Change. Journal of Educational Management & Administration, 31 (1), 9-29.
Wallace, M. and Pocklington, K. (2002) Managing Complex Educational Change. London: RoutledgeFalmer