Executive Summary

Building the Capacity of Health Care Professionals

in Western Province, Zambia -

An Okanagan Zambia Health Initiative (OkaZHI)

Evaluation Report

Cameron Jones, Andrew Silumesii, Lianne Jones, Rebeccah Nelems and Bill Nelems

January 2012

© All Rights Reserved Okanagan Zambia Health Initiative 2008-2011

www.okazhi.org

Contact: billnelems@fastmail.fm

Executive Summary

In 2006, Dr. Bill Nelems made his first trip back to Zambia in over 50 years to meet with his colleague and medical school classmate Professor Chifumbe Chintu. During this visit, Professor Chintu introduced Dr. Nelems to a range of key players in the Zambian medical and health sector to discuss potential international cooperation.

Years later, a series of Canadian-Zambian collaborations have taken place, building on this initial visit:

• Institutional and personal relationships have been established between Zambian and Canadian colleagues and institutions, with the shared vision of promoting and improving health outcomes and quality of care in Zambia;

• A Canadian registered charity, the Okanagan-Zambia Health Initiative (OkaZHI) was founded in 2010, under the auspices of which numerous Canadian health professionals (doctors, nurses, healthcare educators, etc) have joined Nelems in the commitment to improve health care in Zambia;

• The University of British Columbia Okanagan (UBCO) established a Memorandum of Understanding (MOU) for collaboration with the University of Zambia (UNZA);

• OkaZHI is in the process of signing a formal MOU with UBCO since many members are affiliated with both institutions and share common interests in Zambia;

• OkaZHI built a collaborative relationship with the Canadian Network of International Surgery (CNIS - an international NGO committed to sharing knowledge, expertise and experience to promote lasting and sustainable improvements in health and safety in the developing world), with whom it collaborated to deliver two adapted CNIS training modules in Western Province, Zambia;

• OkaZHI developed and delivered an additional four medical training courses and matching nursing courses in Western Province, with the guidance and support of the Zambia Ministry of Health and the Lewanika General Hospital (LGH);

• Between 2009 and 2011, 92 Zambian health professionals from Western Province attended OkaZHI training courses (53 nurses, 25 physicians, 5 medical licentiates, 9 clinical officers);

• UBCO established an annual nursing practicum, under which 45 UBCO 4th year nursing students have traveled to Mongu, Western Province to work for an eight-week period since its establishment in 2010; and

• OkaZHI-UBCO established linkages with other Canadian players working in Zambia, recently becoming a member of the “Zambia-Canada Research Partnership” facilitated by the Canadian Coalition for Global Health Research (CCGHR - a knowledge and relationship broker committed to reducing health inequities through the production and use of knowledge), with a goal of ensuring coordination and harmonization of Canadian efforts in Zambia.

Five years after Dr. Nelems' and Professor Chintu's initial visit, it was time to take stock. In July of 2011, an OkaZHI evaluation team travelled to Western Province, Zambia, to formally and systematically assess contributions of, and challenges to, OkaZHI's efforts since its inception - seeking Zambian colleagues and partners' experiences, perspectives and suggestions for OkaZHI as it moves forward. OkaZHI Executive Director Bill Nelems and Canadian OkaZHI members Lianne Jones, Cameron Jones teamed up with Dr. Andrew Silumesii, the Medical Superintendant for Lewanika General Hospital, Mongu, Western Province, in order to conduct a thorough, qualitative evaluation of OkaZHI. A comprehensive evaluation framework with methodologies was developed and received UBC's ethics approval. Evaluation methodologies included a detailed, anonymous written survey completed by 72 OkaZHI course participants, site visits and one-on-one interviews with 70 OkaZHI course participants. Interviews with 7 other key stakeholders in Lusaka and Western Province were also held.

The data gave evidence to eight key findings. Detailed in the report below, these include:

Finding 1: Zambian course participants reported improved health outcomes for patients, due to improvements in their delivery and quality of care following the OkaZHI courses.

Finding 2: Zambian course participants said they felt more confident in their practice because of the new skills and knowledge they had acquired through their collaboration with OkaZHI.

Finding 3: Course participants found OkaZHI courses to be highly relevant to health care needs in Western province and directly applicable in their day-to-day work.

Finding 4: Course participants found OkaZHI courses, methodologies and instructors to be highly comprehensive, professional and innovative - though courses were commonly critiqued as being too short.

Finding 5: 90% of OkaZHI course participants said they have shared knowledge gained in the courses with colleagues, suggesting OkaZHI is having reach beyond the classroom.

Finding 6: Course participants health professionals noted less than expected use of eGranary digital library due to lack of training and accessibility.

Finding 7: Course participants identified a variety of contextual factors, which affect the overall impact of OkaZHI`s interventions, including workload issues, retention issues and lack of equipment.

Finding 8: Course participants made a number of specific requests about directions for ongoing support from OkaZHI, including teaching at UNZA School of Medicine.

Amidst these findings, three key lessons emerged.

First and foremost, OkaZHI efforts would not have achieved anything were it not for the incredible commitment, partnership and guidance of its Zambian colleagues and partners.

From Dr. Margaret Maimbolwa, Assistant Dean of the University of Zambia School of Medicine, to Dr. Sitali, the Provincial Medical Officer for Western Province, to Dr. Andrew Silumesii, the Medical Superintendant for Lewanika General Hospital in Mongu, OkaZHI has been fortunate to have the highest level of commitment, support and partnership from key Zambian actors and institutions. This has enabled OkaZHI's work to be guided by locally identified priorities and situated within existing institutions and initiatives on the ground.

The time invested by OkaZHI members on the ground in Lusaka and in Western Province in the past several years has helped to enrich these relationships - with the majority of interviewees commenting, without solicitation, on the high degree of trust, partnership and collaboration they have experienced in their interactions with OkaZHI personnel. This sense of true partnership and collaboration is a defining feature of OkaZHI's approach and a key contributing factor to its successes to date.

OkaZHI has also been effective at establishing linkages with other Canadian actors working in both the sector - such as the Canadian Network for International Surgery (CNIS) - and in Zambia, through the CCGHR-facilitated “Zambia-Canada Research Partnership”. Ensuring coordination and collaboration with Zambian, Canadian and other international actors working in the sector in Zambia will continue to be critical to OkaZHI's successes.

Second, OkaZHI efforts to date have yielded significant outcomes in Western Province - especially given the relatively short timeframe in which it has been working in Zambia.

97.2% (70 of 72) of Zambian health professionals surveyed and interviewed reported improved health outcomes for patients, due to improvements in their delivery and quality of care following the OkaZHI courses. (Note: The two respondents who said their skills had not directly contributed to improved quality of care qualified that they had not had the opportunity to apply the skills.)

Every doctor provided examples of decreased or eliminated post-op infections, complications, and declines in patient referrals - the latter being most significant for rural medical staff operating out of remote districts. Doctors, nurses and medical licentiates all reported a decrease in duration of patient stay, due to reduced post-operative infections, stating that this has increased the number of patients addressed daily. Further, 95.8% (68 of 71) of survey respondents reported an increased confidence in their clinical skills following OkaZHI programming. Improved speed, reduced infections and/or mistakes and decreased patient referrals were commonly cited as direct outcomes of an enhanced confidence gained through OkaZHI trainings.

OkaZHI courses were largely regarded as relevant to local health needs. 100% (72 of 72) of participants claimed to have used learned skills in their day-to-day work - and 90% (63 of 72) reported having shared material or knowledge learned with colleagues who did not attend the courses.

This data points to a clear mandate for OkaZHI to continue building on its successes. Interviewees and survey respondents were unanimously positive about their desire for OkaZHI to continue working in Western Province, making a series of commonly expressed recommendations or requests. The recommendations varied widely but several requests were frequently repeated without any prompting. 93% (66 of 71) of participants surveyed requested “further training or support” in at least some aspect of trainings previously received from OkaZHI. Other frequently identified requests pertained to how OkaZHI could amplify its presence and support for healthcare practitioners in Western Province.

The third key lesson that emerged in the evaluation process is that there are a number of external factors that pose challenges to OkaZHI's efforts, including the long-term sustainability of its successes.

Throughout the evaluation, Zambian colleagues identified a range of factors inhibiting the effectiveness of OkaZHI's efforts in the region. For the most part, these issues are well beyond the scope of OkaZHI's work, such as the lack of equipment, workload issues and the retention of human resources for health in Western province. To the extent that it is possible, however, it will be incumbent upon OkaZHI to not only be aware of these challenges, but to find ways to mitigate these challenges with its Zambian partners, thereby maximizing OkaZHI's contributions:

• While OkaZHI is not able to reduce the workload burden experienced by so many health professionals in Western Province, OkaZHI must continue to take this burden into account in the design and delivery of all courses;

• While OkaZHI does not have the mandate to supply or provide equipment, the lack of technical knowledge for how to use existing and unused equipment also places constraints on local health care workers' capacity to use their knowledge and skills. Supporting the training of equipment technicians in the use and maintenance of existing medical equipment (ECG, incubators, Defibrillators, Ventilators) - while it does not resolve all of the equipment challenges - could contribute to the reduction of equipmentrelated constraints;

The poor retention of human resources perhaps poses the most significant challenge to the long-term sustainability of results to which OkaZHI contributes within Western Province. While OkaZHI may be contributing to the enhanced capacity of Zambian health professionals, who will take this capacity wherever they may go, this finding suggests that its contributions to improved health outcomes, quality and delivery of care in Western Province are ultimately less sustainable. OkaZHI needs to develop some mitigation strategies to secure greater long-term sustainability of its efforts. One such strategy could be drawn from the recommendations made frequently by Zambian colleagues over the course of this evaluation process - namely, the establishment of a 'Train the Trainers' program within Western Province. This would support greater local 6 ownership of OkaZHI's work with a view to the potential eventual devolution of major components of the OkaZHI program. The 'Train the Trainers' program was identified by a range of interviewees as a way to support continuing medical education, while also building greater responsibility and collaboration between Canadian and Zambian practitioners. As such, OkaZHI training could be embedded within local institutions, building institutional capacities, instead of focusing on building capacities at the individual level alone. Other strategies could be developed as part of a sustainability strategy, focused on building institutional-level capacities as a way to complement the individual training delivered by OkaZHI.

If OkaZHI is to benefit from this evaluation process, the evaluation team urges it to take the above factors into account in future planning and programming.

Final points for consideration, detailed below in the report, include the following:

1. Given the numerous factors that pose significant challenges to the long-term sustainability of OkaZHI's efforts in Western Province, OkaZHI could consider developing a sustainability strategy that mitigates these factors (e.g. development of institutional capacity).

2. OkaZHI could establish a policy for how it will review and/or respond to requests for the provision of physical equipment to Zambia.

3. OkaZHI could ensure that each of its courses leaves 'hard copy' educational resources behind for medical staff in Western Province.

4. The regular testing of skills retention amongst OkaZHI course participants could support the systematic monitoring of OkaZHI contributions and its sustainability.

5. The course participant identification process could be reviewed in conjunction with Zambian colleagues to ensure maximal relevance of course material to the participants.

6. OkaZHI could consider integrating communications and administration skills into their program of courses.

7. OkaZHI's evaluation and planning efforts could benefit through further collaboration with the Zambia Ministry of Health, with respect to accessing existing health information data for Western Province, Zambia.

8. OkaZHI could consider expanding into teaching about certain 'silent' conditions.

9. OkaZHI could continue to enhance its impact and effectiveness by strategically building on identified opportunities to collaborate with other Canadian and Zambian based NGOs and Universities.

Conclusion

In conclusion, this evaluation found that OkaZHI, in partnership with its Zambian and Canadian colleagues, has contributed to significant outcomes in Western Province.

Through cooperation and support from OkaZHI, Zambian health professionals working in Western Province have reported being able to offer to their patients better quality of care, reduced referrals, reduced infections, reduced hospital stays, better long-term prognoses, and decreased mortality rates.

Based on the input of Zambian participants and stakeholders, OkaZHI has a clear mandate to continue building on these successes in the coming months and years.

As done in the past, this programming will need to be developed in close partnership with Zambian colleagues - as well as Canadian colleagues working in Zambia - to ensure its relevance, effectiveness, sustainability and ultimate impact in terms of improving health outcomes and quality of care in Western Province, Zambia.