Monitoring, Reporting, Evaluating and Improving Part Three:�Evaluation & Analysis of Policy/Program Surveys and Policy/Guidance/Curriculum Documents ��The evaluation or analysis of the results of policy/program surveys or policy/guidance/curriculum documents is a key step in going from data, monitoring and reporting towards the planned systemic improvements needed to address the findings of such studies. This open web meeting of the FRESH Partners will briefly review two recent large-scale studies in Europe and the United States of America as well as revealing the upcoming SFU/ISHN/UNICEF/UNESCO Fact-Finding Survey & Policy/Curriculum Analysis being launched this month. . �
Common issues or questions that should form part of future evaluation & analysis include:�
Scope & Unit of Analysis (no longer focused on school level)
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
OR p4 - inclusion is one of SHE values in their framework FR p57 – integration of HPS within other ministry policies is major implication of the report | FR p1- has broadened traditional view of health to include “social health” FR p2 - review of federal programs, initiatives, and funding that support SH
| FFS Q6, 7. 8 – over-arching scope of survey is inclusion & equity (like UN Goal #4) School Food & Nutrition as example Multi-intervention program (MIP) Heath & Life Skills education (target 4.7.1) as urgent focus * Different interpretation of whole child approach (renewed FRESH Partners goals) – all ministries, not just ED – leads to BOLO, schooling structures, paths to graduation and underlying society values in document/context analysis (ubuntu, indigenous, personal resource etc.) | UNESCO GEM Report (policy only – notes shift from learning disabilities to all students WHO – asks about overall policy on child and adolescent health but detailed focus on different aspects of health |
Status & Use of MCAs and MIPs (One expanded or several with different sectors, scale up by adding issues
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
OR p2 – makes distinction between individual programs (SHP) and use of MCA (HPS) – over 75% do individual programs, less than 25% do coordinated HPS FR p20 – asks if HPS policy is embedded within several ministries policies (mostly in MOH & MOE) FR p21 - minority of countries require coordinated components FR p57, 58 – future implications/entry points include - routes to school, -inclusive, social environment, ESD | FR p2- DOJ and HHS are biggest part but several departments are doing programs with different perspectives/mandates FR p23 - considerable distance between the idealistic coordination modeled in the WSCC framework and current efforts - multiple agencies play SH roles largely without coordination, resulting in parallel funding streams, technical assistance efforts, etc. prioritize SH in different ways. SR p28 – WC-WS-WC model is not addressed in state laws but SR p31 - Most states’ laws (42 states) address creating inter-agency agreements SR – school nutrition (+ other selected issues & cross-cutting themes) continue to be entry point/policy opportunity | FFS - MCAs and MIPs are asked about as barriers to inclusion and equity FFS Q19, 20 – which of 20 MCAs, 19 MIPs are being used in jurisdiction- is it a just several interventions, a statement or are they coordinated by policy?, do they have current action plans FFS Q21 is there an inter-ministry mechanism to coordinate the MCAs and MIPs? DA – should include lead to analysis of the coordination mechanisms and levers used by countries (better practices, strategies FFS Q22, 23, 24 – is there a MIP on school food & nutrition =, what elements? Inter-ministry roles? UN agency roles? DA – will examine use of LBMs (called “constructs” in USA reports such as life skills, SEL, etc) | WHO p 10 - 64% of countries have national standards for health- promoting schools * need to examine docs to determine if they are “statements/ declarations of support, “macro-policies” with requirements to coordinate, allocate for equity etc, current “action plans” with dates for reporting, budgets, roles of ministries etc. |
Status & Use of Core Components
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p21 – 12/24 countries require schools to have health promotion as part of school goals | SR p6 - State law coverage of the WSCC framework ranged from limited to deep (10 were deep, 21 were broad, 18 were limited2 were weak) SR p39 - constellation of topics covered varied | FFS – asks about each component | UNESCO/Goal 4 – asks about most components in a different way but without articulating a framework or the use of several frameworks * Should analysis encourage countries to strengthen their components more than add more topics/ issues to frameworks that are largely ignored? |
Policy & Priorities
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p31 – 21/24 countries have more than 9 specific health (incl. social, environmental & safety) topics are included in their national HPS policy.
FR p42 – SHE coordinators did not report that their countries have a set of priority issues to be addressed (many did not know) FR p 55 – inclusion of special needs students is a key policy hook, most require physical safety conditions in schools | FR p14 – physical health topics (nutrition, PA, hygiene, etc.) are addressed and implemented in isolation from other federal programs | FFS Q48 – has the jurisdiction defined set of inter-ministry priorities? (“Focus Resources” in previous FRESH acronym) “Effective” in FRESH has also led to narrowing of focus | |
Health & Life Skills Education
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p21 – 16/24 countries require health education of some sort
FR p42-43 - 14/24 have HE instruction in two types of curriculum structures
9/24 have a stand-alone HE curriculum
8/24 promote health as a cross-curriculum competency
19/24 have health taught in several subjects
FR p25 – HP and HE activities are included in extended education activities (required in 4/24 and recommended in 12/24 countries | SR p 8 - Health education is either encouraged or required for all grades in all states’ laws. Forty- six states require health education for all grade levels - Nutrition (40 states) and personal health (44 states) are the most prominent topics - In 25 states, laws address or otherwise incorporate the National Health Education Standards as part of the state health education curriculum. - Only 15 states’ laws explicitly address providing professional development for health education Teachers - Thirty-one states laws have high comprehensiveness (72 percent to 100 percent of topics covered in health education topics | FFS Q13 – is health & life skills education mandatory FFS Q15 – has MOE adopted a cross-curricular competency framework FFS Q26, - curriculum structure used (H, PSD, H&LS, H&PE, etc.) FFS Q27 scope/sequence around a generic set of learning outcomes? Q30 – minimum instructional time? Q31 – is extended education activities used on purpose to promote health, personal and social development? Q35 – has MOE studied supply and demand for H&LS education specialists Q64 – how is effectiveness of Health & life Skills education assessed * Session at WERA-AERA in San Francisco in April 2022 | UNESCO/Goal 4 survey – asks about learning domains linked to all core subjects except health & life skills/well-being OECD and UNISDR analyses of PE and DRR education show how curricula can be analyzed OECD survey on social & emotional learning and UNICEF/World Bank on life skills need to be examined in depth for their findings and implications * need to clarify terms such as life skills (transferable or skills for living in families, communities, relationships), SEL & PSD, ESD *Learning/Behaviour Models (LBMs) need to be anchored in a core subject and evaluated in a specific context * need to reinforce idea that “competencies” are comprised of knowledge, attitudes and skills (not just skills |
Psycho-social Environment
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p 26 – report recommends more work on extended education activities and the social environment of schools | FR p2 - Few federal school health efforts include a focus on employee wellness. SR p20 - Laws in 10 states address staff wellness programs. SR p18 - Nearly all states cover at least some topics related to social and emotional climate in Schools Laws in 40 states address school-wide positive behavioral interventions and supports (PBIS) or other multi-tiered approaches Thirty-five states address using early warning systems for chronic absenteeism, but only twenty-nine states follow up Twenty-four states have comprehensive coverage of social and emotional climate provisions | FFS Q45 – defined set of social environment conditions?
Q46- which ones? | OECD Study on SEL *Global Alliance on SEL *Several UN agencies and organizations investigating SEL * UNESCO Forum on Transformative Ed recommends focus on informal learning *Need to differentiate
|
Physical Environment
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p22 - in most all countries legislation requires that school facilities such as the playground, classrooms, toilets, canteen and corridors be student-friendly, safe, clean and promote hygiene
FR p58 – future implications of the report include: - help schools to think critically if school physical environment promotes socialization and wellbeing | | | |
Student Services
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p25 – referral of students is required in 13/24 countries
FR p25 – support for special needs students is required in 15/24 countries | SR p14 - While 45 states address school-based health services, they do so in different ways. Twenty-six states’ laws address access to a school district physician for health assessments or provide a list of services to be provided to students with specific health needs. Nineteen states’ laws address the development of on-campus health centers or clinics for student use - Every state has a law that requires vaccinations for school-aged children. However, laws in 48 states allow for waivers
SR p 14 - Health screenings vary by grade level and type of screening, with dental screenings addressed in law (24 states) less frequently than vision (41 states) and hearing (34 states) screenings. | FFS Q36 – is there a defined set of minimum student services? Q37 – which services are provided? Q38, 39 – how is school food procured and purchased Q65 – how are the effectiveness student services assessed? | WHO p 10 – 44% of countries have defined a package of adolescent health services * Are the services from different ministries, agencies, professionals coordinated at MOE, SD/agency, school and individual level? |
Contextualize, Implement, Maintain, Scale up & Sustain (IMSS)
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
OR p4 – implementation described as being focused several specific topics addressed by schools
OR p 5 – ten top barriers & facilitators identified by SHE coordinators
FR p28 – 72.7% of countries have guidelines for becoming a HP school, 81.8% have guidance tools for schools
FR p29 – 41% of countries use the SHE manual for HP schools | SR p39 – the vast majority of states cover each of the WSCC components in law in some way. However, the constellation of topics covered by each state varies significantly Comprehensive coverage, however, does not mean these policies are integrated or effective, but simply means that state laws cover a myriad of topics that schools must address Yet the report still recommends an issue as entry point strategy | FFS Q47 – ongoing knowledge exchange with similar contexts?
FFS Q49 – use of situation assessment tool?
FFS Q50 – use of published implementation framework or tool?
FFS Q51 0 start up and ongoing costs done at the outset?
FFS Q56, 57 – traditional and strategic considerations planned for IMSS process? | *There has been no systematic review of any defined MCA, only some MIPs Growing body of research into IMSS processes not being used in practice or policy or guidance documents? * Growing body of knowledge that MCAs are not being truly implemented or sustained |
Build Baseline & Operational Capacities
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p 31 – CPD of teachers (58.3%), psychologists (50.0%), childhood educators (45.8%) and school principals (45.8%) is considered by countries for future development
FR p35 – funding for HP schools is provided by many, but inconsistent sources in almost all countries
FR p57 – future implications of the report include: - training SHE coordinators to use SHE School Manual and its accompanying tools | FR p8 the available formula grant programs that may be used for school health activities. Four agencies are represented: ED, HHS, DOJ, and USDA FR p12 Table 2 presents the title and membership of the 10 interagency groups with work that directly addresses school health topics • An average of 22 states’ laws address professional development-related topics only 20 states’ laws provide for CPD of school nurses. | FFS Q40 has gov’t defined qualifications for other professionals who work in or with schools? FFS Q52 – does each MCA have a budget line? in MOE? In other ministries? FFS Q53 – written roles for ministries in each MCA? FFS Q54 – role of front-line staff defined in each MCA? FFS Q67 – has demand, supply, qualifications and years of experience been assessed in last five years for these categories of personnel? | WHO p 10 – 52% of countries have at least one FT staff person working on adolescent health many have FT on specifc health issues
WHO p10- - 52%of countries monitor implementation of HPS standards
UNESCO/Goal 4 – asks about general qualifications of teachers and their preparation for ESD & GC but not H&Ls education *Devise, checklist, champion, train & hope strategies are not sustainable * Most checklists rarely used |
Coordination
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p13 – characteristics and ministry affiliation of SHE coordinators described – most appointed by MOH, minority by MOE) – about half employed by NGO or research organizations FR p15 – several tasks performed by SHE coordinators FR p25 – 12/24 countries require nurses, social workers, psychologists to be involved in schools HP approach | FR p17- Coordinated grant efforts We identified only a handful of examples across discretionary grant programs where grants were deliberately packaged under a single initiative and/or where two or more agencies collaborated on a single grant effort SR p26 - Forty-two states encourage or require districts to establish school-community partnerships or enter interagency agreements to meet student needs (on specific topics) SR p28 - District-level WSCC/wellness councils are addressed most often (16 states), followed by state-level (10 states), and school-level (4 states). | FFS Q58 – coordinated policy & senior leaders FFS Q59 – designated coordinators for all MCAs FFS Q60 – inter-ministry committee for each MCA? FFS Q61 – inter-sectoral coalition for each MCA? Also previous question on inter-ministry agreements | * In-depth investigation of coordination at different levels in different contexts would reveal much and provide guidance into thise “black box” |
Monitoring, Reporting, Evaluating & Improving (MREI) Practices
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p 34 – 10/24 countries use a HPS “label” for schools, with different criteria and procedures FR p56 – 13/24 countries have a national process to monitor/ evaluate health promoting schools FR p58 – future implications of the report include: - reflect on processes to monitor/ evaluate health promoting schools and on attributing HPS label | | FFS Q62 – what is monitoring & reporting procedure for each MCA?
FFS Q63 is MREI process based on continuous improvement
FFS Q66 – is administrative data (EMIS) used to assess programs? DA – are M&R processes followed up by secondary and cross sector analyses DA - is MRE connected to established ministry improvement procedures | |
Knowledge exchange and development
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p47-48 – variable use of SHE manual and little use of SHE self-assessment tool (like the infrequent use of similar self-assessment tools) | FR p14 - Technical assistance Most of the TA initiatives we identified took the form of formal centers, led via grant, cooperative agreement, or contract by nongovernmental organizations (NGOs). �- considerable duplication of TA efforts not only between agencies, but also within agencies several examples of innovative efforts led by HHS and ED to inspire schools to support school health activities student contests, small school incentives, school recognition programs etc. | DA – participation on KDE by cont3ext, not by issue or region | * In-depth analysis of KDE strategies and products, including increasing use of web to manage mentoring, coaching and community of practice strategies |
Integrate H&S Within, Not Add-On or Mainstream
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
FR p55 – only 12/24 countries require that health professionals be involved in school health promotion
FR p58 – future implications of the report include: - understand why most countries consider that the education professionals’ and psychologists’ CPD is a higher priority among health professionals | SR p14 - Although 36 states’ laws address the availability of school nurses on campus at some time during the school day or some days of the week, only one state (VT) requires a nurse on every campus every day SR p18 - Twenty states require districts to set clear guidelines for referring students to law enforcement, and 23 states encourage specialized training for school resource officers (SROs) | FFS Q54 – role of front-line staff defined in each MCA? FFs Q68 – are inter-ministry partnerships truly reciprocal? FFS Q69 – have strategic benefits and risks in each partnership been assessed? FFS Q70 – are inter-ministry partnerships reviewed and renewed each year? FFS – have teacher beliefs, attitudes, concerns been assessed? | * Use ASCD-ISHN draft “White Paper” to initiate in-depth examination of integration strtategies |
Systems-focused Actions
Europe | USA | SFU-ISHN-UNICEF-UNESCO | Other |
| | FFS Q72 – is there a plan to shift towards a systems-focused approach?
FFS Q73 – has systems approach to school food & nutrition been adopted?
FFS Q74 – which of these incremental, systems-focused actions been used? | WHO p10 - 34% of countries have regular government budget allocations to adolescent health *See FRESH webinar and related research on systems paradigms * Session at IUHPE in May 2022 |
Additional FRESH Partners Items
Draft Educator Statement – please send feedback on draft outline
UNESCO Forum on Transformative Education – please send suggestions