LEVANTE · COLOMBIA PILOT SITE

Pilot data collection

in Colombia

Context for interpreting and analyzing the data collected across the three Colombian application sites

Evaluation Center · Universidad de los Andes, Bogotá

Adaptation, qualitative validation and piloting of the LEVANTE child-development instruments · Funded by the Jacobs Foundation · Led by Stanford University

ABOUT THIS PRESENTATION

Why this context matters for the data

Objective

Provide the operational context needed to correctly interpret and analyze the data collected in Colombia. I will try to emphasize how the instruments were adapted and validated, under what conditions the data collection was done and what interests and incentives the different participants had.

Four sections

1 Expert review & adaptation

2 School enrollment

3 Children & teachers’ data collection

4 Caregivers’ data collection

The pilot’s guiding purpose

​

Evaluate the psychometric properties, validity, age appropriateness, and possibility of internationalizing the measures; as well as defining and reviewing data pipelines.

​

In Colombia, we sought to identify efficient schemes for data collection in schools.

Colombia’s role: LEVANTE pilot site — ages 5–12, three longitudinal rounds.

1

LEVANTE · Pilot data collection in Colombia

AT A GLANCE

LEVANTE in Colombia

Bogotá

Capital · Urban

4 schools baseline 3 (-1) follow-up

2

LEVANTE · Pilot data collection in Colombia

4 schools baseline

HOW THE WORK WAS ORGANIZED

The validation & piloting scheme

A three-stage process.

1

Expert review

Review of translations, instructions and formats; iterative preparation, adjustment and internal verification.

2

Validation

Informal think alouds with children – continuous process.

​

Focus groups with caregivers and teachers.

3

Pilot

Pilot application of the full set of instruments across schools and sites.

3

LEVANTE · Pilot data collection in Colombia

OVERVIEW

Timeline: from adaptation to three rounds

2023 – early 2024

First contacts with Bogotá school principals to invite them to participate in project.

Feb – Apr 2024

Expert review of translations, instructions & formats

​

Children task qualitative validation

May 2024 – May 2025

Teachers and caregivers’ questionnaires qualitative validation

​

Pilot Round 1 – Baseline:

​

Sep – Nov 2025

Pilot Round 2 follow-up and  3 new baseline schools

​

​

Second Children qualitative validation

2026 – 2027

Pilot Round 3

​

​

Extended caregiver outreach campaign

​

4

LEVANTE · Pilot data collection in Colombia

SECTION 1

Expert review & adaptation

Reviewing translations, items and delivery formats, validating them qualitatively, and iterating continuously with the Stanford team to report problems and agree adjustments.

5

LEVANTE · Pilot data collection in Colombia

EXPERT REVIEW & ADAPTATION

Reviewing translations, instructions & formats

6

LEVANTE · Pilot data collection in Colombia

EXPERT REVIEW & ADAPTATION

What we adjusted — child-development instrument

Translation/adaptation

• Corrected wording to Colombian usage
• Changed Spanish words to better match the pictures and the sense of each task
• Challenge: keep difficulty levels

Instructions

• Added examples to avoid misinterpreting instructions
• Aligned written and spoken instructions

Visual display

• Adjusted the screen fit so students see all task information
• Fixed non-visible information needed to complete tasks

7

LEVANTE · Pilot data collection in Colombia

EXPERT REVIEW & ADAPTATION

Qualitative validation with children

Objectives — explore:

• Clarity of the instructions
• Application times
• Ease with which children can respond to the tasks
• Different implementation schemes (computer/tablet, group sizes)

Apr 16–18

2024

47

children one school

5–12

years age range

24

sessions varied group sizes

2+2+2

team facilitators · assistants · Stanford observers

A/V

recorded video · audio · screen

8

LEVANTE · Pilot data collection in Colombia

EXPERT REVIEW & ADAPTATION

Findings: engagement, adjustments & group sizes

What we observed

• High engagement across ages 5–12.
• Children aged 7+ handle the tablet with little or no supervision.
• Response times estimations per task.
• New vocabulary and display issues were detected and reported.

​

​

​

Second validation exercise

With Prof. Nancy Palacio

• Nov 5 - 2025
• Quibdó, Chocó – North-pacific region
• 1 school – 8 kids
• Grades 0 & 1 (5-6 years-old)
• Per kid: 1 facilitator & 2 observers
• Recorded video · audio · screen

​

Main conclusion

Tasks worked as well as with kids from other parts of the country.

Difficulties encountered had already been reported or were equally valid for other regions.

17

LEVANTE · Pilot data collection in Colombia

How many children one facilitator can support

Practical rule:

• grades 0–1 up to 2 children
• grades 2–3 up to 4
• grades 4+ up to 10 (3 facilitators per classroom)

EXPERT REVIEW & ADAPTATION

Caregiver/teacher validation

Caregiver & teacher focus groups

Held in September 2024 to validate content and improve translations of the caregiver and teacher questionnaires.

​

• The caregivers’ questionnaire was particularly time-consuming.
• Several wording adjustment needs were identified.
• For many questions, the response options had to be adapted.
• Some sections or questions were provisionally excluded due to ethical or cultural concerns.

9

LEVANTE · Pilot data collection in Colombia

SECTION 2

School enrollment

The enrollment process and our interactions with participating schools — the critical partners for the whole study.

10

LEVANTE · Pilot data collection in Colombia

SCHOOL ENROLLMENT

How schools were enrolled

1

Reach out to organizations & schools

Partners: public and private schools attending low and middle-low SE level children.

Having participated in previous projects or having other links with the Evaluation Center.

2

Contact with principals

Presentation of objectives and participation requirements: logistics - schedules, space and information needs - caregiver consent forms.

3

Incentives

In return for their participation: results’ reports to compare their students’ development with that of other schools and contexts (no individual results). Also, pedagogical support and activities.

Enrollment strategy: meeting principals and coordinators and adapting to their availability.

11

LEVANTE · Pilot data collection in Colombia

SCHOOL ENROLLMENT

Schools are the critical partner

Provided the school is “on board”, the rest follows

• Coordinate the application
• Collect the consent forms
• Provide the space for application
• Convince caregivers to participate

​

Lessons from the 3 new schools (2025)

Logistics were prepared directly with principals, without coordinators.

​

This raised the consent-return rate and streamlined preparation of logistics.

​

Schools invited caregivers on-site to complete the questionnaires.

​

12

LEVANTE · Pilot data collection in Colombia

SECTION 3

Children & teachers’ data collection

The successive waves and rounds of data collection with children and teachers — the main difficulties encountered and the lessons learned.

14

LEVANTE · Pilot data collection in Colombia

KIDS & TEACHERS

Participation

16

LEVANTE · Pilot data collection in Colombia

Administration was done in-person using tablets for all children and for all teachers except 7 from Bogotá in the 1^st round.

KIDS & TEACHERS

Retention by school

% Children retained across both rounds

Overall retention ≈ 82%

​

15% attrition if grade 6 is excluded.

​

Counting the two schools that dropped out after Year-1 platform issues attrition rises to ≈ 30%.

16

LEVANTE · Pilot data collection in Colombia

< 20%

child attrition

KIDS & TEACHERS

Measures per task

Double and single measures per task

Bundles of tasks were used to distribute the tasks among kids and keep the administration time below 50 minutes.

​

Because bundles had to be used not all kids were administered the same tasks.

​

To produce useful information in the school reports, mathematics and language tasks were predominantly administered.

​

Up to this point many tasks have few longitudinal data. We intend to improve those numbers in 2026.

16

LEVANTE · Pilot data collection in Colombia

KIDS & TEACHERS

Difficulties & lessons learned

Difficulties

• Some data got lost, particularly in the 1^st round:
• Platform / dashboard were unstable
• Connectivity — tasks are internet-dependent; schools lack devices and good internet. 40–50 children answering at once.
• Application failures — some children restarted or could not finish.
• New necessary improvements: i.e. speaker button missing on some tasks; younger children (5–6) sometimes need audio playback.

Lessons learned for 2^nd round

• Upload all children’s data before the sessions.
• Provide a fallback connection — a Starlink device now removes dependence on school internet.
• Keep a robust participant registration & tracking system — critical to find data losses and longitudinal follow-up.

18

LEVANTE · Pilot data collection in Colombia

KIDS & TEACHERS

Teachers: process & lessons

Straightforward once the school is on

When the school commits, teachers reliably complete the requirements of the project.

Bogotá — by email

Contacted through the school by email (from Nov 2024).

​

At School B all 7 teachers responded per mail; other urban schools followed in later waves.

In person

Questionnaires applied on-site during the school visit (San Vicente del Caguán, Bogotá and Chiquinquirá 2025).

​

Rural teachers sometimes need help using the tablet.

19

LEVANTE · Pilot data collection in Colombia

SECTION 4

Caregivers’ data collection

The process with caregivers — the difficulties we faced, the lessons learned, and the adjustments made.

20

LEVANTE · Pilot data collection in Colombia

CAREGIVERS

Collection schemes & response pattern

How caregivers were reached

22

LEVANTE · Pilot data collection in Colombia

% Response rate by school and setting

Rural in-person sessions (often linked to mandatory school events) far outperform remote urban contact.

CAREGIVERS

The hardest challenge

Why caregivers are hard to reach

• Urban (Bogotá): caregivers ask to be contacted by email and telephone but often do not respond, despite reminders from the team and the school.
• Rural: low familiarity with tablets, reading and internet make it challenging.
• Survey length & redundancy: reported at over an hour in early administrations; streamlined by Stanford for 2025 but still ~50 minutes on average.
• About one quarter of submitted caregiver surveys contained only partial information (first module only).

​

~50 min

avg. survey length (2025)

~25%

partial submissions

Take-away: caregiver coverage is uneven — do not read it as a representative of the children sample without accounting for mode and setting.

21

LEVANTE · Pilot data collection in Colombia

CAREGIVERS

Incentives & the 2026 outreach campaign

Chiquinquirá rural

School kit (notebooks, pens, pencils) as a lower-key incentive.

2026

School kit (notebooks, pens, pencils)

23

LEVANTE · Pilot data collection in Colombia

READING THE DATA

Consider to interpret the Colombia data

Staggered timing — Rounds differ by site

   Bogotá May 2024/ Sept 2025

   San Vicente del Caguán Oct 2024/ Oct 2025

   Chiquinquirá Oct 2024 - 3 new Oct 2025

Mixed modes —

Children always in person.

Teachers in person and by email.

Caregivers by email (urban) vs. in person (rural).

Expect mode effects.

Evolving instruments —

2024: 6 bundles (2 for pre-readers)

2025: new bundles + adaptive tasks

Caregiver survey streamlined between rounds.

Year-1 platform caveats —

Dashboard instability, restarts and unsaved/incomplete records — treat some Year-1 data with caution.

Caregiver coverage skew —

 Urban ≈ 36% vs. rural ≈ 61%; ~25% partial.

Caregiver data is uneven across settings.

Attrition & constraints — Two schools dropped after Year-1 issues; attrition ≈ 21% (≈15% excl. grade 6). Individual-level data not authorized by the Steering Committee.

24

LEVANTE · Pilot data collection in Colombia

Thank you

Pilot data collection in Colombia · LEVANTE

Evaluation Center · Universidad de los Andes

Funded by the Jacobs Foundation

25

LEVANTE · Pilot data collection in Colombia