1 of 22

Keeping Students Safe: Educator and Student Perspectives on Monitoring Applications (MA) Functionalities

https://research.ced.ncsu.edu/k12studentsafetymonitoring

Florence Martin, PhD (PI)

Co-PIs - Rebekah Davis PhD, Sejal Foxx, PhD, WeiChao Wang, PhD (Co-PIs)

2 of 22

Project Team

3 of 22

Research Practice Partnership

4 of 22

Project Goals

Examine what and how monitoring applications were implemented in North Carolina districts
Examine the impact of the implementation of the monitoring software applications to protect students from threats and harms
Examine the data collected by the software, its access, and data privacy protection
Examine the monitoring needs of North Carolina districts and charter schools
Provide policy recommendations at the State and local levels

5 of 22

Monitoring Applications (MAs)

Tools that “provide teachers and schools with the ability to filter web content, monitor students’ search engine queries and browsing history, view students’ email, messaging, and social media content, view the contents of their screens in real time, and other monitoring functionality” (Grant-Chapman et al., 2021)

6 of 22

Context of the Study

Focused on schools that used MAs during the Pandemic (i.e., 2020-2022)
MA data provided by NCDPI
Student, teacher, administrators were recruited, surveyed, and interviewed about their experience with MAs during that time
Challenges in sampling and response rate

7 of 22

Top MAs

MA	Number
Gaggle	134
GoGuardian	81
Securly	15
LineWize	11
Lightspeed	8

8 of 22

Monitoring Functionalities

Functionality	Description
Behavioral analytics (n=14)	Analyzing, summarizing, and reporting student behavioral data to inform future stakeholder decisions
Content filtering (n=13)	Preventing student access to harmful, distracting, inappropriate, or otherwise unapproved media
Third party app integration (n=13)	Integrating MAs with other hardware, applications, platforms, or services
Recording student activity (n=12)	Logging and storing online activity and engagement for analysis, compliance, etc.
Machine learning (n=9)	Using trained computer algorithms to aid behavioral analysis and surveillance
Student assessment (n=7)	Actively collecting data from students via surveys to measure learning and improve curricula
Escalation protocol (n=7)	A formal implementation of behavioral analytics: Screening notable cases for false positives and following a chain of command depending on priority, severity, etc.

9 of 22

Features of the Most Commonly Used Apps for MAs

10 of 22

Survey Findings

11 of 22

Student Experiences

Perceived Helpfulness & Educational Impact

87% felt apps didn't aid in resolving issues
6.8% reported blocking of educational resources
Mixed feedback on self-harm assistance: The majority found no help or were unaware

12 of 22

Teacher Experiences

Real-World Impact & Incident Response

20% teachers reported they did not have an incidents.
19% teachers reported that they are uncertain about impact of the MAs
67% reported no examples of resolving bullying or self-harm cases
Notable positive feedback: Timely alerts leading to prompt intervention
Annually the number of Incidents ranged from 2 to 100

Teachers who were trained had a higher perception of MAs in preventing inappropriate content

13 of 22

Technology Facilitators and Directors

Technology Directors and Facilitators had higher perceptions that MAs worked well for protecting students from inappropriate content, cyberbullying and self-harm.

14 of 22

Perceptions on MA Use towards Preventing Online Harms

Mean (SD)	Student (n=73)	Teacher (n=47)	Tech Facilitator (n=14)	Tech Director (n=18)
Inappropriate Content*	3.01 (1.22)	3.02 (1.54)	3.57 (0.90)	3.61 (1.30)
Cyberbullying*	2.23 (1.09)	2.94 (1.97)	2.93 (1.22)	3.39 (1.01)
Self-harm*	2.11 (1.13)	2.88 (2.07)	3.07 (1.28)	3.94 (1.08)

Research question 1 focused on the participant perception of monitoring applications in preventing inappropriate content, cyberbullying, and self-harm. The Kruskal-Walli test showed a statistically significant difference among students, teachers, technology facilitators and technology directors on the perceptions of MAs towards preventing cyberbullying (χ2 = 29.468, df = 3, P = 1.785e-06), self-harm (χ2 = 42.267, df = 3, P = 3.522e-09), and inappropriate content (χ2 = 27.348, df = 3, P = 4.976e-06). Post hoc pairwise comparisons indicated that all pairs of groups exhibited significant differences between each other. All the ratings from technology directors are the highest, followed by that of technology facilitators, then teachers, and the students rated the MA harm prevention the lowest. The findings suggest that students, teachers, technology facilitators, and technology directors each have unique views on the effectiveness of MAs in preventing harm, which may be attributed to their varying levels of proximity to using MAs.

15 of 22

Benefits, Challenges and Concerns

Teachers - Benefits

Monitoring - Learning Process

Monitoring - Test Cheating

Students - Benefits

Content filtering

Avoid cheating

Teachers - Challenges and Concerns

Implementation difficulties

Block useful sites

Wrong flags

No notification of inappropriate activities

Distracting student

Peer information

Outside school activities

Students - Challenges and Concerns

Restrict access to useful educational applications

(e.g., math games)

Not being informed

Outside school activities

16 of 22

Interview Findings

27 educational staff, including administrators, technology facilitators, technology directors, school counselors, and teachers participated in the structured interviews

17 of 22

Theme 1:

Seeing the parts vs the whole

Monitoring applications are large, complex, and spread across various devices used by a variety of stakeholders for their own broadly aligned purposes.

Role Responsibility
Collaboration
Accessing and Sharing Data

18 of 22

Theme 2:

Tension within a humanistic aspect

MA itself is an algorithm needing little human participation, the target populations were fully functioning humans with various levels of social, emotional, cognitive, and behavioral patterns

Ethical Considerations
Impact on Users

19 of 22

Theme 3:

Easier to pick up, more difficult to master

All roles indicated that, generally speaking, onboarding to new MAs during the Pandemic was not an unreasonably difficult process. It isn’t to say that it was frictionless–teachers, TFs, and TDs especially bore the brunt of training and troubleshooting, and some wished that more and better training did exist

Training
Technology Needs
Future Needs

20 of 22

Policy Considerations

21 of 22

Ways to better Implement MAs

Decision makers should strive for some first-hand experiences with the MAs to understand their practical application and effectiveness. (Such as asking for a hands-on demonstration.)
Encourage schools and districts to develop effective involvement strategies to engage various stakeholders (including parents, students, and teachers) in the decision making process.
Require that districts and schools conduct awareness programs repeatedly for users to increase transparency and knowledge about monitoring application program features.
Ensure teachers and staff receive training in selected MAs to increase fidelity of usage and understanding of privacy issues.
Gather feedback regularly to mitigate the perception gaps between stakeholders and gain insights on the effectiveness and areas for improvement.

22 of 22

Website & Infographic

https://research.ced.ncsu.edu/k12studentsafetymonitoring
Infographic 1 ( go.ncsu.edu/monitoringig)
Infographic 2 (handout)

References

Grant-Chapman, H., Laird, E., & Venzke, C. (2021). Student activity monitoring software: Research insights and recommendations (pp. 1–7). Center for Democracy & Technology.