1 of 27

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

Qiushi Sun

qiushisun.github.io

@qiushi_sun

Oral Presentation

Page 1

2 of 27

Automating daily computer tasks

OpenAI Operator

Computer-Using Agents

Daily Computer Use

Mobile GUI Agent

Page 2

3 of 27

Safety Concerns

Agent safety research is behind agent deployment!

Page 3

4 of 27

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

Qiushi Sun*, Mukai Li*, Zhoumianze Liu*, Zhihui Xie*, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li , Zichen Ding, Qi Liu, Zhiyong Wu , Zhuosheng Zhang, Ben Kao, Lingpeng Kong

Also Best Paper Award, AIWILD @ ICLR2026

Oral Presentation

Page 4

5 of 27

Safety Issues

Mobile GUI Agents

Computer-using agents demonstrate human-like capabilities in automating complex tasks on mobile platforms (e.g., booking, messaging, scheduling).

Significant Safety Concerns!

This autonomy also introduces severe, underexplored risks:

Privacy Violations: Leaking sensitive personal data.

Offensive Content: Sending inappropriate messages or memes.

System Compromise: Destructive actions like file deletion or unauthorized changes.

…

More autonomy means more ways to go wrong.

Page 5

6 of 27

Safety Issues

Key Challenge: The Detection Gap

Even benign user instructions can trigger unsafe agent trajectories.

Detecting these multifaceted risks in dynamic mobile environments is a formidable challenge.

We lack:

Realistic, comprehensive environment + benchmark, with Compatibility

Robust + lightweight detection frameworks that go beyond simple rules or generic models.

A benign instruction is no guarantee of safe execution.

Page 6

7 of 27

Infra for Safety Research

MobileRisk-Live

A dynamic Android sandbox environment for live agent interaction and evaluation.

Key Feature: Captures not only GUI observations (screenshots, a11ytree) but also a deep System State Trace.

Page 7

8 of 27

Infra for Safety Research

System State Trace

Includes:

Aggregated file-system information: file sizes, owner UIDs/GIDs, modification timestamps
SHA-256 over sensitive system directories
Network activity, permission changes, and installed packages.

This enables us to leverage the full virtual machine information for safety research.

Page 8

9 of 27

Infra for Safety Research

Why Both Live Sandbox and Frozen Benchmark?

Live sandbox is ideal for realism, but hard to evaluate on

Agent capability confounds trajectory generation — can't isolate safety patterns

Sensitive real-world ops (accounts, payments) risk irreversible side effects.

Stochastic apps (e.g., TikTok , YouTube feeds) break reproducibility

Anti-virtualization in production apps (e.g., Meituan , Ele.me many Chinese super-apps) refuse to run or degrade functionality inside emulators

Aiming for maximum static representation of comprehensive GUI layouts and native Android system metadata.

Page 9

10 of 27

Infra + Benchmark for Safety Research

MobileRisk

A static benchmark of “frozen” agent trajectories derived from MobileRisk-Live.

Provides fine-grained, multi-level annotations:

Trajectory-level (Safe/Unsafe)
Step-level (Localization of first unsafe step)
Risk Category (10 types, e.g., Privacy, Destructive)

Enables reproducible and isolated study of safety issues.

Page 10

11 of 27

Benchmark Annotation

We deliberately combined emulator and real-device collection to achieve coverage that pure-emulator benchmarks cannot

Event-Driven Trajectory Data Collection

- Raw touch events are captured via adb getevent on Android devices.

- Operations (Tap, Swipe, or Long-press) are identified based on displacement and duration thresholds.

Page 11

12 of 27

Android Safety Detection

In previous safety detection works (e.g., VLM as a Judge): We mainly focused on multimodal information.

From the VM side:

We haven’t fully utilized the information beneath Android apps

there’s a wealth of runtime data and APIs that can greatly support safety research.

From the agent side:

We often ignore the GUI agent’s actions.

Look beneath the GUI — and at the agent's own actions.

Page 12

13 of 27

OS-Sentinel

Core Idea: A Hybrid Validation Approach

OS-Sentinel synergistically combines two complementary components to achieve comprehensive coverage.

Hybrid Architecture:

Formal Verifier (Rule-Based), analyzes deterministic, system-level changes.

Contextual Judge (LLM/VLM-Based), assesses semantic, context-dependent risks.

Final Verdict

Verdict_Unsafe = Formal_Verifier ∨ Contextual_Judge

(A trajectory is flagged as unsafe if either component detects a risk)

Page 13

14 of 27

OS-Sentinel: Formal Verifier

Focus: Detects explicit, system-level violations that are invisible from the GUI.

Input: System State Trace

Detection Mechanisms:

System State Integrity Monitoring

Computes hashes of file system metadata at each step.
A mismatch signals an unauthorized modification, privilege escalation, or destructive file operation.

Sensitive Keyword & Pattern Matching

Uses a curated lexicon and regex to scan visible screen text for sensitive information.
Detects leakage of: Passwords, Credit Card Numbers, PII, etc.

Strength: Provides a rigorous, auditable, and deterministic safety bottomline.

Page 14

15 of 27

OS-Sentinel: Contextual Judge

Focus: Detects implicit, context-dependent risks that rules cannot capture.

Input: GUI Observations (Screenshots / a11ytree) & Agent Actions

Detection Mechanism:

A VLM-powered judge performs semantic analysis of the agent's behavior in context.

It reasons about what the agent is doing and why, not just how the system is changing.

Risks Captured:

Privacy Violations: e.g., Agent sharing sensitive bank info in a chat.
Harmful/Offensive Content: e.g., Agent sending an inappropriate meme (as in Fig. 1).
Inappropriate UI Manipulation.
…

Actions that are contextually unsafe but do not violate system files.

Strength: Captures the semantics of agent behavior.

Page 15

16 of 27

OS-Sentinel

Page 16

17 of 27

OS-Sentinel

Step-level Detection

Page 17

18 of 27

OS-Sentinel

Page 18

19 of 27

OS-Sentinel

One unsafe step makes the whole trajectory unsafe.

Page 19

20 of 27

OS-Sentinel

Baselines.

Rule-based Verifier (adapted from MobileSafetyBench)
VLM/LLM-as-a-Judge

MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

Observations.

Consecutive Partition τ into non-overlapping windows of W steps; flag τ unsafe if any window fires.
Sampled Uniformly sample N representative transitions — adapts to backbone context length.

Modes. Step-level & Trajectory-level

Page 20

21 of 27

OS-Sentinel

Good results :)

Surpass predominant baselines across model backbones

Avg time cost: ~66ms

+10–30% over baselines, at ~66 ms per check.

Page 21

22 of 27

OS-Sentinel

Online v.s. Offline: Similar trend

Closeness of trajectory-level detection results between MobileRisk-Live and MobileRisk.

Frozen trajectories track the live sandbox — reproducible by design.

Page 22

23 of 27

Baselines are lopsided; OS-Sentinel is balanced across all 10 risk categories

OS-Sentinel

Why this matters: in real deployment, you don't get to choose which category of risk shows up. Balanced coverage is what a safety guard actually needs.

Page 23

24 of 27

https://qiushisun.github.io/OS-Sentinel-Home/

Page 24

25 of 27

SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, ACL 2024

Seminal works on Computer-Using Agents

Foundation Models

Data

Frameworks

Frontier App.

OS-ATLAS: A Foundation Action Model for Generalist GUI Agents , ICLR 2025 Spotlight

OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis , ACL 2025

Breaking the Data Barrier -- Building GUI Agents Through Task Generalization, COLM 2025

AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant , ACL 2025

OS-MAP: How Far Can Computer Use Agents Go in Breadth and Depth?

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows, ICLR 2026

Evaluation

Safety

OS-Sentinel : Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows , ACL 2026

OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent, ACL 2026

ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data, ICLR 2026 Oral

OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis

Page 25

26 of 27

Thanks for listening!

Contact: qiushisun@connect.hku.hk

Page 26

27 of 27