10/14/2025. Changed from model --> agent, based on description in blog post. https://www.theagi.company/blog/android-world

askui AndroidVisionAgent

askui AndroidVisionAgent, Claude 4.5 Sonnet + Claude 4.0 Sonnet

[10/6/2025]: 78.4 --> 91.4; details here
[09/19/2025]: Updated score from 63.0 --> 78.4. For this run, we used GPT-5 for reasoning in combination with Gemini-2.5-Pro for acting, while continuing to rely on a hybrid method of the Accessibility (A11y) tree plus corresponding screenshots to provide both structural and visual context to the agent.

Llama 4-scout, Gemini 2.5 pro, GPT-5 nano

https://minitap.ai/benchmark

[12/16/2025] 84.5% -> 91.4%
[10/1/2025]: 77.6% -> 84.5%
[09/12/2025]: Updated score from 74.1% --> 77.6%
[08/19/2025]: Initial trajectory labeling issue (MarkorCreateNoteFromClipboard) has been corrected by the authors. They provide Discord support for setup issues.
[08/18/2025]: Repo has been reported broken by some users (not independently verified). Some trajectories (e.g., MarkorCreateNoteFromClipboard, BrowserMaze) are labeled as successful by the authors but contain incorrect actions

https://work.aliyun.com/alimail/openLinks/downloadMimeMetaDiskBigAttach?id=netdiskid%3Av001%3Afile%3ADzzzzzzNqZy%3BG8EhE7UV7U9nk2t2O6opysa%2BgRf5Zm6eTqXqtVApm1nDD8W%2BxnLAgdDsJsOSlxfXTx44NU7qRM%2BkJMuIA0smltxGWspXGX2Ox228yqp07rNX%2FIm2QehcGJBgHwUs0f81cz2sX76WNxvxnhpZnqmxiYxHNHRd2hybrPLi2DRQ8bM5ORhI%2BtGG%2B7n%2BcxdY1Ji0

https://hcompai.github.io/android-world-traces/

Agent is based on https://github.com/hcompai/surfer-h-cli - same architecture, but with modifications to prompt and action space

Sonnet 4.5 + Sonnet 4

https://github.com/babelcloud/android_world_benchmark/tree/main/GBOX/trajectory

GBOX uses Claude code as the agent with GBOX mcp. Link to report -> https://github.com/babelcloud/android_world_benchmark/blob/main/GBOX/report.md

[9/26/2025]: Updated score from 75.8 --> 80.2 This is an updated version of AutoGLM-Mobile. In this version, we added more pretraining data and trained the VLM at full image resolution.

Gemini-2.5-Pro+UI-TARS-1.5

Qwen2.5-VL-72B + Qwen2.5-VL-7B

https://github.com/k2-agent/k2-agent/blob/main/androidworld-trajectories.zip

K²-Agent, a hierarchical approach that self-evolves a Qwen2.5-VL-72B for high-level planning and post-trains a Qwen2.5-VL-7B for low-level execution.
Our GitHub repository, which contains a detailed description of our approach, can be found here: https://github.com/k2-agent/k2-agent.

Make further post-training based on the GUI-Owl-32b model. Optimize the memory and knowledge module of the MobileUse framework.

Gemini 2.5 Computer Use

Gemini 2.5 Computer Use

https://github.com/inclusionAI/UI-Venus/blob/main/vis_androidworld/UI-Venus-androidworld.zip

https://github.com/inclusionAI/UI-Venus/blob/main/vis_androidworld/UI-Venus-androidworld.zip

GPT-4o, OS-Atlas-Pro 4B, Qwen2-VL-2B-Instruct

Qwen2-VL-2B (fine-tuned)

A relatively simple prompt involving one LLM /VLM call

A multi-agent architecture involving several LLM calls and a protocol to coordinate the various agents or an LLM wrapped into an advanced agent with memory, subgoal planning, etc.