EAS: Identify Test Cases
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

View only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZAA
1
ItemTest CaseDescriptionEnergy DataTracingKernel statisticsMetricsOwnerStatus
2
1homescreenAndroid home screen is one very useful case for two purposes, the first one purpose is to measure the power data with a "quiet" system, and we can analyze the task placement to find if there have any optimization for small task placement; the second purpose is to use homescreen power data as baseline value and we can compare it for other scenarios (audio, video, uibench, etc...) so we can calculate the delta value, the delta value can be used to demonstrate how much energy consumption introduced by "active" cases.

As result, Android home screen also is one fundamental case for power optimization.
Yes: Power(HS)sched_wake
sched_wake_new
sched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
sched_energy_calc_duration
sched_sugov_util
Interrupt number
EAS path statistics
CPU frequency average: Use tracing event "cpu_frequency" and LISA class: trace.analysis.frequency.plotCPUFrequencies()

CPU cfs util (max, min, mean)
CPU summed util (max, min, mean) : Need add one event in schedutil governor to sum util for cfs + RT.

DDR frequency average: suggest to fix frequency

GPU frequency average: suggest to fix frequency

Task placement percentage (big, LITTLE)
Leo
3
2AudioAudio case is very important for very small workload scenario, and it can be used to check the EAS has good result for tasks placed on single CPU or not. Due usually we have audio island, so AP is usually seldom waken up and very limited tasks are running for audio playback; so we try to use one CPU with minimum CPU OPP.Yes:
Power(Audio) - Power(HS)
sched_wake
sched_wake_new
sched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
sched_energy_calc_duration
sched_sugov_util
Leo
4
3VideoVideo playback case is important for middle workload scenario, video playback have multiple threads and the utilization value is not quite high (< ~50 on Hikey960). For this case, the challenge is to spread middle workload tasks to multiple CPUs to avoid to increase frequency to high OPPs.Yessched_wake
sched_wake_new
sched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
sched_energy_calc_duration
sched_sugov_util
CPU freq (max, min, mean)
CPU cfs util (max, min, mean)
CPU summed util (max, min, mean)

Task placement percentage (big, LITTLE)

FPS: Need check how to gather this info???
Leo
5
4CameraCamera is important for heavy workload scenario for power saving. The camera case may have many tasks for the software pipeline (e.g. > 50 tasks), and they may introduce heavy CPU utilization; but camera case is not like the responsiness cases which require very strictly scheduling latency, after meet the FPS requirement, the camera case needs to optimize the power as possible, otherwise this case can consume much higher power than the expected due the tasks are easily to migrated onto big CPU with high OPPs.

96boards is absent to support camera case due the complex drivers implementations (e.g. bus driver, sensor driver, display/GPU driver, vpu driver, ISP driver, etc). We have one rt-app case: https://github.com/ARM-software/workload-automation/blob/master/wlauto/workloads/rt_app/use_cases/camera-long.json; using this case we can do power profiling but has no sense for "FPS".
sched_wake
sched_wake_new
sched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
sched_energy_calc_duration
sched_sugov_util
CPU freq (max, min, mean)
CPU cfs util (max, min, mean)
CPU summed util (max, min, mean)

Task placement percentage (big, LITTLE)

FPS: need check how to gather this info???
6
5uibench:: InflatingListActivityUibench is UI related cases, we can classify into two types: the first type is related with interactive cases, the second type is related with sustainable cases. The both cases need to use "Janks" to measure the responsiveness, but for unsustainable cases we also need take care for power optimization (e.g. can improve the battery life for gaming case.)

uibench:: InflatingListActivity is to scroll the screen for the list widget; gather "janks" statistics to measure if can meet the performance requirement. Janks% = Jank frames / Total frames.
Yessched_switch
sched_wakeup
sched_wakeup_new
sched_overutilized
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
Janks metrics:
FPS
frame_count
janks
janks% = (janks / frame_count)%
not_at_vsync
frame_time_90percentile
frame_time_95percentile
frame_time_99percentile

CPU Freqeuncy Analysis: Plot CPU and Cluster frequency and calculate the average frequency; this is based on LISA two classes: trace.analysis.frequency.plotCPUFrequencies() and trace.analysis.frequency.plotClusterFrequencies();

CPU Util and Task Placement Analysis: Plot CPU uti and corresponding capacity value; Plot the task placement on every CPU and context switching on every CPU;

Task scheduling Latency: Analyze GPU related tasks scheduling latency so we can get to know these tasks have long scheduling latency or not.

Task runtime: Analyze task runtime, this can get to know the workload and check if the runtime is long so introduce the performance issue; for this case we usually need think how to adjust CPU capacity for these tasks (or set boost margin).

Below tasks will be analyzed:
SurfaceFlinger
RenderThread
hwuiTask1
Binder
mali-cmar-backe
mali-hist-dump
mali-mem-purge

GPU Frequency Analysis: for first step, suggest we fix GPU frequency?
DDR Frequency Analysis: for first step, suggest we fix DDR frequency?

Note: We can use systrace to analyze performance UI at early phase, the systrace can gather kernel and Android layer trace events and it's friendly display with Chrome: chrome can display every frame with Green/Yellow/Red colors and give altert type (e.g. which view has big latency or scheduling latency).
7
6uibench:: ActivityTransitionClick screen to zoom in and zoom out picture;

This case has very small workload but need ensure the performance can be boosted within very short time; so the difficult thing is to avoid too high boost margin which may introduce over "inflation" for other cases.
8
7uibench:: InvalidateActivityReverse color for small grids, invalidate and redraw view;

This case have very heavy workload for GPU and it continously to render the whole screen; so it need to spread tasks and increase OPP to meet the performance requirement. On the other hand, this case is a good case to optimize the power data for GPU sustainable case with good performance result.
9
8Jankbench::list_viewhttps://github.com/ARM-software/lisa/blob/master/ipynb/examples/android/benchmarks/Android_Jankbench.ipynb

Most devices today refresh their screens 60 times a second. If there’s an animation or transition running, or the user is scrolling, applications need to match the device’s refresh rate and put up a new picture, or frame, for each of those screen refreshes. When one fails to meet this budget - 16.6 ms, the frame rate drops, and the content judders on screen. This is often referred to as jank, and it negatively impacts the user's experience. jankbench is used to count the jank frames for different types of activities: list and image list view fling, text render, text editing, etc.
10
9Jankbench::image_list_view
11
10Jankbench::shadow_grid
12
11Jankbench::low_hitrate_text
13
12Jankbench::high_hitrate_text
14
13Jankbench::edit_text
15
14Galleryfling
Launch gallery with hundreds pictures and fling pictures
16
15RecentflingOpen recents screen and fling recent tasks list
17
16EmailflingOpen email app and fling emails list
18
17BrowserflingOpen browser icecat and fling the web page
19
launch appLaunch time is an important index for user experience. We measure the 2nd times of launch not the 1st time, because the 1st time launch includes many initialization or how-to-use steps. After installation, we will launch the APP 1st time and finish the initialization and tutorial (if it has), then force-stop the APP, clean the cache. Then, we measure the 2nd time launching.Kevin
20
18Vellamohttps://github.com/ARM-software/workload-automation/blob/next/wa/workloads/vellamo/__init__.py#L31

Android benchmark designed by Qualcomm.
Vellamo began as a mobile web benchmarking tool that today has expanded to include three primary chapters. The Browser Chapter evaluates mobile web browser performance, the Multicore chapter measures the synergy of multiple CPU cores, and the Metal Chapter measures the CPU subsystem performance of mobile processors. Through click-and-go test suites, organized by chapter, Vellamo is designed to evaluate: UX, 3D graphics, and memory read/write and peak bandwidth performance, and much more!
21
19Temple RunTemple RUN is a gaming case to check the power data and performance; for performance we can measure the FPS or janks, this case is a good case due it has a big thread and this big thread heavily impacts the final performance result. So we can use this case to verify EAS can meet the performance requirement for the case with a one big thread and mutiple small threads.YESsched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
CPU freq (max, min, mean)
CPU cfs util (max, min, mean)
CPU summed util (max, min, mean)

Task placement percentage (big, LITTLE)
Yingshiuan
22
20Beach buggy blitzBeach Buggy Blitz is a racing game. At the begging screen, it has a camera from 3-rd person angle rotating at a fixed speed. So that the scenario is periodical and reproducible and it also contains various type of scen, like ocean, waterfall. Those various scens can be used to test how fast the dvfs governor adjusts to the proper frequency.YESsched_switch
sched_load_avg_cpu
sched_load_avg_task
cpu_capacity
cpu_frequency
cpu_idle
CPU freq (max, min, mean)
CPU cfs util (max, min, mean)
CPU summed util (max, min, mean)

Task placement percentage (big, LITTLE)
Yingshiuan
23
21
24
22
25
23
26
24
27
25
28
26
29
27
30
28
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...