Introduction to Hardware Acceleration

Droidcon Paris 2015

Cyril Vlaminck

Software engineer

at AdBuddiz

https://github.com/cyr62110

https://goo.gl/i2KDTF

Hardware Acceleration

Introducted with Honeycomb (API level 11)

Enabled by default if Target API >= 14

Two drawing model for View and Drawable

  • Traditional model
  • Hardware accelerated model

Q before slide: Who is using Hardware acceleration?

Well I bet that everyone is using it, if you compile your application with a target API of 14, then hardware acceleration is enabled by default.

Hardware acceleration was introduced with Honeycomb to accelerate the drawing of View and Drawable.

There is now 2 different drawing model for View and Drawable:

Traditional drawing model

Software-based drawing model

  • Drawing operation are handled by the CPU on the main thread
  • Canvas operations will draw on a native bitmap

But let start with the traditional drawing model,

If you are not targeting ICS as you Target API, your application will use this model even on newer version of Android.

2D drawing operation on View are handled by the processor on the main thread.

To ensure 60 fps, you will need you drawing code + buisness logic to run within 16ms.

Otherwise, you will start to see Android complaining in the logcat and starting to dropping frame.

The software drawing pipeline rely on a two phase process:

The invalidation phase occurs when you change your UI. ex. setText() on a TextView will cause an invalidation.

It will mark all the view that are dirty and that need to be redrawn during the draw phase.

Child view can be redrawn even if they content or visual apparence has not change.

Ex. If you have a parent layout and a text view on top of it. Calling setBackgroundColor on the background layout will cause a

invalidation of both the layout and the text view as the text view is semi-transparent and needs to render the new background color.

More information -> Skia & SkCanvas <- native API used below Canvas operation.

Traditional drawing model

Drawing pipeline

  • Invalidate the hierarchie
  • Draw the hierarchie

Invalidation

Hello

Droidcon

<LinearLayout>

<TextView />

</LinearLayout>

Invalidation

Update the text in the TextView.

  • Invalidate the TextView
  • Mark it as needed to be redrawn

Is it enough for the rendering to be correct?

Hello

Droidcon

Invalidation

NO

We need also the parent background to be redrawn

Hello

Droidcon

Bonjour

Droidcon

Invalidation

Invalidate the TextView

Invalidate the LinearLayout

Draw the LinearLayout

Draw the TextView

Bonjour

Droidcon

Bonjour

Droidcon

Hardware accelerated drawing model

OpenGL-based drawing model

  • Drawing operation are handled by the GPU
  • GLES20Canvas backed by an OpenGL display list - RenderNode
  • All drawing operation are stored in display lists to be rendered latter

With hardware acceleration enabled, most drawing operations are handled by the GPU.

The canvas you will get in your onDraw, will mostly be an GLES20Canvas since OpenGL ES 2 is available on all newer hardware.

With hardware acceleration, every view is backed with a display list or a RenderNode in Android API.

The GLES20Canvas is streaming the drawing operation to a native component called OpenGLRenderer.

The OpenGLRenderer is in charge of transforming all your drawing operation in Glop (structure containing everything to issue the OpenGL drawing call).

Then this Glop is used by the RenderState to issue the proper OpenGL ES API calls that will be stored in the display list.

So even if the drawing operations are handled by the GPU and stored in a display list, they are still delivered synchronously to the GPU using OpenGL API when

you call the method on Canvas. If you are still doing to much in onDraw, you can still cause Android to skip frame.

At this point, you are probably wandering why we still need the older software-based drawing model. Well, just because not all drawing operation are supported

by the hardware acceleration. So the question is “What is supported?”

Documentation: in plateform_frameworks_base

Display list

Layout XML

<LinearLayout>

<ListView />

<Button />

</LinearLayout>

To better understand what is a display list, lets take a basic sample.

Here we have a linear layout with two views inside, a list view and a button.

As I mentionned, every hardware accelerated view is backed with its own display list,

so the ListView has one and the button has one.

The display list of our layout will be very simple and will consist in drawing the display list of its two children.

Display list

LinearLayout display list

Layout XML

<LinearLayout>

<ListView />

<Button />

</LinearLayout>

DrawDisplayList(ListView)

DrawDisplayList(Button)

To better understand what is a display list, lets take a basic sample.

Here we have a linear layout with two views inside, a list view and a button.

As I mentionned, every hardware accelerated view is backed with its own display list,

so the ListView has one and the button has one.

The display list of our layout will be very simple and will consist in drawing the display list of its two children.

Hardware accelerated drawing model

Drawing pipeline

  • Invalidate the hierarchie
  • Record and update display list
  • Draw display list

Display list

What will happen if we change the alpha on the ListView?

What will happen if we change the alpha on the ListView.

Display list

What will happen if we change the alpha on the ListView?

Software drawing model

  • Invalidation of the ListView
  • Invalidation of the LinearLayout
  • LinearLayout draw()
  • ListView draw()

setAlpha() will cause a full redraw of your Layout and its subviews.

In traditional software drawing model.

When setAlpha is called on a standard view (View may handle alpha by themself but it does not enter in our exemple),

invalidateViewProperty is called -> On non hardware accelerated, this will call invalidate

the ListView by itself will be invalidated, which also causes its parent to be invalidated.

So both the listview and the layout drawing will be executed.

Display list

What will happen if we change the alpha on the ListView?

Software drawing model

  • Invalidation of the ListView
  • Invalidation of the LinearLayout
  • LinearLayout draw()
  • ListView draw()

setAlpha() will cause a full redraw of your Layout and its subviews.

Hardware accelerated drawing model

  • Invalidation of the LinearLayout
  • LinearLayout draw() update the display list.

setAlpha() will only cause the LinearLayout to update its display list.

ListView draw() will not be called.

With hardware acceleration, some transformation operation are fully optimized to use the display list.

setAlpha still call invalidateViewProperty(). On hardware accelerated view, it will only cause the view parent to invalidate itself.

The LinearLayout draw() will be called to updates its display list.

As the transformation operation does not need the display list of the ListView to be modified, the ListView draw() will not be called.

Display list

LinearLayout display list

Layout XML

<LinearLayout>

<ListView

android:alpha=”0.5” />

<Button />

</LinearLayout>

SaveLayerAlpha(0.5)

DrawDisplayList(ListView)

Restore

DrawDisplayList(Button)

Display list

What transformations are optimized?

  • alpha
  • rotation / rotationX / rotationY
  • scaleX / scaleY
  • pivotX / pivotY
  • translationX / translationY / translationZ
  • offsetTopAndBottom / offsetLeftAndRight
  • elevation

What is supported?

All platform views supports hardware acceleration out-of-the-box.

More and more 2D drawing operations are supported

clipPath

18

drawTextOnPath

16

drawArc

21

drawRoundedRect

21

setAntiAlias

16-18

drawVertices

X

setLinearText

X

setPathEffect

X

setShadowLayer

X

Xfermode

X

Hardware acceleration is supported by all views in the system.

If your application is only using views provided by the platform, you can enable hardware acceleration wihout any worry.

For those who are using custom views, not all drawing operations are supported.

Since Honeycomb came out with the hardware accelaration support, more and more 2D drawing operations are added version after version. Also some operations are not available at all. You really should take a look to the full list of operations available on the documentation when you start using hardware acceleration.

Depending on the API you want to support, you may need to turn off hardware acceleration or modify your drawing code.

For information, if you call an unsupported operation while in hardware acceleration, the call will just be ignored and everything will be drawn as if you have never called the method.

Enable/Disable Hardware Acceleration

For a specific window

For a specific view (only disable)

For all your application

For a specific activity

<application

android:hardwareAccelerated=”true”

...>

<activity

android:hardwareAccelerated=”true”

...>

view.setLayerType(

LAYER_TYPE_SOFTWARE, null);

getWindow().setFlags(

WindowManager.LayoutParams.HARDWARE_ACCELERATED,

WindowManager.LayoutParams.HARDWARE_ACCELERATED);

Since hardware acceleration is enabled by default if your application targets API 14, you may need to disable it for custom view using unsupported

operation on some device.

You can enable/disable Hardware acceleration on three level:

In an hardware accelerated, the hardware acceleration can be desactivated ponctually on a view.

On an window using the sofware-based drawing model, you cannot activate ponctually hardware acceleration on a View.

Hardware acceleration must be enabled

Check for Hardware acceleration

On a view:

Returns true if the Window is hardware accelerated.

On a canvas:

Returns true if the canvas is hardware accelerated.

view.isHardwareAccelerated()

canvas.isHardwareAccelerated()

Due to the aforementioned limitations, it can be interesting to check if the hardware acceleration is enabled.

Android provides us two for doing it:

One thing to note: When you call isHardwareAccelerated on a view, it will return true if the WINDOW is hardware accelerated.

It will return true if:

The canvas you get in onDraw MAY not be hardware accelerated. For example, if you or the frameworks draws on a Bitmap for caching purpose.

Exemple of usage: Disable animation on complex view for device not supporting HA.

If you need to adapt your drawing code at runtime, you MUST only check on the canvas.

Let’s accelerate a Custom View

Enable/Disable Hardware Acceleration

For a specific window

For a specific view (only disable)

For all your application

For a specific activity

<application

android:hardwareAccelerated=”true”

...>

<activity

android:hardwareAccelerated=”true”

...>

view.setLayerType(

LAYER_TYPE_SOFTWARE, null);

getWindow().setFlags(

WindowManager.LayoutParams.HARDWARE_ACCELERATED,

WindowManager.LayoutParams.HARDWARE_ACCELERATED);

Let’s come back one second on this slide.

For the first three method, we are changing the hardware accelerated property on something:

This seems logical. We are playing with hardware acceleration so we modify the hardware accelerated property.

Enable/Disable Hardware Acceleration

For a specific view

For all your application

For a specific activity

For a specific window

<application

android:hardwareAccelerated=”true”

...>

<activity

android:hardwareAccelerated=”true”

...>

view.setLayerType(

LAYER_TYPE_SOFTWARE, null);

getWindow().setFlags(

WindowManager.LayoutParams.HARDWARE_ACCELRATED,

WindowManager.LayoutParams.HARDWARE_ACCELRATED);

But wait, I said that hardware acceleration can be disabled on a view by modifying something called layer type.

Why this is not setHardwareAccelerated(false) or some kinds of flag like the window?

Ok, that was a poor introduction, but let’s talk about LayerType

LayerType

Introduced with Honeycomb (API Level 11)

Create a layer where the view will be drawn

Allow to specify specific blending mode, transparancy and color filter when the layer is composed. ex: Black and White view

2 layer types:

  • Sofware
  • Hardware

Setting the LayerType

By code:

Or in your layout XML:

view.setLayerType(LAYER_TYPE_SOFTWARE, null); //LAYER_TYPE_HARDWARE, LAYER_TYPE_NONE

<View

android:layerType=”sofware” /> <!-- hardware or none -->

Layer type can be changed dynamically. You can go from a software to hardware to none.

One thing to consider is that each time you change the layer type, the layer need to recreated causing

you onDraw() to be called. Changing to often may cause severe performance issue.

LAYER_TYPE_SOFTWARE

DO NOT DISABLE HARDWARE ACCELERATION ON THE VIEW

Create a bitmap to be used as drawing cache

  • onDraw() will draw on the bitmap
  • The bitmap will be uploaded as an OpenGL texture
  • The display list of the view will consist to render the drawing cache.

Invalidation is costly

Alpha, … have the same behavior as if your canvas was hardware accelerated

First thing, if your Window is hardware accelerated, this does not disable hardware acceleration for your view.

If you check View.isHardwareAccelerated on a view with this layer type, this will still return true.

But if you check Canvas.isHardwareAccelerated during your onDraw, it will return false.

In the fact, the layer type software

If your view is invalidated, first the drawing cache will be considered dirty and will be redrawn.

Then as your drawing cache has been updated, we need to update the display list and the OpenGL texture.

This cause a lot of data transfert between your CPU and you GPU, drawing performance are largely impacted when you are using

a software layer.

LAYER_TYPE_HARDWARE

DO NOT ENABLE HARDWARE ACCELERATION ON A VIEW

Create an OpenGL texture to be used as drawing cache

  • onDraw() will draw on the texture

Reduce the number of OpenGL operation required to render the view

  • May help to animate view with complex display list at 60fps.

This DOES NOT ENABLE HARDWARE ACCELERATION for a view. Worst, if you set this layer type to view attached to a non hardware accelerated Window,

this will have the same as LAYER_TYPE_SOFTWARE.

Bibliographie

Android Open Source Project

  • View, Canvas, HardwareCanvas, RenderNode, …
  • library HWUI: OpenGLRenderer, RenderState, Glop, …
  • library Skia: SkCanvas, …

Android Developers website:

  • Hardware acceleration guide

Question