Generic Sensors
Kenneth R. Christiansen - @kennethrohde
Hi! I am Kenneth
@kennethrohde
Our team mission
Move the Web Platform forward
What are sensors good for?
Motion sensors sense speed, direction, or state of rest.
...and much more!
What are sensors good for?
Environmental sensors monitor environmental properties around you.
...and much more!
The starting point
The starting point: DeviceOrientation/Motion has many issues
The landscape
The idea
Make is easy to create new sensors specs and refer to the main spec for all the details
The idea
"The goal of the Generic Sensor API is to promote consistency across sensor APIs, enable advanced use cases thanks to performant low-level APIs, and increase the pace at which new sensors can be exposed to the Web by simplifying the specification and implementation processes."
https://w3c.github.io/sensors/
Simple API
Base API shown as TypeScript
Any API inherits from this
Simple API
Absolute and Relative Orientation Sensors just extend
Grouping
Sensors are either low-level, derived or fusion (used data from multiple)
Sensor
Accelerometer
Gyroscope
Ambient Light
OrientationSensor
...
AbsoluteOrientationSensor
RelativeOrientationSensor
LinearAccelerationSensor
Life cycle
Simple life cycle - cannot fingerprint or know if options can be followed before activation
Security threads
Mitigation
Checked by security experts from Intel, Google and the W3C TAG
Introduction to motion sensors
The low level sensors include
*Not a motion sensor as such, but often used with
Accelerometer
A raw accelerometer sensor measures changes in acceleration in 3 different directions, but is affected by gravity
The Accelerometer sensor is an inertial-frame sensor
Accelerometer
Accelerometers are less useful by themselves and often take part in other fusion sensors, but they do have some purposes like registering shakes, steps and the like
Often for such use-cases the developer is interested in the linear acceleration which is the acceleration without the gravity, called gravity compensation; or the developer is interested in the isolated gravity, in order to know the gravity vector, which can be useful for some kinds of sensor fusion like creating a magnetic compass.
Accelerometer and filters
High and low pass filters
For acceleration, you usually care about the big changes and want to avoid noise, like the gravity, thus a high-pass filter can help isolate the linear acceleration and a low-pass filter can help isolate the gravity. A low-pass filter can thus be useful for measuring a tilt.
Unfortunately, any high-pass filter or low-pass filter introduces a delay, which may or may not be acceptable.
Accelerometer and filters
Low pass: allows low frequencies to pass
High pass: allow high frequencies to pass
A common way to create a low-pass filter is to only use a percentage of the latest value and take the rest from the existing value.
In a way this means that the filter remembers common values and thus smoothes out uncommon values which most often are a result of noise. As it uses a big percentage of the existing value, this solution introduces a delay in registering the actual events.
Example low-pass filter
Accelerometer values
Notice, as accelerometers report acceleration, you need to integrate to get velocity:
v = ∫a×∂t
And again to get position:
x = ∫v×∂t
An integral creates drift, and a double integral amplifies that:
So position from an accelerometer is very imprecise and not very useful.
How an accelerometer work
Most accelerometers are MEMS (Micro Electro-Mechanical Systems)
Gyroscope
A gyroscope senses angular velocity, relative to itself, thus it measures its own rotation, using an inertial force called the Coriolis effect.
Gyroscopes oscillate at relative high frequency in order to measure this and are thus one of the most power hungry motion sensors.
This also means that they can easily be affected by other vibrations, like a vibration (rumble) motor or speaker on the same device.
How a gyroscope work
When a mass is moving in a particular direction with a particular velocity and when an external angular rate will be applied (green arrow) a force will occur (blue and red arrow), which will cause perpendicular displacement of the mass.
So similar to the accelerometer, this displacement will cause change in capacitance which will be measured, processed and it will correspond to a particular angular rate.
Gyroscope values
In order to get rotation (angle) from a gyroscope, which senses angular velocity, you need to perform a single integration.
f ≡ frequency
∫cos(2π×ft)) = (1/(2π×f)) × sin(2π×ft)
But be aware that integration turns noise into drift.
As we see above, the integration gets a 1/f outside, meaning that high frequency (f) noise disappears with integration, i.e. a noise of frequency will drop by a factor of a 100, but a very low frequency will be amplified, meaning the gyroscope will drift over time.
Magnetometer
Magnetometers are magnetic field sensors, which means that without any strong magnetic influence close by, it will sense the Earth’s magnetic field, which more or less points in the direction of North, but not true North.
Magnetometers are very sensitive to outside influence, like anything on a table that has been slightly magnetized, and it is even affected by other things inside a device, though the device manufacturer can compensate for this somewhat. In practise though, these sensors work quite well for most common use-cases.
The most common use-case for magnetometers are as part of sensor fusion, in order to generate an Orientation Sensor which is stationary to the Earth plane, or a compass, which is basically the former with corrections to the declination depending on geolocation position, such that it points to the true North.
How a Magnetometer work
Actually almost 90% of the sensors on the market use the Hall Effect
If we bring some magnetic field near a conductive plate with current flowing through it we would disturb the straight flow and the electrons would deflect to one side of the plate.
That means if we put a meter now between these two sides we will get some voltage which depends from the magnetic field strength and its direction.
High-level sensors
Each sensor has its own issues, such as noise and drift, and often need some kind of compensation using input from a different sensor.
Put another way, one sensor might not be very precise on its own, but...
The sum of multiple sensory input can be much more stable.
High-level sensors
Unfortunately, sensors require power, and the more sensors and the higher the measuring frequency, higher the power consumption (remember gyroscope)
It is always important to consider the minimum set of sensors which solves a task satisfactory.
As many devices today can do certain kinds of sensor fusion in hardware, it most often makes sense to use these from a power and performance point of view.
High-level sensors
Sensor type | Underlying physical sensors |
Accelerometer, Gyroscope, MUST NOT USE Magnetometer | |
Accelerometer, Magnetometer, AND (when present) Gyroscope | |
Accelerometer, Magnetometer, MUST NOT USE Gyroscope | |
Accelerometer, AND EITHER Gyroscope OR Magnetometer |
Below is a list of fusion sensors and what sensors they usually are made up of:
Example: Absolute Orientation Sensor
The Absolute Orientation Sensor, is one of the common use-cases of a magnetometer
Represents an orientation stationary (fixed to the magnetic field vector and gravity vector) to the Earth plane.
As the reference frame of an absolute orientation sensor is stationary, it is not useful as a controller for say a driving game on a phone, as it would not allow you to move around, even slightly or slowly, without affecting your driving direction.
How is an Absolute Orientation Sensor implemented
By taking the cross product between the the magnetic field vector and gravity vector (isolated from accelerometer), we get a vector which points East on the ground plane, using the right hand rule.
Now if we take the cross product between the gravity vector and the newly found East vector, the resulting vector will point in the northern direction towards the Earth’s magnetic field.
How is an Absolute Orientation Sensor implemented
The illustration below represents the case where the device is at rest and y-axis points towards the North.
The reading from the Magnetometer is {x: 0, y: 11, z: -16} and Accelerometer reports {x: 0.11, y: 0.07, z: 9.81} acceleration.
The uG is a unit vector representing the gravity
uB represents magnetic field vector
uE = uB × uG points East.
uN = uG × uE points to the northern direction.
That means an Absolute Orientation Sensor is a fusion sensor of the Magnetometer and the Accelerometer, and potentially the Gyroscope for better isolated gravity.
Example: Geomagnetic Orientation Sensor
A Geomagnetic Orientation Sensor, is like a Absolute Orientation Sensor, but doesn’t use the Gyroscope, which means it uses less power.
This also means that it is more sensitive to shakes and movement.
As the main use-case for a Geomagnetic Orientation Sensor is to create a compass, or use compass direction within a mapping application, this is not much of a problem since people usually hold the device steady for these use-cases.
The actual heading (N, S, E, W) can be found by adjusting the rotation vector with the local declination compensation calculated from the current geolocation position.
Example: Gravity and Linear Acceleration
On most sensor hubs,
gravity is isolated from the accelerometer using the gyroscope,
linear acceleration is isolated by removing the gravity, from the accelerometer
This avoids the delay which low and high pass filters introduce.
Example: Relative Orientation Sensor
You can think of a relative orientation sensor as a gyroscope adjusted to the gravity vector
One way of doing this is using a Kalman filter or complementary filter.
As a complementary filter yields quite good results and is easy to implement in hardware, this is a common solution.
Complementary filter
A complementary filter can be thought of as a low-pass filter and high-pass filter in one, complementing the gyroscope values with the accelerometer values:
θn = α × (θn-1 + ω × ∂t) + (1.0 - α) × a
With α being the weight constant, a the acceleration from accelerometer, ω the angular velocity from gyroscope and ∂t being the time between measurements.
A common value for α is 0.98, which means that 98% of the weight lays on the gyroscope measurements.
Complementary filter implementation
When rotating you get most from the gyro
When still you get the existing values with no difference in rotation, so the gravity wins
Fun: a device-adjusting, relative orientation sensor
From the example, we notice that the alpha represented the initial heading orientation.
We also know that this heading might drift over time due to being based on the gyroscope.
Fun: a device-adjusting, relative orientation sensor
In some situations you might want the orientation to drift towards your current position.
This can be useful for a controller inside a virtual reality environment, where you want a car to follow the heading of your controller, but you might move and turn around while playing.
That would more or less work like driving a real car.
Fun: a device-adjusting, relative orientation sensor
Changing one line in the above accomplishes that.
�
This example shows how useful manual fusion can be at times.
With the above 2% of the alpha consists of the value 0. Thus, when the device is being held more or less steady, the heading will move towards 0, meaning being adjusted to your current device position and not positioned according to the surroundings.
What exists today?
Browser support?
Implementation
Implementation in Chrome
48
It can take less than
100 lines of code
to add a new
sensor type
Implemented under feature flags
Implementation in Chrome
49
Implementation in Chrome
50
Enabling Generic Sensors for an origin
51
<!-- Origin Trial Token, feature = Generic Sensors, origin = https://example.org, expires ="2018-01-18" -->�<meta http-equiv="origin-trial" data-feature="Generic Sensors" data-expires="2018-01-18" content="..."> |
<script type="module">�import { Gyroscope, AbsoluteOrientationSensor } from './motion-sensors.js';�</script> |
Polyfill works great on iOS and Android
52
Embedded applications
54
Demos and articles
Some fun stuff
Let's look at a video!
This is the Daydream
controller
What about
Smart devices?
That is quite tiny
Thanks for listening!
You can participate!
67