�CS60055: Ubiquitous Computing: Background Topics 

Location, Gesture and Activity Sensing

The Fundamentals of Motion Tracking

INDIAN INSTITUTE OF TECHNOLOGY

KHARAGPUR

Sandip Chakraborty

sandipc@cse.iitkgp.ac.in

​

Department of Computer Science and Engineering

We are living in the edge of wearables ...

Indian Institute of Technology Kharagpur

Wearables -- Head to Toe

Indian Institute of Technology Kharagpur

Range of Devices

Indian Institute of Technology Kharagpur

Sensors on Wearables

IMU �Altimeter�Bio-acoustic�Blood Pressure�Brightness�Camera�ECG�EDA�EMG�Fiber Optic Sensors�Compass�

GPS

GSR

Humidity

Magnetometer

IR Proximity

IR Temperature 

LED 

Glucometer 

Pressure 

Microphone 

Piezoelectric 

RFID

Spectrometer 

Thermometer 

Textile sensors 

Ultrasound 

Weight 

Optical sensor 

mmWave sensor 

Electrodes 

Inductive wire 

IR Photo Reflective 

Indian Institute of Technology Kharagpur

Our First Problem ...

vs

with a

Indian Institute of Technology Kharagpur

IMU / Inertial Sensors

• Accelerometer, Gyroscope, and sometime Magnetometer 
• Measures 
• Specific force of a body 
• Angular Rate 
• Orientation 
• Detects the linear acceleration and the rotation rate (angular rate) 
• Also used for measuring the orientation of a device (one of the primary use-cases for wearables) 
• Largely used for maneuver in a 3D space and navigation purpose 

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Acceleration relative to an observer who is in free fall (relative to an inertial frame of reference)
• Different from coordinate acceleration
• Accelerometer measures the proper acceleration

​

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Acceleration relative to an observer who is in free fall (relative to an inertial frame of reference)
• Different from coordinate acceleration
• Accelerometer measures the proper acceleration
• What acceleration value would be reported by an �accelerometer placed on a table at our classroom? 

​

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Acceleration relative to an observer who is in free fall (relative to an inertial frame of reference)
• Different from coordinate acceleration
• Accelerometer measures the proper acceleration
• What acceleration value would be reported by an �accelerometer placed on a table at our classroom? 

​

0 m/s²

0 m/s²

9.8 m/s²

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Acceleration relative to an observer who is in free fall (relative to an inertial frame of reference)
• Different from coordinate acceleration
• Accelerometer measures the proper acceleration
• What acceleration value would be reported by an �accelerometer placed on a table at our classroom? 
• A free-fall accelerometer will read zero acceleration!

​

0 m/s²

0 m/s²

9.8 m/s²

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Acceleration relative to an observer who is in free fall (relative to an inertial frame of reference)
• Different from coordinate acceleration
• Accelerometer measures the proper acceleration 

Assume that the ball is in rest

Indian Institute of Technology Kharagpur

Motion and the Frame of Reference

• Rest Frame: The coordinate system in which the particle is at rest 
• Proper acceleration: Rate of change of velocity of a body in its instantaneous rest frame 
• Accelerometer measures the proper acceleration 

Can we represent the motion of every object with this principle?

Indian Institute of Technology Kharagpur

Consider the following scenario ...

With an accelerometer mounted on Mr. Bean's headphone, can you detect that he is enjoying the music? 

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

“A little reflection will show that the law of the equality of the inertial and the gravitational mass is equivalent to the assertion that the acceleration imparted to a body by a gravitational field is independent of the nature of the body. For Newton’s equation in motion in a gravitational field, written out in full, it is: 

(inertial mass).(acceleration) = (intensity of the gravitational field).(Gravitational mass) 

It is only when there is numerical equality between the inertial and gravitational mass that the acceleration is independent of the �nature of the body.” 

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

A falling object behaves exactly the same on a planet or in an equivalent accelerating frame of reference.

Image Source: Wikipedia

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

• In its simplified form: The effects of gravity on an object are indistinguishable from its acceleration 

Y

X

Z

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

• In its simplified form: The effects of gravity on an object are indistinguishable from its acceleration 

Y

X

Z

g

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

• In its simplified form: The effects of gravity on an object are indistinguishable from its acceleration 

Y

X

Z

g

We call this reference frame as the local inertial frame of the object

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

• In its simplified form: The effects of gravity on an object are indistinguishable from its acceleration 

Y

X

Z

g

We call this reference frame as the local inertial frame of the object

Accelerometer measures the acceleration with respect to this local inertial frame

Indian Institute of Technology Kharagpur

Equivalence Principle and Its Impact

• In its simplified form: The effects of gravity on an object are indistinguishable from its acceleration 

Y

X

Z

g

We call this reference frame as the local inertial frame of the object

Accelerometer measures the acceleration with respect to this local inertial frame

The "gravity offset" must be subtracted to obtain the actual acceleration, when the object moves on the earth's surface

Indian Institute of Technology Kharagpur

Impact of Rotation

Indian Institute of Technology Kharagpur

Impact of Rotation

Y

X

Z

The gravity offset is towards the y-axis

Indian Institute of Technology Kharagpur

Impact of Rotation

Y

X

Z

X

Z

Y

The gravity offset gets different impact on different axes -- how to subtract the gravity offset?

The local inertial frame changes due to the rotation

Indian Institute of Technology Kharagpur

Impact of Rotation

Y

X

Z

X

Z

Y

Y = g

Y ≠ g

Indian Institute of Technology Kharagpur

Angle Transformation

Y

X

Z

Indian Institute of Technology Kharagpur

Angle Transformation

Y

X

Z

y

x

z

We need to figure out the components of the gravity offset on each of the components of the local inertial frame

Indian Institute of Technology Kharagpur

Orientation Update

Image source: Wikipedia

Indian Institute of Technology Kharagpur

Six Degrees of Freedom

Image Source: Wikipedia

The three angles, roll, pitch and yaw are called the Euler angles to denote the orientation of an object, with respect to a frame of reference

Indian Institute of Technology Kharagpur

Euler Angle and Angle Transformation

Y

X

Z

y

x

z

Φ

Φ = Roll

Θ = Pitch

Ψ = Yaw

Step 1: Rotate the Z-Y plane on the roll angle to adjust the Z axis (rotation across X-axis)

Indian Institute of Technology Kharagpur

Euler Angle and Angle Transformation

Y

X

Z

y

z

Φ

z'

Φ

x

y'

Φ = Roll

Θ = Pitch

Ψ = Yaw

When you try to adjust the Z-axis, the Y-axis also observes an additional rotation on the roll angle

Indian Institute of Technology Kharagpur

Euler Angle and Angle Transformation

Y

X

Z

y

z

Φ

z'

x

y'

Φ = Roll

Θ = Pitch

Ψ = Yaw

Step 2: Rotate the X-Z plane on the pitch angle to adjust the X-axis (Rotation across Y-axis)

Θ

Indian Institute of Technology Kharagpur

Euler Angle and Angle Transformation

Y

X

Z

y

z

Φ

z'

x

y'

Φ = Roll

Θ = Pitch

Ψ = Yaw

Θ

The Y-axis needs to be adjusted now!

Indian Institute of Technology Kharagpur

Euler Angle and Angle Transformation

Y

X

Z

y

z

Φ

z'

x

y'

Φ = Roll

Θ = Pitch

Ψ = Yaw

Θ

Step 3: Rotate the Y-X plane for the yaw angle with respect to the Z-axis

Ψ

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• Consider two reference frames R₁ and R₂. Let R₁ be the fixed reference frame of the observer and R₂ be the local reference frame of the object. To figure out the orientation of the object, we need to rotate the object to transform R₂ to R₁
• The rotations follow right hand thumb rules, and always in the right (clockwise) direction
• The first rotation with respect to any one of the three axes x, y or z
• The second rotation along with a different axis
• The third rotation either w.r.t. the first axis or the leftover axis
• We have 12 possible combinations in this way: 1-2-1, 1-3-1, 2-1-2, 2-3-2, 3-1-3, 3-2-3, 1-2-3, 1-3-2, 2-3-1, 2-1-3, 3-1-2, 3-2-1
• The 3-2-1 (yaw, pitch, roll) is one of the most popular ones, and used widely in aircraft navigation

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• The rotational matrices across the three axes (corresponds to the three rotations is given by): 

We can replace α with the corresponding roll, pitch or the yaw angles depending on the rotations. For example, for 3-2-1 rotations, the angles corresponding to M1, M2, and M3 would be the roll, the pitch, and the yaw angles

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• Subsequently, the Direction Cosine Matrix (DCM) is computed as the multiplication of the three rotational matrices across the three axes reading from right two left.
• Ex: The DCM for the 2-3-1 Euler angles will be evaluated as follows:

​

​

• Similarly, the DCM for a 3-2-1 Euler angle will look as follows. 

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• Let [a_x, a_y, a_z] be the acceleration reading on the tilted frame T(X'Y'Z')
• Acceleration on the level frame L(XYZ) is [0 0 g] 
• We consider 3-2-1 Euler angle (Yaw Pitch Roll)
• Can we observe gravity's impact on the three axes of the accelerometer, and based on the acceleration values, measure the Euler angles? 

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• Let [a_x, a_y, a_z] be the acceleration reading on the tilted frame T(X'Y'Z')
• Acceleration on the level frame L(XYZ) is [0 0 g] 
• We consider 3-2-1 Euler angle (Yaw Pitch Roll)
• Can we observe gravity's impact on the three axes of the accelerometer, and based on the acceleration values, measure the Euler angles?
• This will help us to compute the orientation of the object when in static 

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• However, using acceleration, we can only compute the roll and the pitch angles, but now the yaw angle
•   As gravity is always downwards (z-axis), a rotation across the z-axis will not have any impact on the accelerometer readings

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• However, using acceleration, we can only compute the roll and the pitch angles, but now the yaw angle
•   As gravity is always downwards (z-axis), a rotation across the z-axis will not have any impact on the accelerometer readings
• So, we ignore the yaw angle

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• Let us assume the yaw angle to be zero
• We obtain the following DCM: 

​

​

​

• Now, we can have the kinematic equations as follows:

​

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• We obtain the following equations by multiplying the matrices:

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• We obtain the following equations by multiplying the matrices:

​

​

​

• Finally, we obtain the roll and the pitch angles as follows:

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• But how do we compute the yaw angle?

Indian Institute of Technology Kharagpur

Computing the Euler Angles

• But how do we compute the yaw angle?
• We need a second modality: Gyroscope

Indian Institute of Technology Kharagpur

Gyroscopes

• Measures the angular velocity and the orientation of the object
• Angular velocity is measured based on the rotation rate with respect to an inertial frame
• We already know the roll and the pitch angle from the�accelerometer
• The measured angular velocities across the three axes �are then used to compute the orientation �(the yaw angle)

Image Source: Wikipedia

Indian Institute of Technology Kharagpur

Putting them all together ...

• We know,

​

​

​

• Accordingly, we can compute the roll and the pitch angles as;

Indian Institute of Technology Kharagpur

Putting them all together ...

• Gyroscope measures the angular velocity [ω_x, ω_y, ω_z]
• Multiplying these values with the time (δ_t) will give the angular movement within these time unit, which is indeed the three angles -- roll, pitch and yaw
• However, the orientation angles calculated with this method return noisy values (due to minute calculation of the time)
• So, IMU (Inertial Measurement Units – combine acclerometer, gyroscope, and magnetometer) uses a Kalman Filter or Complimentary filter based approach to reduce the measured noises with the help of roll and pitch values computed from the accelerometer
• Indeed, it also takes help of the yaw computed from magnetometer as 

Indian Institute of Technology Kharagpur

How Can We Use Gyroscope for Self-Balancing?

Indian Institute of Technology Kharagpur