1 of 33

1963: Dance With Me

Edmund Chan, Theophilus Ko, Qiqi Li, Tiffany Quon

Self-sponsored project, ENPH 479

February 24th, 2020

2 of 33

The team

Edmund Chan

Qiqi Li

Tiffany Quon

Theophilus Ko

3 of 33

Outline

3

  • Background
  • Summary of Concept
    • Demo
  • Visualizations
    • You Versus You
    • You Versus Previous
    • You Versus All
  • Technical Optimizations
  • User Feedback�

4 of 33

How do we feel about the data we generate and share?

4

5 of 33

Tools

5

  • Kinect for Xbox One
    • Motion sensor
    • Tracks up to six skeletons and body silhouettes�
  • Processing
    • Programming language for visual arts
    • Good for generating images pixel by pixel�

The Kinect motion sensor.

6 of 33

Dance With Me is an interactive art installation that visualizes movement similarities

6

7 of 33

Dance With Me is an interactive art installation that visualizes movement similarities for you,

7

8 of 33

Dance With Me is an interactive art installation that visualizes movement similarities for you, for you and the previous user,

8

9 of 33

Dance With Me is an interactive art installation that visualizes similarities for you, for you and the previous user, and for you and the past 25 users.

9

10 of 33

Perhaps the data we generate and share can connect us with others

10

11 of 33

Perhaps the data we generate and share can connect us with others :-)

11

12 of 33

Perhaps the data we generate and share can connect us with others :-)?

12

13 of 33

The Plan (i.e. Summary of concept)

13

  • Interactive art installation exploring relationships with our digital footprint

​

  • Track body movements to visualize the connections with
    • Yourself
    • Another person
    • The community

​

  • Fun experience involving self reflection with data and uniquely body-generated artwork

14 of 33

14

Video

https://drive.google.com/file/d/1tXG7wZ4PPT3PCPfmK2c1MS6VLQdPgLTt/

15 of 33

15

16 of 33

Key Components of Dance With Me

16

  • You Versus You�
  • You Versus Previous�
  • You Versus All�
  • Technical optimizations�

17 of 33

You Versus You

17

Visualizes movement similarities for you

Colouring a heat map for frequency of visits in space

18 of 33

You Versus You

18

  • How often do we visit certain places?
    • Heat map

​

  • Assign colour to each pixel according to the total number of visits at the associated area

​

  • Blank canvas that can be painted by visiting different locations

​

  • Simple “intro” to visualizations

​

​

19 of 33

You Versus Previous

19

Visualizes movement similarities for you and the previous user

Mapping physical overlaps to shapes and joint orientations to colour

20 of 33

You Versus Previous

20

  • Checks the user’s relative joint positions; compares and looks for matches with the previous user�
  • Takes number of matches, and map to some hue value between 0 and 360

Example of You Versus Previous.

21 of 33

You Versus All

21

Visualizes movement similarities for you and the past 25 users

Displaying people who were at the same place at the same time

22 of 33

You Versus All

22

​

  • At each frame, looks at user's right hand.�
  • Looks through database of past 25 users to find someone whose right hand was at a similar position in space at the same frame.�
  • If someone is found, their silhouette is displayed behind the current user and stamped on the output image.

Example of You Versus All.

23 of 33

Example

23

Live image

Output image

24 of 33

Colour

24

  • Used to distinguish silhouettes from each other.�
  • RGB colour is determined by body position.�
    • Right arm → R value
    • Left arm → G value
    • Legs → B value

Examples of position mapping to color.

25 of 33

Technical Optimizations

25

  • Storage of data in RAM
  • Reading and writing data to file
  • Cleaning image artifacts

26 of 33

Storage of Data in RAM

26

  • Reduced lag caused by several operations such as adjusting pixel brightness
  • A necessary procedure to read and write to file quickly
  • All algorithms that require access to each pixel’s data is sped up.

Screen columns

Screen rows

Stored in RAM as a 1D array

27 of 33

Reading and Writing Data to File

27

  • 80 seconds at 16FPS = 1200 frames → 261MB of depth data

Strategy

Write time (seconds)

Read time (seconds)

Iterative loop

63

31

Bytes read/write function

<1

<1

File read and write times for 1200 frames of depth data

Frame 1

Frame 2

Storage of data in a 1D array for fast file read and write:

28 of 33

Image Noise Filtering

28

  • Image noise appearing on screen, see image on right.
  • Median filtering algorithm applied, see image below.

Image noise in the form of speckles surrounding the silhouette

Image above is taken from Mathworks’ MDBUTMF function installation page.

29 of 33

Image Noise Filtering

29

30 of 33

Summary

30

  • Created three visualizations that each allow a user to generate artwork.
    • You Versus You: heat map of user's location over time.
    • You Versus Previous: body position similarities between current and previous user.
    • You Versus All: past users at same place at same time as current user.�
  • Decreased file read/write times and smoothed silhouettes for better UX.

Summary of output images from Dance With Me.

31 of 33

User Study

31

Tested Dance With Me with four people.

​

Key feedback points:

​

  • Provide more context at beginning of Dance With Me.

​

  • Provide more instructions before each visualization.

​

  • Would be nice to show all output images at the end of Dance With Me.

​

  • Would be nice to add a countdown near the end of each visualization.

32 of 33

32

We generate a lot of data about ourselves. Can it be a good thing?

33 of 33

Thank you

33