1 of 60

Definition of VR, history

IITM-Finland - Mastering VR:�Fundamentals to Practice!

2 of 60

Where Are We?

Chapter 1: Intro
Chapter 2: Bird’s Eye View
Chapter 3: The Geometry of Virtual Worlds
Chapter 4: Light and Optics
Chapter 5: The Physiology of Human Vision
Chapter 6: Visual Perception
Chapter 7. Visual Rendering
Chapter 8. Modeling of Motion
Chapter 9. Tracking Systems�

https://lavalle.pl/vr/

3 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

4 of 60

Are you enrolled in Lovelace?

ⓘ

Click Present with Slido or install our Chrome extension to activate this poll while presenting.

5 of 60

Are you on Discord?

ⓘ

Click Present with Slido or install our Chrome extension to activate this poll while presenting.

6 of 60

Have you ordered your Cardboard?

ⓘ

Click Present with Slido or install our Chrome extension to activate this poll while presenting.

7 of 60

Announcements

Are you enrolled in Lovelace? https://lovelace.oulu.fi/iitm-finland/iitm-finland/

Only 299 students on Lovelace, out of 600…
You won’t get a grade in the course, unless you complete course work on Lovelace

Are you on Discord? (only ~330 students are there now) https://discord.gg/4q5tMx8Grz
Do you already have a Cardboard?
Did you successfully calibrate your Cardboard (HW1)?

How did it go? Are you happy? How many iterations did it take?

While you are waiting for the Cardboards, start on HW2:

HW2: VR Critic
The first part of HW2 is the reading only, with questions following the reading

Deadline for HW1 will be extended

8 of 60

Definition of VR

Inducing targeted behavior in an organism

by using artificial sensory stimulation, while

the organism has little or no awareness of the interference

9 of 60

Examples of VR?

https://www.youtube.com/watch?v=aOYHg8qdxTE

10 of 60

Examples of VR?

https://www.youtube.com/watch?v=1DJOTEDBA2c

"Intracellular dynamics of hippocampal place cells during virtual navigation." By Christopher D. Harvey, Forrest Collman, Daniel A. Dombeck & David W. Tank. Nature, Vol. 461 No. 7266, October 14, 2009

So here is a different organism on a different treadmill. In this case, it is a mouse running on a spherical treadmill, also exploring a virtual environment.

(click) The mouse is suspended by its head that is attached to a brain imaging device. The scientists can literally look into the brain while the mouse explores the environment. The body of the mouse is being tracked by a cave-like projected system.

Is this VR? We have the organism, behavior, engineered stimulation to many senses, and awareness. The mouse seem quite engaged into exploring a completely virtual/simulated world, possibly for some kind of reward, like food in the end of a tunnel. So is this VR according to our definition or not?

(click) This work was published in Nature in 2009. A top journal in the world for anything bio related.

Mouse in VR Maze. From "Intracellular dynamics of hippocampal place cells during virtual navigation." By Christopher D. Harvey, Forrest Collman, Daniel A. Dombeck & David W. Tank. Nature, Vol. 461 No. 7266, October 14, 2009

11 of 60

Examples of VR?

https://www.youtube.com/watch?v=1ezL8nGo--I

Work published in 2010 in Nature Methods: http://dx.doi.org/10.1038/nmeth.1468 ��It was also highlighted in June 2011 WIRED magazine article "Micro Machinist" Video uploaded by Gus Lott, designer of the high speed motion tracking system in this project.

Even a more bizarre setting with a fruit fly on a treadmill. The treadmill in this case is only about 2mm in diameter, it is a nearly frictionless ball which is suspended by air.

(click) Let’s see what is going on in this experiment. The fly is tethered by its head with the brain exposed to a microscope.

And what do the scientists observe as the fly moves: The upper left image shows a single neuron in the brain, injected with a fluorescent dye. So the scientist literally see the neurons activity..

This neuron fires when something moves laterally in the visual field of the fly. The blue bars are the virtual world that keeps moving as the fly explores it, and every time the bars move, the corresponding neuron fires.

(click) The work was published in 2010 in Nature Methods: http://dx.doi.org/10.1038/nmeth.1468 It was also highlighted in June 2011 WIRED magazine article "Micro Machinist" Video uploaded by Gus Lott, designer of the high speed motion tracking system in this project.

12 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

13 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

14 of 60

Definition of VR

Inducing targeted behavior in an organism by using artificial sensory stimulation, while the organism has little or no awareness of the interference

How to define awareness?

15 of 60

How would you define awareness? How do you know if you are completely immersed in a virtual environment?

ⓘ Start presenting to display the poll results on this slide.

16 of 60

Mainstream Use of VR vs Science

Neuroscience, neurobiology, perceptual psychology:

To study neural processes of organisms
To reverse engineer the brain of an organism�

Mainstream VR and our class:

Entertain the organism�

Results from neuroscience will help us define awareness.

Awareness is defined using special neural structures:

Place cells
Grid cells

Well it turned out that something fascinating happened lately in science,

(click) virtual reality became extremely popular in neuroscience, neurobiology and perceptual psychology. The scientists in these fields use VR extensively for their research.

(click) The reason is that VR allows to study the neural processes of organisms. Just like you saw in the videos with the mouse and the fruit fly. The scientists present controlled stimuli to the organisms and by inserting electrodes in their brain, or using microscopes, or other imaging techniques they can literally see the neural activity during the experiment, while controlling their behavior. ��(click) Essentially, they use VR exactly according to our definition, but the goal is quite different from what we do in our class. They want to reverse engineer the brain, while we will want to simply entertain the organism.

(click) So the mainstream use of VR

(click) is mostly for entertainment

(click) For the purposes of our class, the recent groundbreaking results from neuroscience will help us define awareness. How do we know that the organism is really fooled? :) Is this really an engineering concept or something metaphysical?

(click) It turns out that the definition of awareness is now something quite concrete. It can be defined using neural structures many organisms including humans have

(click) place cells

(click) grid cells

17 of 60

2014 Nobel Prize in Physiology or Medicine

Defining awareness = new place cells?

https://www.youtube.com/watch?v=lfNVv0A8QvI

(click) The following discovery was awarded the Nobel Prize in Physiology or Medicine in 2014.

(click) There is something called place cells in our brains and the brains of rodents, like mice and rats, where a particular neuron in the brain fires ONLY when you are in a particular place.

(click) So watch this video of a rat in a labyrinth attached to a single unit recording device(the dangling wire) that measures the firing of a single neuron (the boxes on the left), and see that the neurons only fire when the rat is roughly in the same place! These experiments were repeated on many mice and gerbils and on various environments including virtual.

So these place cells is what the definition of being fooled could be. If your brain decides that the space is new and real enough, then it designates a special neuron for this place!!!! If there are no new place cells, then the experience wasn't immersive or realistic enough!

There are also grid cells, special neurons designed to recognize the spatial grid we are in, which were also awarded the Nobel prize for their discovery.

18 of 60

Awareness = place cells + grid cells!

https://www.youtube.com/watch?v=1kv6r_JkopQ

19 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

20 of 60

Definition of VR: Warning for VR Developers

Who is the laboratory rat and who is the scientist?
A VR developer is BOTH the scientist and the lab rat!
This is very dangerous!
The developer is trying to fool himself with his own creation.
Developer’s bias:

Bias by the desire for the system to succeed.
Bias by the knowledge of what the system is supposed to do.

Here are a couple of warnings for VR developers.

(click) When neuroscientists work with mice or rats, or other lower form organisms, there is a clear distinction between who is the scientist and who is the lab rat.

(click) This is quite different from the setup we have in engineering and computer science. When a VR developer creates a virtual reality experience for humans - the developer is usually both the scientist and the lab rat!

(click) THIS is very dangerous.

(click) Think about it: the creator is trying to fool himself with the system while developing it.

(click) This can result in developer’s bias.

(click)For example, if you are developing your final project, you might be simply biased by the desire for the project to work, especially if it is the night before the deadline.

(click) Another bias: the developer usually knows what the experiment is supposed to accomplish. On the other hand, fresh/unaware subjects may have a completely different impression of the experience. Try scaring yourself, or tickling yourself, or surprise yourself.

21 of 60

Definition of VR: Warning for VR Developers

The creator might adapt to/unconsciously avoid the flaws of the system!
The flaws might effectively become invisible.
Developers of VR systems must test their experiences to avoid bias:

On subjects who never tried the experience.
On subjects who have never tried VR.
On subjects who DO get motion sick.

HW2. VR Critic: Best Practices Guide.

(Click) Just to rephrase: the creator might adapt to flaws and even unconsciously avoid them,

(click) so that the flaws effectively become invisible to the creator!

(click) The conclusion is that: it is very important to test VR experiences on

(click) other people who didn’t try it before,

(click) maybe even on people who have never been exposed to VR, those who never played first person shooter games,

(click) those who DO get motion sick.

Will your experience make them uncomfortable? Will your friends/ or grandma be sick for the rest of the day after trying your final project?

(click) In this course we will start addressing these issues of comfort right from the start. Your HW2 will be to study the Oculus VR best practices guide, which is an official Meta guide to developers about how to create best possible VR experience, learn how to avoid flaws, and train yourself to recognize flaws in VR.

22 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

23 of 60

Definitely VR… Socializing in Virtual Spaces

Now I will go through many modern Applications of VR.

There are a lot of virtual spaces out there where you can hang out, for example:

Virtual conferences,

Virtual weddings, like this groom in Japan marrying his virtual anime girlfriend.

(click)Even virtual funerals.

And of course Metaverse is supposed to be where we will be doing about almost anything.

—---

Today is a very entertaining lecture. So just sit back and relax and let’s enjoy some cool examples of VR. And I will go through many examples of VR, but there are many that I skip, so if I skipped your favorite VR application, please take a note of it and I will have a slide later where you can type your favorite VR Application that I missed.

(click) Many people use virtual spaces for socializing, could be in VR or just on screens.

2nd life and OpenSim are examples of such systems.

Facebook wants to build MetaVerse where we will all be socializing, too.

There are various communities of people out there already who have various events happening in VR:

Grooms in Japan can now officially get married to a anime cartoon, virtual reality brides, a ceremony in Tokyo showed.

The concept comes from anime game 'Niitzuma LovelyxCation', where the players are able to develop a relationship and have a girlfriend in the VR world. In addition to playing Niitzuma at home, thanks to its creators, fans of the game can now have this virtual marriage ceremony, to bond themselves to their cartoon girlfriends.

Funeral of Stanley W (2008)
And a developers conference. After attending one of these, Steve LaValle for examples wasn't sure after some time if he met those people in real life or not. He remembers them as real people. Some guy in Finland came to him and said he liked his talk, and he answered - really, which avatar were you?

24 of 60

Definitely VR… Architecture and Real Estate

https://youtu.be/kPHiui51vrg?t=79

25 of 60

Definitely VR… Movies

https://www.youtube.com/watch?v=NkLZXKCGf1M

Another interesting application is VR cinema.

(click and stop after a few moments)

So this one is a weird one. The first time I heard about it, I thought it was lame. In the early Oculus day, there was a 22 year old korean software developer who visited Oculus and showed us his implementation of a VR cinema. And he was very excited about it. So imagine that you are sitting in a VR movie theater, the surround sound is there, the acoustics are adjusted for the walls, the flickering projectors is right behind you and you can turn around and see it flickering. You can pick any seat. And you can stream any movie you already have, by leveraging on all of the existing streaming technology. And you can invite friends to visit.

(click)

SO is this cool or not?

This software is freely available in Oculus Store.

26 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

27 of 60

Definitely VR… Panoramas, Earth VR

https://www.atmosphaeres.com/video/581/Overhanging+Cliffs+With+View+To+Cap+Formentor

28 of 60

Definitely VR… VR + Robots

https://www.facebook.com/makeyourfutures/videos/709112912563319/

29 of 60

Definitely VR… First-Person Shooter Games

https://www.youtube.com/watch?v=K3Ze-zfqGdU

30 of 60

Definitely VR… Dumpy the Elephant

https://www.youtube.com/watch?v=yaFOMCOgqUA&t=52s

31 of 60

What is an obvious visual flaw of this VR experience?

ⓘ Start presenting to display the poll results on this slide.

32 of 60

Definitely VR… Thrill Seekers

https://www.youtube.com/watch?v=jLtcPTm5dTg

33 of 60

Definitely VR… Experiences

https://youtu.be/N8zkbl6mKXE

34 of 60

Definitely VR… Body Swapping

https://youtu.be/dOSJETowuik?t=146

35 of 60

Definitely VR… Flying Like in Your Dreams, Birdly VR

https://youtu.be/HOdjdS-x22I

36 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

37 of 60

VR or Not VR?

VR experience with an Oculus Quest 2
Playing Second Life (first person video game)
Playing a third person video game
Augmented reality
Watching a movie
Video conferencing
Listening to music
Talking on the phone
Reading a book
Looking at a painting
Being under the influence of a hallucinogenic drug
Wearing thermal clothes

So now let's have a bit of fun, I intentionally picked examples that are right on the border of our definition of VR. We could argue about whether they are VR or not, according to our definition.

(click) Let’s start with simple things : an actual VR experience using say an Oculus Quest 2

(click) Now what about playing Second Life (a first person video game on a desktop) - is that VR

(click) a bit harder - playing a third person video game

(click) now augmented reality, you wear say some kind of glasses that have a chat window in them. Or what if it augments more and more of the world?

(click) Watching a movie?

(click) Video conferencing, this lecture that we are in right now, is it VR? Do you feel present, will you create similar neural structures as if you were sitting in a real classroom?

(click) more bizarre settings: listening to music

(click) talking on the phone

(click) reading a book

(click) or staring at a painting?

Have you ever been immersed in to a conversation, or a book, or an image so much that you forget where you were? Are these VR?

(click) What about being under the influence of a hallucinogenic drug? In this case it is more clear - the creator has little control of what the subject will experience or which senses are going to be affected? Many drugs affect the brain directly bypassing the senses. Is that VR?

(click) And now completely controversial: wearing thermal clothes? Is it a VR or not? Can you full yourself into believing it is summer with wearing really good winter clothes?

Probably not.

So help me draw the line here, where does VR end and not VR start?

38 of 60

When did VR start?

Paintings?

Cueva de El Castillo, or the Cave of the Castle, Oldest known cave painting is in Spain 40,800 years old; possibly created by Neanderthals! Paint was blown onto the walls.

A 30,000-year-old painting from the Bhimbetka rock shelters in India (photo by Archaeological Survey of India).

39 of 60

When did VR start?

An English painting from around 1470 that depicts John Ball encouraging Wat Tyler rebels (unknown artist)

40 of 60

When did VR start?

Definitely �VR?

A painting by Hans Vredeman de Vries in 1596.

41 of 60

When did VR start?

An impressionist painting by Claude Monet in 1874.

42 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

43 of 60

When did VR start? Motion pictures?

https://www.youtube.com/watch?v=IEqccPhsqgA

1878, Edward Muybridge

44 of 60

When did VR start? Motion pictures?

https://www.youtube.com/watch?v=b9MoAQJFn_8

L'arrivée d'un train à La Ciotat (1895) - frères Lumière

Criticism:

Blurry
Shaky
Low res
B/W

45 of 60

When did VR start? Motion pictures?

https://youtube.com/watch?v=vKW-Gd_S_xc

Gravity, 2013

46 of 60

Realism vs Simplicity in Cartoons

Fantasmagorie 1908

Star Wars: The Clone Wars, 2003

The South Park, 1997

47 of 60

Realism vs Lower Cost and Portability

Cheap
Portable

Another motivation to lower quality and lower realism is portability and cost.

(click) you can see here families in 1950s gather around a tube TV to watch these tiny blurry B/W screens because they were FREE broadcasts INSIDE their homes

(click) even though they could go to theaters and watch high resolution, color, panoramic and 3D movies (there was some vibration in the seats) at the time!

These tube TV seem intolerable to us now, because nowadays we

(click) are even able to carry screens with us (like this smartphone) and do almost everything on it.

But we also sacrifice quality when we carry our phones with us. What do we sacrifice? The quality of image, the quality of the internet connection, but we gain portability and lower cost!

Humans seem to like things that are

* cheap and

* portable,

even if we lose in quality!

48 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

49 of 60

How many FPS is enough?

2 FPS is enough to perceive motion, but the motion is not continuous.

https://www.youtube.com/watch?v=-Qk7ZSXujRo

50 of 60

What FPS rate is enough to perceive continuous motion?

ⓘ Start presenting to display the poll results on this slide.

51 of 60

Questions about FPS

How many FPSs is enough to perceive motion? 2FPS
How many FPSs is enough to perceive continuous motion? 12FPS
How many FPSs is enough to watch movies comfortably?

In a movie theater - 24FPS
On a TV?
On a phone?
In VR?

If you increase FPS - gain quality but increase the cost.
How many FPSs is enough for VR?

It all depends on: the hardware specifications, distance to the screen, and the types of interaction you are having with the content.

Things that might cause discomfort: judder, flicker, flicker fusion, tracking, perception of stationarity, etc.

People studied very carefully how many frames per second is enough to

(click) perceive motion (not continuous yet) - it is 2FPS

(click) perceive continuous motion - is at least 12 FPS

(click) to perceive motion comfortably in movies.

(click) Well, that depends on what device you are watching you movie on, in a movie theater it is currently 24, but filmmakers have to avoid certain things that would require more than that. They avoid fast motions of the camera, as well as fast motions of objects crossing the screen.

(click) on a TV - it is a different requirement, from 24-30-60

(click) on a phone 60-90-120

(click) in VR 90 - 120 - ?

Because it all really depends what is the distance to the screen and how fast the objects are allowed to move on the screen.

(click)What happens if you start increasing the number of frames to 50, 90, or 100. It increases the quality of the experience. So why isn’t it done in the movies? Well, it would make making movies much much more expensive, but would it improve the experience? Not necessarily.

(click) In something like the Oculus Rift the requirements for the number of frames per second are even higher, though, because the users have more control over how they interact with the content. They can move their heads and eyes really fast, which results in discontinuity in motion, and even flicker. Flicker can lead to eye strain and fatigue.

(click) In conclusion, the specification of FPS depends on many factors: hardware, distance to the screen and the types of interaction that is allowed with the content.

(click) Therefore, we will study very carefully what is the optimal number of FPS in VR to improve the comfort while minimizing the cost.

52 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

53 of 60

Evolution of Computer Games

54 of 60

Realism vs Simplicity in Computer Games

55 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.

56 of 60

History of VR

https://youtu.be/zf61nlyBC3M

57 of 60

History of VR

https://www.youtube.com/watch?v=eVUgfUvP4uk

A big leap wrp to that was Ivan Sutherland's sword of Damocles, which is considered the first functional VR headset. It was built in 1968 and it used the power of modern computers at the time. Ivan Sutherland is considered the father of modern VR.

It was extremely heavy. It had to be not only head mounted but ceiling mounted :) It was suspended by a metal construction attached to the ceiling. It was more AR than VR. A computer would genertel simple objects like prisms, cubes, and spheres and would project them onto the outside world using something very similar to common glasses. So you had this illusion of new objects in the real world. Just like AR glasses.

Also this was the first time when sensors and tracking system needed to be a part of VR. You can’t see a large computer here, but there was a computer a size of a wall located near this device to do all of the computations.

58 of 60

Consumer VR

Nintendo Virtual Boy in 1995 marked the death of VR
…until Palmer Lucky in 2012

1960-1990 - big hype, first wave of VR

Cost, resolution, latency, content, heavy, bulky, expensive….

(click) Starting with the sixties and up to the 90s there was a lot of hype around VR.

(click)First commercial products started appearing in 1980s,

(click) especially with Jaron Lanier's company VPL,

(click) popularized goggles and gloves already at the time.

(click) However, there were always problems with cost, resolution, latency and content, the devices were also heavy, bulky and expensive.

(click) The first commercial VR game ever released was Nintendo Virtual Boy in 1995, which

(click) marked the death of VR, because after all of the big hype this game generated, it didn't even track your head, and the content was red and white! not even in colors.

So the common though about VR from the 90s until 2012 was that if Nintendo couldn't do it, then nobody could. And after that for almost 2 decades nobody wanted to try to reinvent VR again,

(click) until the 19 year old palmer lucky revived the hype with the use of all of the new technology such as smartphones and mems that made it possible.

59 of 60

Consumer VR

https://www.youtube.com/watch?v=STMcWUtQr1Y

60 of 60

Audience Q&A Session

ⓘ

Click Present with Slido or install our Chrome extension to show live Q&A while presenting.