INTRODUCTION to Mixed Reality

01

Alpana A. Borse

X-reality Technologies and Applications

MR

• Mixed Reality
• As immersive technology is advancing how we can interact with digital immersive spaces and objects is changing so another important term to learn is Mixed Reality or MR
• It is easiest to think of MR as AR but with advanced digital-physical interaction that narrows the gap between the real and the virtual

Digital immersive spaces

• Refers to digitally created environments that deeply engage users by stimulating multiple senses and creating a feeling of presence—as if the user is “inside” the digital world rather than just observing it
• Technology-enabled environments where users interact with virtual or augmented content in real time using devices like VR headsets, AR-enabled mobiles, XR systems, large projection rooms, or interactive displays.

Core Technologies Involveds

• Virtual Reality (VR): Fully virtual environments (e.g., VR labs, virtual campuses)�Augmented Reality (AR): Digital overlays on the real world (e.g., AR labs, markers, image tracking)�Mixed Reality (MR): Real and virtual objects interact together�Extended Reality (XR): Umbrella term covering AR, VR, MR�Spatial Computing: 3D interaction with digital objects in physical space�AI & Sensors: Gesture tracking, eye tracking, voice interaction

3. Key Characteristics

�Immersion: Sense of presence and engagement

Interactivity: User actions influence the environment

• Real-time Response: Immediate system feedback
• Multi-sensory Experience: Visual, audio, haptics (sometimes smell)�Context Awareness: Adapts to user location or actions

Types of Digital Immersive Spaces

�Immersion: 4. VR-based Immersive Spaces�

• Virtual classrooms�
• Virtual labs and simulations�
• AR-based Immersive Spaces�
• Smart classrooms�
• Museum guides�
• Industrial maintenance�
• Projection-based Spaces�
• CAVE systems�
• Dome theatres�
• Immersive classrooms with wall/floor projections�
• Metaverse Spaces�
• Persistent virtual worlds�
• Social and collaborative environments�
• Hybrid Physical–Digital Spaces�
• Smart labs�
• Interactive learning rooms

Adapts to user location or actions

6. Advantages

�Enhanced learning retention�

• Safe simulation of risky scenarios�
• Cost-effective long-term training�
• Accessibility and remote collaboration

​

MR - Term: Anchoring

• Anchoring means fixing or attaching a digital object to a specific position in the real world, so that it stays in the same place relative to real objects even when you move.
• Anchoring is the process of locking a virtual object to a real-world location
• https://www.youtube.com/watch?v=QZi9DcGW9iU

​

Real vs Digital

• Real: Table, room, walls�Digital: Book, anchor data�Anchor: The link between real space and digital object

What Anchoring Does

• Anchoring Keeps digital objects stable
• Maintains real–digital alignment
• Allows persistent Mixed Reality experiences

Anchoring: Example

• A digital book is anchored to your bedside table
• If you walk around the room or remove and wear the headset again
• The book appears in the same spot on the table
• This happens because its position is stored using a spatial anchor

1. World Anchoring (Spatial Anchoring)

• Virtual objects are fixed to a real-world location
• Objects stay in place even when the user moves
• Uses SLAM, spatial mapping, planes, and depth data

Example:

• A virtual chair placed on the floor using ARCore/ARKit
• HoloLens world anchors

Used in: AR, MR�Tools: ARKit, ARCore, HoloLens, Vuforia

1. Spatial

• In AR, VR, XR, and digital immersive spaces, spatial describes how digital objects:
• Exist in 3D space
• Maintain position and orientation
• Respond to user movement
• Interact with the real environment

Spatial Examples:

• Spatial mapping: Understanding room geometry
• Spatial tracking: Tracking user movement
• Spatial audio: Sound that changes based on position
• Spatial anchors: Fixing virtual objects in real-world locations

Spatial Examples:

• Spatial mapping: Understanding room geometry
• Spatial tracking: Tracking user movement
• Spatial audio: Sound that changes based on position
• Spatial anchors: Fixing virtual objects in real-world locations

2. User Anchoring (Body / Head Anchoring)

• Virtual content is anchored relative to the user
• Moves with the user’s head, body, or hand

Example:

• HUD (speed, score) fixed to the user’s view
• Wrist-mounted AR menus

Used in: AR, VR

​

3. Object Anchoring (Target-Based Anchoring)

• Virtual objects are anchored to real-world objects�Requires a known reference�a) Image Anchoring
• Uses images as targets�Example: 3D model appearing on a book cover�b) Object Anchoring
• Uses 3D objects as targets�Example: AR model attached to a machine part�Tools: Vuforia Image Targets, Object Targets

4. Marker-Based Anchoring

• Uses fiducial markers (QR codes, AR markers)�High accuracy, simple implementation�Example:
• AR cube on a printed marker�Used in: AR (education, labs)

5. Markerless Anchoring

​

• No physical markers required�Uses environment understanding

Example:

• Pokémon GO placing objects on flat surfaces

Used in: Modern AR systems

6. Surface / Plane Anchoring

​

​

• Anchors objects to detected horizontal or vertical planes�Example:
• Placing a sofa on the floor�Hanging a painting on a wall�Used in: AR interior design apps
• ​

7. Persistent Anchoring (Cloud Anchors)

​

​

​

• Anchors objects Anchors remain persistent across sessions and users�Stored in the cloud�Example:
• Same AR object visible to multiple users in the same place�Tools:
• ARCore Cloud Anchors�Azure Spatial Anchors

8. Virtual Anchoring (VR Anchoring)

• Anchors Anchoring within a fully virtual environment�Objects are fixed relative to the virtual world�Example:
• VR furniture in a virtual room�Used in: VR only

9. Geo Anchoring (Location-Based Anchoring)

​

• Uses GPS, compass, and location data

Example:

• AR navigation overlays
• Location-based AR games�Used in: Outdoor AR

MR Examples: Book

• Mixed Reality Book Reading Experience
• Example of Mixed Reality Interaction
• Mixed Reality in Everyday Life
• Interacting with a Digital Book in a Mixed Reality Environment
• Anchoring and Manipulating Digital Objects Using Mixed Reality
• A Step-by-Step Example of Mixed Reality Using Hand Tracking

MR Examples: Book

• Mixed Reality Example: Real–Digital Interaction Using Hand Tracking
• Mixed Reality Application: Digital Object Anchoring and Natural Interaction
• Mixed Reality Experience with Real-World Anchored Digital Content
• “Mixed Reality Example: Real–Digital Interaction Using Hand Tracking”

Step 1: Real World Environment

• What is real:
• Your bedroom
• Bedside table
• Your hands and fingers
• Role:
• Provides the physical space where the digital content will exist.

Step 2: Digital Object Creation

• What is digital:
• A virtual book (not physically real)
• Role:
• The book exists only as computer-generated content.

Step 3: Anchoring Digital to Real World

• What happens:
• The digital book is anchored to a real location (the bedside table).
• What is real:
• Bedside table.
• What is digital:
• Virtual book.
• Why Mixed Reality:
• The digital object stays fixed in a real-world position..

Step 4: Wearing the Headset

• When headset is used:
• When you want to see and interact with the digital book
• Why headset is needed:
• To display the digital book in your real environment.
• To track:
• Your hands
• Your finger movements
• Furniture around you

Step 5: Interaction with the Digital Object

• What you interact with:
• Digital book
• How you interact:
• Pick up the book using your real hand
• Turn pages using real fingers
• What is real
• Your hands and gestures.
• What is digital:
• Book pages reacting to gestures

Step 5: Interaction with the Digital Object

• What you interact with:
• Digital book
• How you interact:
• Pick up the book using your real hand
• Turn pages using real fingers
• What is real
• Your hands and gestures.
• What is digital:
• Book pages reacting to gestures

Step 6: No Touch Screen Involved

• Interaction method:
• Natural hand tracking and gestures
• Technology used:
• Sensors and cameras in the headset
• Why this matters:
• Creates a more natural and immersive experience.

Step 7: Placing the Book Back

• What happens:
• You place the digital book back on the bedside table
• What is real:
• Physical table
• What is digital:
• Virtual book remains anchored
• Key feature:
• The book stays in the same real-world location even after you stop using it.

Step 8: Ending the Experience

• When you remove the headset:
• The book is no longer visible
• But:
• The digital book still “exists” at that location.
• Next time:
• Wear the headset again → the book appears exactly where you left it.

Summary

Existence of digital data

• Even after removing the headset, the digital book does not physically exist but remains stored as digital data with a fixed spatial anchor.
• When the headset is worn again, the object reappears at the same location.
• To remove the digital object, the user must explicitly delete it using gestures, voice commands, virtual menus, or by resetting the environment or application.

How to remove the digital book?

• Digital objects in Mixed Reality exist as data and spatial anchors
• not as physical objects
• Removing the headset only hides them;
• deleting the anchor or deleting the object data permanently removes them

How to remove the digital book?

• Method 1: Gesture-Based Removal (Most Natural)
• Wear the headset
• Look at the digital book
• Perform a delete gesture (e.g., pinch and hold, grab and throw, long press in air)
• The system removes the digital object

​

✔ Common in advanced MR systems

How to remove the digital book?

• Method 2: Voice Command
• Say a command like:
• “Remove book”
• “Delete this object”
• Voice recognition triggers deletion

​

✔ Hands-free and intuitive

How to remove the digital book?

• Method 3: Virtual Menu / UI
• Wear the headset
• Open a floating digital menu
• Select the book
• Choose Delete / Remove

​

✔ Used when precise control is needed

How to remove the digital book?

• Method 4: Reset the Environment
• System option like:
• “Clear room”
• “Reset anchors”
• All anchored digital objects are removed

​

✔ ✔ Useful when changing rooms or users

How to remove the digital book?

• Method 4: Method 5: App-Level Removal
• The digital book belongs to an MR app:
• Closing or uninstalling the app removes all its digital objects

​

✔ ✔✔ Simplest method

Frameworks That Actually Perform Anchoring

Frameworks That Actually Perform Anchoring

References

https://www.zappar.com/mixed-reality/

THANK YOU!

Write a closing statement or call-to-action here.