Curriculum �for �Second Year B.Tech � Information Technology ��BIT24MD02 : Augumented Reality � �A.Y 2025-26�Sem – II

By-

Alpana A Borse

Asst. Prof.

IT Department

PCCOE, Pune

​

​

Introduction to Augmented Reality (A.R.)

1

Unit III: Computer Vision for Augmented Reality & A.R. Software (8Hrs)

Introduction to Augmented Reality (A.R.)

2

Syllabus

PCCOE, Pune Introduction to Augmented Reality (A.R.)

3

• Computer Vision for Augmented Reality - Marker Tracking, Multiple-Camera Infrared Tracking
• Natural Feature Tracking by Detection, Simultaneous Localization and Mapping, Outdoor Tracking
• Augmented Reality Software - Introduction, Major Software Components for Augmented Reality Systems, Software used to Create Content for the Augmented Reality Application

Computer Vision for Augmented Reality & A.R. Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

4

• Fundamentals → vision techniques → AR integration → applications
• Computer Vision acts as the “eyes” of AR systems. It helps the system:
• Understand the real world
• Detect objects, surfaces, and motion
• Overlay virtual objects correctly

👉 Example: Snapchat filters

Definition of Computer Vision

PCCOE, Pune Introduction to Augmented Reality (A.R.)

5

Computer Vision is a field of Artificial Intelligence that enables computers to capture, process, and interpret visual information from the real world (such as images and videos) in order to make meaningful decisions (analyze data from real world).

It allows machines to “see and understand” like humans by:

• Identifying objects
• Recognizing patterns
• Understanding scenes

Definition of Computer Vision

PCCOE, Pune Introduction to Augmented Reality (A.R.)

6

🔹 Example:

• Face detection in mobile phones
• Object recognition in AR applications
• Self-driving cars detecting roads and obstacles

​

Computer Vision for Augmented Reality & A.R. Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

7

• Augmented Reality (AR) relies on computer vision techniques
• To integrate digital objects into the real world
• Two key tracking methods used in AR are
1. Marker Tracking and
2. Multiple-Camera Infrared Tracking

Computer Vision for Augmented Reality & A.R. Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

8

• ​

Earth Explore using AR

​

• https://drive.google.com/file/d/17XODZU0YHU5L_nNlGe7qTyBB0lGKdv6a/view?usp=sharing

To integrate digital objects into the real world

PCCOE, Pune Introduction to Augmented Reality (A.R.)

9

1. Marker Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

10

• Marker-based AR uses predefined patterns (markers) that a camera can detect to determine position and orientation in 3D space.

Working:

• The camera captures a black-and-white marker (usually a QR code-like pattern).
• The AR system analyzes the marker's shape and position using computer vision.
• The system superimposes digital content on the marker in real time.

🔹 Marker Tracking - Applications

PCCOE, Pune Introduction to Augmented Reality (A.R.)

11

• AR-enabled books and educational materials
• AR business cards and interactive ads
• AR gaming (e.g., tabletop AR games)

2. Multiple-Camera Infrared Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

12

• This method uses multiple infrared cameras to track objects in 3D space with high precision.
• It is commonly used in motion capture systems and advanced AR applications.

Working:

• Infrared cameras emit IR light that reflects off markers or body-worn sensors.
• The cameras capture multiple angles, and computer vision algorithms triangulate the position.
• The system processes the data to create real-time AR overlays.

2. Multiple-Camera Infrared Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

13

Multiple-Camera Infrared Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

14

Applications:

• Motion capture for AR-based animation and gaming
• AR-based medical simulations and training
• High-precision industrial AR applications

�Motion capture for AR-based animation and gaming�

PCCOE, Pune Introduction to Augmented Reality (A.R.)

15

PCCOE, Pune Introduction to Augmented Reality (A.R.)

16

1. Introduction to Tracking Technology

PCCOE, Pune Introduction to Augmented Reality (A.R.)

17

Categorization of augmented reality tracking techniques.

Difference Between Marker-Based and Markerless Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

18

• Marker-based tracking relies on predefined physical markers for positioning and is highly accurate.
• Markerless tracking uses AI and computer vision to recognize objects or environments, providing more flexibility but requiring higher processing power.

Marker Based

PCCOE, Pune Introduction to Augmented Reality (A.R.)

19

Categorization of augmented reality tracking techniques.

Marker Based

PCCOE, Pune Introduction to Augmented Reality (A.R.)

20

Categorization of augmented reality tracking techniques.

Marker less

PCCOE, Pune Introduction to Augmented Reality (A.R.)

21

Categorization of augmented reality tracking techniques.

Marker less

PCCOE, Pune Introduction to Augmented Reality (A.R.)

22

Categorization of augmented reality tracking techniques.

Marker less

23

24

Marker-based AR; Marker-less AR; Projection AR; Superimposition-based AR.

Difference Between Marker-Based and Markerless Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

25

Computer Vision for Augmented Reality: Natural Feature Tracking, SLAM & Outdoor Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

26

• Augmented Reality (AR) heavily relies on computer vision techniques to track objects and align virtual elements with the real world.
• Three important tracking methods used in AR are:

​

• Natural Feature Tracking (NFT)
• Simultaneous Localization and Mapping (SLAM)
• Outdoor Tracking

1. Natural Feature Tracking (NFT)

PCCOE, Pune Introduction to Augmented Reality (A.R.)

27

• NFT is a markerless tracking technique
• It recognizes real-world objects or textures instead of predefined markers
• Imagine your phone or AR glasses recognizing and tracking real-world objects, such as buildings, posters, or books, and overlaying digital information on them
• This technology allows augmented reality (AR) applications
• To accurately position and anchor virtual content based on natural features in the environment, enhancing the realism and interactivity of AR experiences

1. Natural Feature Tracking (NFT) - Working

PCCOE, Pune Introduction to Augmented Reality (A.R.)

28

• The camera captures an image
• Detects natural features like corners, edges, or textures
• A computer vision algorithm (like SIFT, SURF, or ORB) extracts key feature points
• The AR system matches these features against a database and overlays virtual objects

1. Natural Feature Tracking (NFT) - Working

PCCOE, Pune Introduction to Augmented Reality (A.R.)

29

1. Natural Feature Tracking (NFT) - Working

PCCOE, Pune Introduction to Augmented Reality (A.R.)

30

Applications

PCCOE, Pune Introduction to Augmented Reality (A.R.)

31

• AR product packaging (e.g., interactive labels)
• AR museum guides (tracking paintings, statues, etc.)
• AR user manuals (overlaying instructions on real-world devices)

2. Simultaneous Localization and Mapping (SLAM)

PCCOE, Pune Introduction to Augmented Reality (A.R.)

32

• SLAM is a real-time tracking technique that enables AR systems to understand and map an unknown environment while tracking the user’s position
• Working
• The AR system continuously captures camera frames and detects key feature points
• It creates a 3D map of the environment while estimating the camera’s position
• The system updates the map dynamically, allowing virtual objects to stay fixed in place

Applications:

PCCOE, Pune Introduction to Augmented Reality (A.R.)

33

• AR navigation (e.g., indoor AR maps)
• AR gaming (e.g., Pokémon GO)
• AR remote assistance (placing virtual annotations in a real-world scene)

Block diagram of the IMU and SLAM relationship to Pose and the 3D map

PCCOE, Pune Introduction to Augmented Reality (A.R.)

34

3. Outdoor Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

35

• Outdoor tracking enables AR experiences in large open spaces, such as streets, parks, or city environments
• It relies on GPS, computer vision, and sensor fusion to accurately place digital content

Working

• GPS & IMU sensors determine rough position and orientation
• Computer vision algorithms analyze surroundings for fine-tuned tracking
• The AR system aligns virtual objects with the real-world environment

Summary of Tracking Methods

PCCOE, Pune Introduction to Augmented Reality (A.R.)

36

• NFT: “Track features”
• SLAM: “Track + build map”
• Outdoor Tracking: “Track in real world using GPS + SLAM”

👉 NFT → SLAM → Outdoor

• NFT = basic
• SLAM = advanced
• Outdoor = real-world application
• ​
• ​

Summary of Tracking Methods

PCCOE, Pune Introduction to Augmented Reality (A.R.)

37

NFT, SLAM, Outdoor Tracking

PCCOE, Pune Introduction to Augmented Reality (A.R.)

40

• Link to refer NFT SLAM��https://docs.google.com/document/d/1B9JyDA3OEspKzXhJ2z4meLMOG0vFuu-9/edit?usp=sharing&ouid=103921989690192741052&rtpof=true&sd=true
• ​

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

41

• Link: https://en.wikipedia.org/wiki/List_of_software_related_to_augmented_reality
• Augmented reality is an enhanced, interactive version of a real-world environment achieved through digital visual elements, sounds, and other sensory stimuli via holographic technology.
• Link: https://thectoclub.com/tools/best-augmented-reality-software/

​

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

42

Best Augmented Reality Software Shortlist

I’ve assessed tons of AR tools, selecting these for my shortlist:

1. Unity — Best for powerful game development and interactive real-time 3D content

2. Vossle — Best for quick and easy creation of web-based AR experiences

3. EasyAR — Best for comprehensive AR SDK with diverse tracking modes

4. ARKit — Best for detailed and realistic AR experiences on Apple devices

5. Scope AR — Best for integrating AR into workforce training and remote assistance

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

43

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

44

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

45

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

46

AR Software

PCCOE, Pune Introduction to Augmented Reality (A.R.)

47

Major software components of Augmented Reality (AR) Systems:

PCCOE, Pune Introduction to Augmented Reality (A.R.)

48

1. Tracking & Registration Module

Ensures accurate alignment of virtual objects with the real-world environment.

Uses Marker-Based Tracking, Markerless Tracking, SLAM (Simultaneous Localization and Mapping), or GPS-based tracking.

Technologies: OpenCV, ARKit (Apple), ARCore (Google), Vuforia.

2. Scene Rendering Engine

Renders 3D virtual objects in real-world scenes.

Uses 3D graphics engines like Unity3D, Unreal Engine, or WebAR libraries.

Handles lighting, shading, and occlusion for realism.

Major software components of Augmented Reality (AR) Systems:

PCCOE, Pune Introduction to Augmented Reality (A.R.)

49

3. Sensor & Input Processing

Integrates data from cameras, GPS, accelerometers, gyroscopes, and depth sensors.

Ensures accurate motion detection and environmental interaction.

Software APIs: ARKit, ARCore, Vuforia.

4. Interaction & User Interface (UI) Module

Manages user interactions via touch, voice commands, gestures, or controllers.

Uses gesture recognition, hand tracking, and speech processing.

Libraries: OpenXR, Leap Motion, MRTK (for HoloLens).

Major software components of Augmented Reality (AR) Systems:

PCCOE, Pune Introduction to Augmented Reality (A.R.)

50

5. Augmented Content Management System

Stores, retrieves, and delivers 3D models, animations, audio, and text in real-time.

Uses cloud or local databases for content delivery.

Examples: Google Cloud, Firebase, AWS for cloud-based AR content.

6. Networking & Cloud Integration

Enables multi-user collaboration, remote rendering, and cloud-based processing.

Uses edge computing and 5G networks for real-time performance.

Technologies: WebRTC, Photon (for multiplayer AR), Google AR Cloud.

Major software components of Augmented Reality (AR) Systems:

PCCOE, Pune Introduction to Augmented Reality (A.R.)

51

7. AR Development Platforms & SDKs

Provides tools and APIs for building AR applications.

Examples:

ARKit (Apple) – iOS AR development.

ARCore (Google) – Android AR development.

Vuforia – Cross-platform AR SDK.

Wikitude – WebAR & mobile AR development.

• 9-STEP AUGMENTED REALITY APP DEVELOPMENT GUIDE
• Link: https://codeit.us/blog/how-to-create-an-augmented-reality-app

​