Curriculum �for �Second Year B.Tech � Information Technology ��BIT24MD02 : Augumented Reality � �A.Y 2025-26�Sem – II
By-
Alpana A Borse
Asst. Prof.
IT Department
PCCOE, Pune
Introduction to Augmented Reality (A.R.)
1
Unit IV: AR Techniques- Marker based & Markerless tracking (8Hrs)
Introduction to Augmented Reality (A.R.)
2
Syllabus
PCCOE, Pune Introduction to Augmented Reality (A.R.)
3
Marker Tracking in Augmented Reality
Marker-based AR uses predefined visual patterns that a camera detects to determine position and orientation in 3D space — forming the foundation of many AR experiences.
PCCOE, PUNE
INTRODUCTION TO AUGMENTED REALITY
How Marker Tracking Works
The Detection Pipeline
01
Detect the Marker
Camera captures a predefined marker in the scene.
02
Analyze Shape & Position
Computer vision algorithms parse the marker's geometry.
03
Calculate Pose
The system estimates camera position and orientation.
04
Render Virtual Object
Digital content is superimposed on the marker in real time.
Key Idea: Camera detects marker → calculates pose → renders virtual object
Marker Tracking — Applications
Education
AR-enabled books and educational materials bring static content to life with interactive 3D overlays.
Marketing
AR business cards and interactive ads create memorable, engaging experiences for audiences.
Gaming
Tabletop AR games use physical marker cards to trigger and control virtual game elements.
Taxonomy of AR Tracking Techniques
Two Major Branches
AR tracking techniques are broadly divided into two families:
Marker-Based
Uses fiducial markers or tags (direct/indirect, RFID, barcode).
Markerless
Uses sensor data or computer vision — model-based, Visual SLAM, SFM, and more.
Marker Based
PCCOE, Pune Introduction to Augmented Reality (A.R.)
8
Categorization of augmented reality tracking techniques.
Marker Based
PCCOE, Pune Introduction to Augmented Reality (A.R.)
9
Categorization of augmented reality tracking techniques.
Marker less
PCCOE, Pune Introduction to Augmented Reality (A.R.)
10
Categorization of augmented reality tracking techniques.
Marker less
PCCOE, Pune Introduction to Augmented Reality (A.R.)
11
Categorization of augmented reality tracking techniques.
Marker less
12
Difference Between Marker-Based and Markerless Tracking
PCCOE, Pune Introduction to Augmented Reality (A.R.)
14
Feature | Marker-Based Tracking | Markerless Tracking |
Definition | Uses predefined physical markers (QR codes, AR markers, fiducial markers) for tracking. | Uses computer vision and AI to recognize objects, features, or environments without predefined markers. |
Tracking Method | Camera detects the position and orientation of the marker to place virtual objects. | Analyzes the environment using feature detection, SLAM (Simultaneous Localization and Mapping), and AI-based recognition. |
Examples | - QR codes in AR apps | |
Marker-Based vs. Markerless Tracking
Choosing the Right Approach
Marker-based tracking excels in controlled environments where high accuracy is paramount. Markerless tracking opens the door to broader, real-world applications but demands significantly more computational resources and sophisticated algorithms.
Summary of Tracking Methods
PCCOE, Pune Introduction to Augmented Reality (A.R.)
16
| | |
Tracking Method | Key Features | Best Use Cases |
Natural Feature Tracking (NFT) | Uses real-world textures instead of markers | Product packaging, museum exhibits, manuals |
SLAM | Creates a dynamic 3D map of the environment | AR navigation, gaming, remote assistance |
Outdoor Tracking | Uses GPS, IMU, and vision for large-scale AR | AR tourism, outdoor gaming, city navigation |
Feature | Natural Feature Tracking (NFT) | SLAM | Outdoor Tracking |
Purpose | Track features in images | Build map + track position | Track in real outdoor environment |
Input | Camera images | Camera / LiDAR / sensors | GPS + sensors + camera |
Main Work | Detect & follow points | Localization + Mapping | Large-scale tracking |
Map Creation | ❌ No | ✅ Yes | ✅ (sometimes via SLAM) |
Position Tracking | ❌ Limited | ✅ Yes | ✅ Yes |
Environment | Small/local scenes | Indoor + outdoor | Mainly outdoor |
Complexity | Low | High | Medium to High |
Feature | Natural Feature Tracking (NFT) | SLAM | Outdoor Tracking |
Purpose | Track features in images | Build map + track position | Track in real outdoor environment |
Example | 📱 Instagram filter tracks your face features (eyes, nose) and keeps the filter fixed while you move | 🤖 Robot in a room creates a map of walls and furniture while moving and knows its position | 🚗 Google Maps navigation tracks your location on roads using GPS while you travel |
Types of AR Markers
Template Markers
Simple black-and-white patterns recognized via image matching. Example: square markers with unique internal designs.
2D Barcode Markers
Encoded markers like QR codes that store additional data (IDs, URLs). More robust and information-rich than template markers.
Imperceptible Markers
Invisible to the naked eye — use infrared or UV light. Preserve visual aesthetics while enabling precise tracking in advanced AR systems.
Camera Pose Estimation
What Is Camera Pose?
Camera pose describes exactly where the camera is in 3D space and how it is oriented relative to a detected marker.
Pose Estimation Steps
1
Detect Marker Corners
Locate the four corner points of the marker in the image.
2
Match Stored Pattern
Compare against the known marker template in the database.
3
Estimate Transformation
Calculate the spatial relationship between marker and camera.
4
Compute Pose Matrix
Output a transformation matrix used for rendering.
Position
x, y, z coordinates in 3D space
Orientation
Rotation along each axis
Mathematical Representation — Transformation Matrix
In AR, all spatial transformations are encoded as matrix multiplications. The complete transformation combines translation, rotation, and scaling into a single compact form.
Basic Formula
Transformation
Translation × Rotation × Scaling
The 4×4 Transformation Matrix T
Why a 4×4 Matrix?
Using homogeneous coordinates allows both rotation and translation to be expressed as a single matrix multiplication — essential for efficient real-time rendering in AR pipelines.
R (3×3)
Encodes rotation along x, y, and z axes
t (vector)
Encodes the 3D translation (position offset)
Final Rendering Equation
Once the pose matrix is computed, the virtual object is projected onto the screen using the standard graphics pipeline formula:
Projection Matrix
Converts 3D coordinates to 2D screen space using camera intrinsics (focal length, sensor size).
View Matrix
Represents the camera's pose — where it is and which way it is looking in the scene.
Model Matrix
Defines the virtual object's own position, rotation, and scale in world space.
Visual Tracking Techniques
Frame-by-Frame Tracking
Visual tracking continuously analyzes the live camera feed, updating the marker's position and orientation with every new frame to keep virtual objects locked in place.
Edge Detection
Identifies marker boundaries by finding sharp intensity changes in the image.
Corner Detection
Uses algorithms like Harris Corner Detector to find stable reference points on the marker.
Pattern Matching
Compares detected regions to stored marker templates for robust identification.
Marker-less Tracking in AR
Unlike marker-based approaches, marker-less tracking uses natural features and sensors to anchor digital content to the real world — no predefined targets required.
CHAPTER 2
"Track real-world features instead of artificial markers"
Localization-Based Augmentation
Position estimation techniques anchor AR content to real-world coordinates using a combination of hardware and algorithmic inputs.
GPS
Outdoor AR positioning via satellite-based geolocation
IMU Sensors
Accelerometer and gyroscope data for orientation and movement
SLAM
Simultaneous Localization and Mapping for dynamic environments
SLAM: Mapping and Tracking in Real Time
How SLAM Works
SLAM is the backbone of modern marker-less AR. It solves two interdependent problems simultaneously — without requiring any pre-existing map.
Map Building
Constructs a spatial model of the unknown environment in real time using visual and depth data
Camera Tracking
Continuously estimates the camera's position and orientation relative to the evolving map
Real-World Applications
Marker-less AR is already embedded in consumer and enterprise products across industries.
AR Navigation
Google Maps Live View overlays directional cues onto live camera feeds for pedestrian navigation
Furniture Placement
IKEA Place uses surface detection to preview true-to-scale furniture in your space before purchase
Mobile Gaming
Pokémon GO anchors virtual creatures to GPS coordinates and real-world surfaces
Industrial AR
Maintenance systems overlay repair instructions and diagnostics onto physical machinery
CHAPTER 3
Visual Tracking
Camera-Based Object Tracking
Visual tracking processes raw camera input using image analysis algorithms to follow objects or surfaces across frames — no markers needed.
Optical Flow
Tracks pixel motion between consecutive frames to infer movement direction and velocity
Template Matching
Compares regions of interest against a reference template to detect and follow known objects
Contour Detection
Identifies and tracks object boundaries using edge-detection algorithms
Feature-Based Tracking
Detects unique keypoints in a scene and matches them across frames to estimate camera motion and object pose.
SIFT
Scale-Invariant Feature Transform — robust to scale and rotation changes; computationally heavier
SURF
Speeded-Up Robust Features — faster approximation of SIFT using integral images and Hessian matrices
ORB
Oriented FAST and Rotated BRIEF — lightweight, real-time capable, and royalty-free; preferred for mobile AR
Feature-Based Tracking Pipeline
Estimate Motion
Compute pose via homography or PnP
Match Features
Find correspondences across frames
Extract Descriptors
Compute compact feature vectors
Detect Keypoints
Identify salient points in image
This four-stage pipeline is the core engine behind algorithms like ORB and SIFT — converting raw pixel data into reliable pose estimates for stable AR rendering.
Hybrid Tracking
Combines vision-based and sensor-based inputs for greater reliability across challenging real-world conditions.
Vision-Based
Camera and image processing for spatial understanding
Sensor-Based
IMU, depth, and LiDAR data for motion and orientation
Key Advantage
Maintains accuracy in low-light conditions and during occlusion
Initialization and Recovery
Initialization
The first step of any tracking session. The system identifies the starting position by detecting reliable features or surface planes and establishing an initial coordinate frame for AR content placement.
Poor initialization is the most common cause of unstable AR rendering — robust feature detection at startup is critical.
Tracking Recovery
Triggered when tracking is lost — due to fast motion, occlusion, or sudden lighting change. The system re-detects keypoints or re-localizes against the existing map to resume stable tracking.
SLAM-based systems can re-localize against a previously built map, enabling faster and more reliable recovery.
Marker-Based vs. Marker-less
Understanding the trade-offs helps teams choose the right tracking strategy for each deployment context.
Feature
Marker-Based
Marker-less
Requirement
Predefined printed marker
No marker needed
Accuracy
High (controlled environments)
Moderate (scene-dependent)
Flexibility
Low — tied to marker placement
High — works in open environments
Complexity
Simple to implement
Algorithmically complex
Use Case
Education, demos, product packaging
Real-world AR apps, navigation, gaming
For production-grade AR targeting uncontrolled environments, marker-less tracking is the preferred foundation — despite its higher implementation complexity.