Advanced Computer Vision

BTAIC702 | Semester VII

CO1: Demonstrate a solid understanding of fundamental computer vision & image processing concepts.

Prepared By: Mr. A. D. Thorat

UNIT 1 -Introduction to Computer Vision & Image Processing

What is Computer Vision?

• Extracting meaningful information from images/videos
• Mimicking human visual perception capabilities
• Combining image processing with machine learning

CV Applications

• Object detection and recognition
• Medical image analysis
• Autonomous vehicles and robotics
• Face recognition and biometrics
• Video surveillance and tracking

Image Representation

Digital Image: 2D discrete signal

• Pixel: Basic unit with intensity value
• Grayscale: Single channel (0-255)
• Color: RGB channels (3 channels)

Resolution: Width × Height × Channels

Image Processing Operations

• Point Operations: Pixel-by-pixel transformations
• Neighborhood Operations: Using surrounding pixels
• Global Operations: Using entire image

Image Filtering & Convolution

Convolution: Mathematical operation combining two signals

• Kernel/Filter: Small matrix for operations
• Stride: Movement step of kernel
• Padding: Border handling technique

Image Enhancement Overview

Improving image quality for analysis:

• Enhance contrast and brightness
• Reduce noise and artifacts
• Prepare for further processing
• Highlight important features

Contrast Stretching

Expand pixel intensity range

• Stretches narrow intensity range to full range (0-255)
• Improves visibility of low-contrast images
• Linear transformation technique

Formula: O = ((I - min) / (max - min)) × 255

Histogram Equalization

Redistribute intensity values uniformly:

• Improves overall contrast
• Flattens histogram distribution
• Better visibility in dark/bright regions

Limitation: May cause over-enhancement

Adaptive Histogram Equalization (AHE)

Process image in small regions:

• Contrast enhanced locally, not globally
• CLAHE: Contrast Limited AHE
• Prevents noise amplification

Better for medical and specialized images

Wavelet-based Enhancement

Multi-scale image decomposition:

• Decomposes into detail and approximation
• Enhance specific frequency bands
• Preserve edges while reducing noise

Useful for edge enhancement and denoising

Smoothing Filters (Noise Reduction)

Remove noise and reduce high frequencies:

• Linear Filters: Box, Gaussian
• Non-linear Filters: Median, Min, Max

Trade-off: Smoothing vs. detail preservation

Linear Smoothing Filters

Box Filter: Simple average of neighborhood

• Fast computation
• Can blur edges significantly

Gaussian Filter: Weighted average (bell curve)

• Natural smoothing appearance
• Better edge preservation

Non-linear Smoothing Filters

• Median Filter: Middle value of neighborhood
• Min Filter: Minimum value (erosion)
• Max Filter: Maximum value (dilation)
• Better for impulse noise and edge preservation

Sharpening & Edge Enhancement

Laplacian Filter: Second derivative operator

• Enhances edges and fine details
• Sensitive to noise
• Used for edge detection

Opposite of smoothing: Enhance high frequencies

Filtering: Practical Applications

• Denoising medical images
• Enhancing document scans
• Preprocessing for object detection
• Video frame preprocessing

Advanced Computer Vision

BTAIC702 | Semester VII

CO2: Apply various computer vision algorithms and techniques in image processing.

Prepared By: Mr. A. D. Thorat

UNIT 2 -Image Transformation & Restoration

Image Transformation Basics

Convert image to different domain:

• Spatial domain: Original pixel representation
• Frequency domain: Decompose into frequency components
• Other domains: Wavelet, Haar, etc.

Geometric Transformations

Scaling: Resize image dimensions

• Upsampling: Increase resolution
• Downsampling: Reduce resolution

Rotation: Rotate around center point

• Requires interpolation for non-integer angles

Discrete Fourier Transform (DFT)

Converts spatial image to frequency domain:

• Decomposes image into sine and cosine components
• FFT: Fast Fourier Transform for efficiency
• Used for filtering in frequency domain

Output: Magnitude and Phase components

Discrete Cosine Transform (DCT)

Uses only cosine basis functions:

• Similar to DFT but real-valued output
• More suitable for image compression
• Used in JPEG compression

Better energy concentration in fewer coefficients

Discrete Sine Transform (DST)

Uses only sine basis functions:

• Real-valued transform
• Complementary to DCT
• Used in signal processing

Good for handling boundary conditions

Walsh-Hadamard Transform

Uses rectangular pulse basis functions:

• Orthogonal transform with ±1 values
• Fast computation (no multiplication needed)
• Used in pattern recognition

Simpler than Fourier but less optimal

Slant Transform

Combination of Walsh and Fourier properties:

• Orthogonal transform
• Fast computation
• Better energy concentration than Hadamard

Useful in image compression applications

Haar Transform

First wavelet transform basis:

• Orthogonal, non-sinusoidal basis
• Good edge detection properties
• Foundation of wavelet theory

Used in compression and feature extraction

Noise Models Overview

Understanding noise characteristics:

• Additive vs. multiplicative noise
• Statistical properties and distributions
• Source: Sensor, transmission, processing

Critical for selecting restoration methods

Gaussian & Rayleigh Noise

Gaussian: Bell curve distribution (most common)

• Natural noise from sensors
• Symmetric distribution

Rayleigh: Asymmetric, positive values

• Range imaging, radar applications

Erlang & Exponential Noise

Erlang: Generalized exponential distribution

• Used in modeling waiting times
• Positive values only

Exponential: Memoryless distribution

• Rapidly decreasing tail

Uniform & Salt & Pepper Noise

Uniform: Equal probability across range

• Random noise with constant PDF

Salt & Pepper: Impulse noise

• Random black (0) and white (255) pixels
• Common in digital transmission

Mean Filters for Restoration

Arithmetic, Geometric, Harmonic averaging:

• Arithmetic: Simple average (Gaussian noise)
• Geometric: Multiplicative noise
• Harmonic: Brightness preservation

Trade-off: Noise reduction vs. detail loss

Contraharmonic Mean Filter

Handles mixed noise effectively:

• Parameter Q controls noise type response
• Q > 0: Removes dark spots
• Q < 0: Removes bright spots

Good for salt & pepper noise restoration

Median Filter

Non-linear ordering-based filter:

• Replace pixel with median of neighborhood
• Excellent for salt & pepper noise
• Preserves edges better than mean

Widely used in medical and real-world applications

Midpoint Filter

Average of min and max in neighborhood:

• Combines min/max filter benefits
• Good for Gaussian and uniform noise
• Preserves fine details

Balance between noise reduction and detail

Restoration Workflow

• 1. Identify noise type from image analysis
• 2. Select appropriate filter method
• 3. Apply restoration with parameters
• 4. Evaluate quality metrics (PSNR, SSIM)

Evaluating Restoration Quality

PSNR: Peak Signal-to-Noise Ratio

• Higher values indicate better quality

SSIM: Structural Similarity Index

• Perceptual quality metric
• Closer to 1 is better

Restoration Applications

• Medical image denoising
• Satellite image processing
• Old photo restoration
• Video noise reduction

Advanced Computer Vision

BTAIC702 | Semester VII

CO3: Apply various computer vision algorithms and techniques to solve real-world engineering problems, such as object recognition, motion analysis, and texture.

Prepared By: Mr. A. D. Thorat

UNIT 3 - Segmentation, Texture & Motion Analysis

Image Segmentation

Divide image into meaningful regions:

• Group pixels into homogeneous regions
• Identify object boundaries
• Prepare for object-level analysis

Edge Detection Overview

Find object boundaries using gradients:

• Edges: Sharp changes in intensity
• First derivative: Detects edges
• Multiple algorithms with trade-offs

Foundation for segmentation and recognition

Prewitt Edge Detector

3×3 gradient-based operator:

• Separates horizontal and vertical edges
• Simple and fast computation
• Sensitive to noise

Good for high contrast edges

Sobel Edge Detector

Weighted gradient operator:

• Emphasizes central pixels more
• Better noise robustness than Prewitt
• Computes gradient magnitude and direction

Widely used in edge detection pipelines

Canny Edge Detector

Multi-stage optimal edge detection:

• 1. Gaussian smoothing for noise reduction
• 2. Gradient calculation (Sobel)
• 3. Non-maximum suppression
• 4. Double thresholding and edge tracking

Most powerful, used extensively in practice

Thresholding Techniques

Convert grayscale to binary image:

• Global: Single threshold for entire image
• Otsu's Method: Automatic threshold selection
• Local: Adaptive thresholding per region

Critical for segmentation and object detection

Region-based Segmentation

Group pixels into connected regions:

• Flood Fill: Expand from seed pixel
• Region Growing: Merge similar neighbors
• Watershed: Treat gradient as topography

More robust than edge-based methods

Texture Analysis Basics

Visual appearance and surface properties:

• Repeat patterns in image regions
• Important for material classification
• Complementary to shape analysis

Key features: Coarseness, contrast, directionality

Gray Level Co-occurrence Matrix (GLCM)

Statistical texture descriptor:

• Measures pixel pair frequency
• Captures spatial relationships
• Extract features: Contrast, Homogeneity, ASM

Effective for regular and structured textures

Local Binary Patterns (LBP)

Local texture descriptor:

• Compare center pixel with neighbors
• Create binary patterns
• Fast computation and rotation invariant

Widely used in face recognition and detection

Gabor Filters for Texture

Frequency and orientation analysis:

• Gaussian envelope with sinusoidal plane
• Bank of filters at different orientations
• Mimic human visual cortex responses

Excellent for directional texture features

Laws' Texture Energy Measures

Convolution-based texture features:

• Separable filter kernels
• Detect edges, spots, ripples, waves
• Compute energy from filter responses

Efficient alternative to GLCM

Texture-based Segmentation

Group regions with similar texture:

• Extract texture features from regions
• Cluster similar textures together
• Useful for image content analysis

Material classification, scene understanding

Motion Analysis Overview

Analyze movement between frames:

• Optical flow: Pixel motion vectors
• Background subtraction: Detect moving objects
• Motion segmentation: Separate objects by motion

Critical for video understanding and tracking

Optical Flow Estimation

Estimate apparent pixel motion:

• Based on brightness constancy assumption
• Lucas-Kanade & Horn-Schunck methods
• Sparse vs. dense flow

Foundation of video analysis and action recognition

Lucas-Kanade Method

Sparse optical flow estimation:

• Assumes constant motion in small neighborhoods
• Solves for motion vectors of corner points
• Fast and practical implementation

Widely used in feature tracking

Horn-Schunck Method

Dense optical flow estimation:

• Computes flow for every pixel
• Uses smoothness constraint
• Iterative optimization approach

Complete motion field but computationally heavier

Background Subtraction & Deep Optical Flow

Background Subtraction: Detect foreground objects

• Compare with background model (Gaussian Mixture)

FlowNet: Deep learning optical flow

• CNN-based end-to-end flow estimation

Motion-based Segmentation

Segment regions by motion patterns:

• Group pixels with similar motion vectors
• Separate moving objects from background
• Used in video object tracking

Useful for surveillance and action analysis

Advanced Computer Vision

BTAIC702 | Semester VII

CO4:Analyze and interpret results obtained from computer vision algorithms, and critically evaluate their performance and limitations.

Prepared By: Mr. A. D. Thorat

UNIT 4 - Feature Matching Algorithms

Feature Extraction Overview

Extract distinctive, invariant features:

• Local descriptors of image regions
• Robust to scale, rotation, illumination
• Enable matching and recognition

SIFT: Scale-Invariant Feature Transform

Most influential feature descriptor:

• Detects keypoints at multiple scales
• Invariant to rotation and scale
• Robust to occlusion and noise

Industry standard for many years

SIFT: Keypoint Detection

Find characteristic points:

• Build scale space with Gaussian blur
• Compute Difference of Gaussians (DoG)
• Find local extrema across scales

Detects corners and blobs at multiple scales

SIFT: Descriptor Computation

Create rotation-invariant descriptors:

• Assign dominant orientation
• Compute gradient histogram in sub-regions
• Normalize for illumination invariance

128-dimensional vector per keypoint

SURF: Speeded-Up Robust Features

Faster SIFT alternative:

• Uses integral images for fast computation
• Hessian matrix for keypoint detection
• 64-dimensional descriptors

Good speed-accuracy trade-off

BRISK: Binary Robust Invariant Scalable Keypoints

Binary descriptor for efficiency:

• Fast binary comparison instead of floating point
• AGAST keypoint detection
• Rotation and scale invariant

Ideal for mobile and real-time applications

Feature Descriptor Comparison

SIFT

Accurate, slow, patented

SURF

Fast, good accuracy

BRISK

Very fast, binary

Feature Representation

Build features for classification:

• Create feature datasets from images
• Normalize and standardize features
• Prepare training data for classifiers

Critical preprocessing step

Bag-of-Words Model

Image representation for classification:

• Create visual vocabulary (codebook)
• Quantize descriptors to visual words
• Histogram of word occurrences

Effective for image classification tasks

Vector Quantization

Discretize feature space:

• K-means clustering of feature descriptors
• Assign each descriptor to nearest cluster
• Create vocabulary of K visual words

Reduces feature dimensionality

Feature Classification Methods

Classify images using features:

• Support Vector Machines (SVM)
• K-Nearest Neighbors (KNN)
• Random Forest

Choose based on data and computational constraints

Support Vector Machines (SVM)

Find optimal decision boundary:

• Maximize margin between classes
• Kernel tricks for non-linear separation
• Robust to outliers with soft margins

Excellent for binary and multi-class problems

K-Nearest Neighbors (KNN)

Instance-based classification:

• Find K closest training samples
• Vote for majority class
• Simple, non-parametric approach

Good baseline, sensitive to K selection

Random Forest Classifier

Ensemble of decision trees:

• Multiple random trees with bootstrap samples
• Aggregate predictions (voting)
• Reduce overfitting through ensemble

Robust and handles high-dimensional data well

Feature Matching Techniques

Match features between images:

• Nearest neighbor matching
• Lowe's ratio test for ambiguity
• RANSAC for robust matching

Foundation for image stitching and registration

RANSAC: Robust Matching

Remove outliers in feature matches:

• Random sample consensus algorithm
• Fit model to random samples
• Keep matches consistent with model

Essential for image matching robustness

Feature Matching Applications

• Image stitching and panorama creation
• 3D reconstruction from multiple views
• Copy detection and plagiarism
• Object instance recognition

Feature Extraction Pipeline

• 1. Select feature detector (SIFT, SURF, BRISK)
• 2. Extract keypoints and descriptors
• 3. Build feature representation (BoW, etc.)
• 4. Train classifier (SVM, RF, KNN)
• 5. Evaluate and optimize performance

Best Practices & Considerations

Optimize feature extraction:

• Normalize features across dataset
• Handle imbalanced training data
• Tune classifier hyperparameters
• Cross-validate for robustness

Advanced Computer Vision

BTAIC702 | Semester VII

CO5:Implement and evaluate computer vision algorithms using programming languages and libraries commonly used in the field, such as Python and OpenCV.

Prepared By: Mr. A. D. Thorat

UNIT 5 - Computer Vision with Deep Learning

Deep Learning in Computer Vision

Revolutionary approach to visual recognition:

• Learns features automatically from data
• End-to-end trainable architectures
• Superior performance on complex tasks

Convolutional Neural Networks (CNN)

Specialized for image processing:

• Convolution layers: Feature extraction
• Pooling layers: Dimension reduction
• Fully connected: Classification

Backbone of modern computer vision

CNN Architecture Design

Sequence of operations:

• Early layers: Low-level features (edges)
• Middle layers: Mid-level features (shapes)
• Deep layers: High-level features (objects)

Hierarchical feature learning

Classic CNN Architectures

• LeNet: Early CNN (handwriting)
• AlexNet: Deep learning revolution
• VGGNet: Simple, deep networks
• ResNet: Residual connections

Attention Mechanisms

Focus on important image regions:

• Channel attention: Important features
• Spatial attention: Important locations
• Improves model accuracy and interpretability

Modern enhancement to CNN architectures

Vision Transformers (ViT)

Transformer architecture for vision:

• Divide image into patches
• Self-attention across patches
• Challenge to CNN dominance

State-of-the-art on many benchmarks

Image Classification with Deep Learning

Assign category labels to images:

• ImageNet: 1000 categories benchmark
• Transfer learning: Pretrained models
• Domain adaptation techniques

Practical applications: Medical, autonomous systems

Generative Models Overview

Generate new images from data:

• Generative Adversarial Networks (GANs)
• Variational Autoencoders (VAE)
• Diffusion models

Create synthetic images and style transfer

Generative Adversarial Networks (GAN)

Generator vs. Discriminator game:

• Generator: Creates realistic images
• Discriminator: Detects fake images
• Adversarial training improves both

Applications: Style transfer, image synthesis

Object Detection Overview

Localize and classify objects:

• Predict bounding boxes and labels
• Handle multiple objects per image
• Performance metrics: mAP, IoU

Critical for surveillance, autonomous vehicles

Region-based CNN (R-CNN)

First deep learning object detector:

• Generate region proposals with selective search
• Classify each region with CNN
• Slow but accurate approach

Foundation for modern detectors

Fast R-CNN

Improved efficiency and accuracy:

• Single CNN on entire image
• RoI pooling: Extract fixed-size features per region
• Faster training and inference

Practical improvement over R-CNN

Faster R-CNN & Mask R-CNN

Faster R-CNN: Region Proposal Networks

• Learnable region proposals instead of selective search
• End-to-end trainable

Mask R-CNN: Instance segmentation

• Adds pixel-level mask prediction

SSD: Single Shot Detection

Fast one-stage detector:

• Predict boxes and classes in single forward pass
• Multi-scale feature maps
• Real-time performance

Good balance between speed and accuracy

YOLO: You Only Look Once

Fastest real-time detector:

• Treat detection as regression problem
• Grid-based approach
• YOLOv3, v4, v5: Continuous improvements

Industry standard for real-time applications

Semantic Segmentation

Pixel-level classification:

• Assign class label to every pixel
• Encoder-decoder architecture
• Applications: Autonomous driving, medical imaging

Precise object boundary delineation

U-Net: Semantic Segmentation

Popular encoder-decoder network:

• Encoder: Downsample to capture context
• Skip connections: Preserve spatial information
• Decoder: Upsample for detailed segmentation

Standard for medical image segmentation

Centroid-based Object Tracking

Follow objects across frames:

• Detect centroids in each frame
• Compute distance between frames
• Associate centroids across time

Simple but effective approach for multi-object tracking

Deep Learning: Applications & Challenges

Applications: Autonomous vehicles, medical diagnosis, robotics

Challenges: Data requirements, computational cost, interpretability

Active research area with rapid advancement