Introduction and Course Overview

Ujaval Gandhi

ujaval@spatialthoughts.com

Mastering GDAL Tools

​

CC BY-NC 4.0

Introduction

• Background in GIS and Remote Sensing
• Intern at Indian Institute of Remote Sensing (IIRS), Dehradun, India
• MS in Geospatial Information Engineering from University of Wisconsin - Madison, USA
• 15 years of professional experience
• One of the early employees at Google Inc.
• Moved to India in 2006 and established the maps team
• Led the GIS and Aerial Imagery team in India from 2007-2019
• Wrote pipelines for processing terabytes of satellite and aerial imagery.
• Presently
• Left Google in 2020 to work on startup Spatial Thoughts.
• Building open-source content and helping people master advanced geospatial skills.

Ujaval Gandhi

Introduction

• Masters in Remote Sensing
• Interest in Automating Workflows
• Training Associate
• Class logistics
• Project support

​

Vigna Purohit

What is GDAL?

• Geospatial Data Abstraction Library (GDAL)
• Software library for reading, writing, and conversion between Raster and Vector data formats
• All major geospatial software uses GDAL under the hood
• QGIS, ArcGIS, Google Earth, FME etc.

What is OGR?

• OpenGIS Simple Features Reference Implementation
• Now it is simply called OGR Simple Features Library
• A library for working with vector data.
• Now it is integrated with GDAL. GDAL is now also referred to as GDAL/OGR

GDAL/OGR Utility Programs

Why use GDAL/OGR Commands?

• Free and Open-source software
• Easy to batch process and automate data processing
• Can run on servers/remote machines without GUI
• Often faster than using Desktop based applications
• Advanced options for compression and publication
• Easy to script ETL pipelines
• Unique capabilities not available in other software packages

Why learn GDAL/OGR Tools?

Image Source: Matt Forrest

How do you pronounce GDAL?

Course Overview

Merging Tiles

Converting Formats

Compressing Output

Setting NoData Values

Writing Cloud-Optimized GeoTIFF (COGs)

​

Processing Elevation Data

​

Basic Raster Processing

(3 Hours)

Creating Previews

Create a Tile Index

Mosaic and clip to AOI

Creating Overviews

​

Creating RGB and False Color Composites

Histogram Stretch and Color Correction

Raster Algebra

Pan Sharpening

​

Processing Aerial and Satellite Imagery

(3 Hours)

​

Listing WMS Layers

Downloading WMS Layers

​

Georeference with Bounding Box Coordinates

Georeference using GCPs

​

Read a CSV data source

convert it to point data

Assign CRS

Extract subset

​

​

WMS, Georeferencing and ETL Basics

(3 Hours)

​

​

Merging Vector Files

Reproject vector layers

Creating Buffers

Perform Spatial Queries

Data Cleaning

​

Running commands in batch

Automating OGR/GDAL Jobs

Geoprocessing and Spatial Queries

(3 Hours)

Assignment and Evaluation

Assignment*

(30 mins)

​

​

Assignment

​

Session 1

Session 2

Module 3

Module 4

Class Exercises

Live Quizzes

Class Exercises

Live Quizzes

* Assignment is due before the start of Session 3

​

Live Session Structure

• All the live sessions are hands-on.
• We will have 10-15 minutes of hands-on work on a problem, followed by a 5-minute exercise
• All participants are expected to complete the exercises during the class.
• You can ask questions at any point during the session. Just unmute and ask, or use the chat.
• We will have a break of 10 minutes during each 3-hour session.
• All sessions are recorded and will be uploaded to the class page. If you miss any session, watch the recording and submit the exercises by email.

Certification

• Each participant would be given a certificate of completion
• Certificates are issued by Spatial Thoughts and come with a unique ID that can be verified at spatialthoughts.com/verify
• To earn the certificate
• Participants must be enrolled in a course
• Submit all exercises and assignment*

​

*If you miss any session, submit the exercises by email

Let’s get started

Open a Terminal

Windows

• Search for Anaconda Prompt* in the Windows Start Menu and click on it.
• Type conda activate gdal to activate the environment.
• Type gdalinfo —version to check if the GDAL tools are installed.

​

* Do NOT use Anaconda Powershell Prompt

Mac/Linux

• Open a new Terminal
• Type conda activate gdal to activate the environment.
• Type gdalinfo —version to check if the GDAL tools are installed.

Locate the Data Folder

Windows

• Use cd command to navigate to the gdal-tools folder.
• Use dir command to list and check files.

Example, if you unzipped the gdal-tools.zip in your Downloads folder, you would do this.

cd Downloads\gdal-tools

dir

Mac/Linux

• Use cd command to navigate to the gdal-tools folder.
• Use ls command to list and check files.

Example, if you unzipped the gdal-tools.zip in your Downloads folder, you would do this.

cd Downloads/gdal-tools

ls

​

Notes for Windows Users:

• If your data folder is in on another drive, change the drive letter first. (D: to change to D drive)
• You may have your Downloads folder in OneDrive at C:\Users\User\OneDrive\Downloads

Overview of GDAL/OGR Programs

GDAL Programs

• gdalinfo: Lists source and metadata
• gdal_translate: Format conversion
• gdalwarp: Resampling and Reprojection

GDAL Programs (contd.)

• gdaladdo
• gdaltindex
• gdalbuildvrt
• gdaldem
• gdal_rasterize

​

• gdal_contour
• gdal_grid
• gdaltransform
• nearblack
• gdallocationinfo
• gdalsrsinfo
• gdal-config
• gdalmanage
• gdal_viewshed
• gdal_create

GDAL Python Scripts

• gdal_retile.py
• rgb2pct.py
• pct2rgb.py
• gdalattachpct.py
• gdalmove.py
• gdal_proximity.py
• gdal_edit.py
• gdal_polygonize.py
• gdal_sieve.py
• gdal_fillnodata.py
• gdal2tiles.py
• gdal2xyz.py

• gdal_merge.py
• gdal_calc.py
• gdal_pansharpen.py

OGR Programs and Scripts

• ogrinfo: Lists source and metadata
• ogr2ogr: Format conversion and transformation
• ogrmerge.py: Merge vector layers
• ogr_layer_algebra.py: Geoprocessing operations (clip, intersection, union etc.) with vector layers

​

• ogrlineref
• ogrtindex

​

Running GDAL/OGR Programs

• Commands are typically run from a terminal.
• You can specify input and/or output along with options
• Running Pipelines
• Multiple commands can be combined via a script for running a data processing pipeline
• Batch Processing
• Same command can be run on multiple inputs via a script.
• Notebooks
• You can run also the commands via Jupyter notebooks as part of your data science workflow

Specifying Options

• Long Syntax
• Mainly used for common options across all programs
• Example: --help, --formats
• Short Syntax
• Used for program-specific options
• Example: -of, -a_srs