1 of 89

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

Kenneth Wong

Dec 2021

​

kennethwong12.netlify.app

medium.com/@khwongk12

@Kenneth_KHW

​

2 of 89

Rundown

Part I: Background Concepts of ModelBuilder

  • Grasp the understanding of the components of ModelBuilder
  • Methods to “abstract” your analytical process in ArcPro

Part III: Labs (4 in total)

  • How to actually use the functionalities of ModelBuilder
  • Create a pipeline for analysis process
  • Build geoprocessing functions
  • Best way to learn is to gain hands-on experience

2

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

3 of 89

Topics covered

  • Understand basic concepts of ModelBuilder
  • Add data (inputs) and geoprocessing tools (functions) to ModelBuilder
  • Modify elements in ModelBuilder
  • Connect data and tools
  • Validate a model
  • Run the model in ModelBuilder
  • Edit existing model
  • Create new variable from tools
  • Create model parameters
  • Run the model as a geoprocessing tool

3

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

4 of 89

About Me

  • Teaching Assistant & Researcher (Urban Analytics) @ HKU
  • Information Designer/Map Maker (Cartography and Data visualisation)
  • Spatial data Analyst
  • Writer on urban analytics and urban planning

4

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

5 of 89

What I do: analyse spatial data, make maps and write about cities

5

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

6 of 89

6

medium.com/@khwongk12

​

twitter.com/Kenneth_KHW

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

7 of 89

ModelBuilder: Basic Concepts

7

What, How and Why

8 of 89

Workflow as flowchart

8

Image Source: https://cargocollective.com/loveallthis/Hey-Jude-other-flowcharts

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

9 of 89

Geoprocessing workflow as Model

9

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

10 of 89

ModelBuilder: What is it?

“a visual programming language for building geoprocessing workflows”

– From https://pro.arcgis.com/en/pro-app/latest/help/analysis/geoprocessing/modelbuilder/what-is-modelbuilder-.htm

Visual programming language:

  • Computation logic represented in diagrams

Building geoprocessing workflows:

  • Utilise several existing tools and specify their sequences

10

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

11 of 89

ModelBuilder: How does it work?

A graphical user interface in ArcPro to build, validate, run (and more) models

11

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

12 of 89

ModelBuilder: Why need to use it?

  • “Chaining up/connecting” multiple geoprocessing functions and data
  • Iteratively process every feature class, raster, file, or table in a workspace
    • Save your time in doing repetitive tasks
  • Visualize your workflow sequence as an easy-to-understand diagram
    • "button clicker" vs. coding
  • Remove unnecessary mid-products
    • Clarify/Clean your analysis process

​

ModelBuilder is a productivity tool

12

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

13 of 89

ModelBuilder: Basic Vocabs

​

13

14 of 89

Elements of flowchart

  • Nodes and links
  • Nodes:
    • Shape
    • Color
  • Links:
    • Line type
    • Direction

14

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

15 of 89

Elements of ModelBuilder

Geoprocessing tools (Functions)�- Perform various operations on data�- Accepts inputs and produces some output

Variables (Data)�- “Hold” a value or a reference to data

Connectors (Links)�- connect data and values to tools

Groups �- Visual elements that group related tools together (Does not alter the model)

15

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

16 of 89

Data variable: Input Data

Input data variables are model elements that store paths and other properties of data on disk. Common data variables include feature class, feature layer, raster dataset, and workspace.

​

​

16

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

17 of 89

Data variable: Output/Derived Data

Derived or output data is new data created by a tool in the model. When a geoprocessing tool is added to a model, variables for the tool's output parameters are automatically created and connected to the tool.

17

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

18 of 89

Value variable

Value variables are values such as strings, numbers, Boolean (true/false values), spatial references, linear units, or extents. Value variables contain anything but references to layers or data stored on disk.

​

18

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

19 of 89

Tool: Geoprocessing tool

Tools are geoprocessing tools added to the model. These include all tools you will find in a system toolbox as well as custom model and script tools.

19

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

20 of 89

Advanced tools (ModelBuilder-only tools)

Iterators:

  • repeat the same operation or loop through a set of input data or values.

Logical tools:

  • For loops / switch cases / if-else conditions
  • branch the model based on different conditions.
  • control the logical flow of models.

20

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

21 of 89

Model parameters

Output variables

  • added to the map when the model is run as a geoprocessing tool.

Model parameters

  • appear as tool parameters when the model is run as a geoprocessing tool, allowing you to specify different data or values for your model to process than what was specified when the model was created

​

​

21

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

22 of 89

Lab 0: ModelBuilder Scratchpad

22

23 of 89

Background

  • You are investigating the morphology (urban layout) of Kwun Tong
  • As a starting point, you want to define the boundary of your study: a 500 m buffer (straight-line distance) from all MTR station exits in Kwun Tong

23

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

24 of 89

Normal workflow w/o ModelBuilder: Run the Buffer geoprocessing tool

24

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

25 of 89

Results: 500 m buffer

25

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

26 of 89

How to use ModelBuilder to turn this geoprocessing pipeline into a model?

26

27 of 89

ModelBuilder Ribbon

27

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

28 of 89

Add model components

28

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

29 of 89

Connect the nodes (input data and tool)

29

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

30 of 89

Specify parameters of a function

30

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

31 of 89

Connected model

31

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

32 of 89

Validate and Run a model

32

Ran process will have a grey drop-shadow

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

33 of 89

Q: How to add results in the Model to the map?

33

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

34 of 89

Q: Where are my computed data stored?

34

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

35 of 89

Recap: What you have done

  • Open ModelBuilder
  • Add data (inputs) and geoprocessing tools (functions) to ModelBuilder
  • Modify elements in ModelBuilder
  • Validate a model
  • Run the model in ModelBuilder
  • Add the results (derived data in ModelBuilder) back into the map

35

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

36 of 89

Lab 1: Chaining up process

36

37 of 89

Background

  • You are investigating the morphology (urban layout) of Kwun Tong
  • As a starting point, you want to define the boundary of your study: a 500 m buffer (straight-line distance) from all MTR station exits in Kwun Tong
  • After that, you would like to create a feature class/dataset of road network within the buffer for future analysis

​

37

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

38 of 89

Background

  • You have spatial analysis workflow of computing values of a raster grid
  • Add 2 rasters together
  • Create a buffer from a point
  • Use the buffer polygon to clip the raster

​

  • Use Tool A,
  • Throw the output data as input of Tool B
  • Throw the output data as input of Tool C
  • …
  • Get the “real” output data

38

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

39 of 89

1) Buffer to create the 500 m buffer from MTR exits

39

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

40 of 89

2) Clip to extract the data only within the clipping polygon

40

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

41 of 89

How to use ModelBuilder to turn this geoprocessing pipeline into a model?

41

42 of 89

Q: How to edit an existing model?

Catalog > Toolboxes > Right-click your model > Edit

42

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

43 of 89

The existing model

43

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

44 of 89

Add new input data (road centreline) and function (clip)

44

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

45 of 89

Connect the components

45

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

46 of 89

Auto layout

46

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

47 of 89

Run the model and add result data to display

47

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

48 of 89

Recap: What you have done

  • Edit existing model
  • Add data (inputs) and geoprocessing tools (functions) to ModelBuilder
  • Modify elements in ModelBuilder
  • Connect data and tools
  • Validate a model
  • Run the model in ModelBuilder

48

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

49 of 89

When to use it?

  • “Chaining up/connecting” multiple geoprocessing functions and data
  • Iteratively process every feature class, raster, file, or table in a workspace
  • Visualize your workflow sequence as an easy-to-understand diagram
  • Remove unnecessary mid-products that pollutes your working environment

​

49

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

50 of 89

Lab 2: Create custom geoprocessing function

50

51 of 89

Background

  • There are various collection points in Hong Kong that collects different types of materials for recycling
    • https://geodata.gov.hk/gs/view-dataset?uuid=8379b096-2c23-4dfb-9e4c-5c60d4683b8f
  • You need to count the number of specific type of materials collected in each district
    • e.g. Only count collection points which rechargeable battery
  • You need to share analysis results (and data) to your colleagues. However, the file format they need are different:
    • some need a table (.csv) for quick reference
    • some need shapefile (.shp) for further analysis in ArcPro
    • some need web-friendly data format like geojson (.geojson) for updating the web dashboard

51

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

52 of 89

Open data of recycling collection points in HK

52

https://geodata.gov.hk/gs/view-dataset?uuid=8379b096-2c23-4dfb-9e4c-5c60d4683b8f

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

53 of 89

0) JSON To Features to convert downloaded geojson (.geojson) to feature class

53

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

54 of 89

1) Select Layer By Attribute to select points which collects rechargeable battery

54

Types_of_Material_Collection LIKE '%RECHARGEABLE BATTERY%'

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

55 of 89

2) Summarise Within to count number of selected points within each district

55

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

56 of 89

Symbolise the number of facilities

56

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

57 of 89

3) Export the results as: 1. Shapefile (.shp); 2. Geojson (.geojson); 3. Table (.csv)

57

1

Shapefile

2

Geojson

3

CSV table

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

58 of 89

Geoprocessing tools we used

  1. Select by Attributes to select Recycling Points with materials collected we are interested
  2. Summarise Within to summarise the number of selected Recycling Points within each district
  3. Export:
    1. Feature Class to Feature Class to export as shapefile
    2. Features to JSON to export as geojson
    3. Table to table to export the attribute table

58

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

59 of 89

How to use ModelBuilder to turn this geoprocessing pipeline into a model?

59

60 of 89

Turning geoprocessing pipeline into model

60

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

61 of 89

How to turn it into a single function and allow users to specify the collection materials ?

61

62 of 89

Model parameters

Output variables

  • added to the map when the model is run as a geoprocessing tool.

Model parameters

  • appear as tool parameters when the model is run as a geoprocessing tool, allowing you to specify different data or values for your model to process than what was specified when the model was created

​

​

62

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

63 of 89

Create additional Variables (parameters) of the functions

63

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

64 of 89

Create model parameters

64

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

65 of 89

Rename the model parameters shown in the output

65

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

66 of 89

66

Before

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

67 of 89

67

After

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

68 of 89

Run model as a geoprocessing tool

Catalog > Toolboxes > Double-click Your model

68

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

69 of 89

Run model as a geoprocessing tool

69

After

Before

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

70 of 89

Reorder the parameters in the tool window

70

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

71 of 89

What is the number of recycling points collecting fluorescent lamps in each district?

71

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

72 of 89

What is the number of recycling points collecting fluorescent lamps in each district?

Specify the parameters and generate the results automatically!

72

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

73 of 89

Output .shp .csv & .geojson in one single (large) geoprocessing function

73

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

74 of 89

Symbolise the number of recycling points in each district

74

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

75 of 89

75

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

76 of 89

Recap: What you have done

  • Create new variables from functions
  • Create parameters of a model
  • Reorder sequence of parameters in the model
  • Run the model as a geoprocessing tool

76

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

77 of 89

When to use it?

  • “Chaining up/connecting” multiple geoprocessing functions and data
  • Iteratively process every feature class, raster, file, or table in a workspace
  • Visualize your workflow sequence as an easy-to-understand diagram
  • Remove unnecessary mid-products that pollutes your working environment

​

77

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

78 of 89

Bonus Task

  • Your colleagues are not satisfied by the results of grouping the recycling points by 18 districts - they believe other spatial units could fit better the needs of analysis.
  • you want to change the model to incorporate any spatial units, like:
    • By TPUs
    • By LSBGs
    • … any other possible boundaries
  • How could you edit the model in order to fit the need of this task?

78

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

79 of 89

Lab 3: Site search

79

80 of 89

Background

Site selection: Spatial analysis method for determining the best site(s) for certain uses, such as a new store, landfill, school, recreation center, vineyard, or the location of plant or animal habitats\

Site selection criteria: the basis for finding geographic areas (in polygon format or raster cells) that are most suitable.

80

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

81 of 89

81

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

82 of 89

What have you learnt?

82

83 of 89

Using ModelBuilder

  • Understand basic concepts of ModelBuilder
  • Add data (inputs) and geoprocessing tools (functions) to ModelBuilder
  • Modify elements in ModelBuilder
  • Connect data and tools
  • Validate a model
  • Run the model in ModelBuilder
  • Edit existing model
  • Create new variable from tools
  • Create model parameters
  • Run the model as a geoprocessing tool

83

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

84 of 89

ModelBuilder: Why need to use it?

  • “Chaining up/connecting” multiple geoprocessing functions and data
  • Iteratively process every feature class, raster, file, or table in a workspace
    • Save your time in doing repetitive tasks
  • Visualize your workflow sequence as an easy-to-understand diagram
    • "button clicker" vs. coding
  • Remove unnecessary mid-products
    • Clarify/Clean your analysis process

​

ModelBuilder is a productivity tool

84

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

85 of 89

ModelBuilder as a tool

As a tool, as you learn some advanced spatial analysis methods and tools in the future, the usage of ModelBuilder may be limited.

85

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

86 of 89

ModelBuilder as a tool to conceptualise workflow

As a tool, as you learn some advanced spatial analysis methods and tools in the future, the usage of ModelBuilder may be limited.

However, as a tool to conceptualize/visualise your workflow, ModelBuilder would be useful regardless of your technical skills

86

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

87 of 89

What we missed - programming concepts

Iterators

ModelBuilder Report

Inline variable substitution

Flow control

87

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

88 of 89

Where to learn more?

ESRI provides tons of user manual and well-tempered documentation on their tools. Always read it if you would like to master the tool.

https://pro.arcgis.com/en/pro-app/latest/help/analysis/geoprocessing/modelbuilder/what-is-modelbuilder-.htm

88

Using ModelBuilder to Build Geoprocessing Workflows for Urban Analytics

89 of 89

Thank you!

​

kennethwong12.netlify.app

@Kenneth_KHW

​

89

Medium Articles

Portfolio