HaMMon: Automated Photogrammetric Workflow for Environmental Digital Twin Generation and Hazard Assessment

Leonardo Pelonero - Mauro Imbrosciano INAF OACT

Spoke 3 III Technical Workshop, Perugia 26-29 Maggio, 2025

Scientific Rationale

HaMMon (Hazard Mapping and vulnerability Monitoring)

​

The project aims at extending the current knowledge in hazard mapping, monitoring and forecasting from industrial perspectives by means of innovative technologies, for the Italian territory

​

The activities involve intensive use of scientific visualization and artificial intelligence technologies, especially for assessing and extracting meaningful information on risk-exposed assets

​

WP 2: Post-event Natural Disasters

​

Objective:

​

• Workflow for data acquisition and creation of digital twin
• Development of algorithms to identify and classify objects and features within 3D models and 2D images
• Enabling analysis of areas affected by extreme natural events that would otherwise not be possible
• Management of a web tool: Displaying digital twins enriched with collected data and analyses

Technical Objectives, Methodologies and Solutions

T2.1: A Science Gateway for Automated Post-Event Analysis and Digital Twin Creation

​

We present a Science Gateway framework for the development of portable and fully automated post-event analysis pipelines integrating Photogrammetry techniques, Data Visualization and Artificial Intelligence technologies, applied on aerial images, to assess extreme natural events and evaluate their

impact on risk-exposed assets

​

The principal scientific contribution consists in the migration and integration of validated but standalone workflows–based on Photogrammetry, AI and Data Visualization–into a single orchestrated pipeline managed through Directed Acyclic Graphs (DAGs) in Apache Airflow, leveraging Common Workflow Language (CWL) for portability and containerized with Docker, within a Science Gateway platform, to assess extreme natural events and analyze their effects on assets at risk

​

KPI:

• Dissemination: submitted on 17th International Workshop on Science Gateways (IWSG2025), 17-19 June 2025
• Code Availability: https://github.com/VisIVOLab/Post-Event-Analysis-Workflow

Technical Objectives, Methodologies and Solutions

Technical Objectives, Methodologies and Solutions

T2.4: Automatic (or semi-automatic) analysis

Technical Objectives, Methodologies and Results

Target: Enhancement of the 3D model with semantic details From 2D data to 3D: Semantic segmentation masks generated from images are converted into three-dimensional maps within digital twins. This enables the highlighting of features such as roads, buildings, water bodies, vehicles, and disaster-related elements like debris, blocked or flooded roads, and damaged or collapsed buildings. Cracks on building facades can also be detected and highlighted.

• Research on ML models for semantic segmentation
• Research on public UAV image datasets concerning post-disaster scenarios
• Research on public datasets for crack segmentation in buildings

UAV nadir images (FloodNet and RescueNet):

• Tiramisù: training completed, results below expectations
• Attention U-net : initial training cycles performed, with promising results
• Fine-tuning of bigger models

​

Cracks on concrete surfaces:

• Tiramisù: test training completed with promising results
• Data Augmentation
• Refinement of the training

Main Results

T2.4: Preliminary Semantic Segmentation Output

Monte Busca�(Tredozio)

Main Results

T2.1: 3D Representation: Tiled Model and Point Cloud

Tiled Model�Monte Busca (Tredozio)

Point Cloud�Monte Busca (Tredozio)

Main Results

T2.1 & T2.4: Point Cloud classification output

Three-dimensional masks of roads and buildings within the point cloud on the original dataset. Preliminary results

Main Results

T2.1: Demo experiments on importing classification into 3D models

2D Masks -> Point Cloud -> Classification -> Coloring -> Colored 3D Model

Point Cloud Classification

Export Point Cloud and overwrite with a chosen color

3D model generated from Point Cloud source

KPI: Code Availability: https://github.com/Fliki1/CesiumDemo

Technical Objectives, Methodologies and Solutions

T2.1: Assessment and testing of CesiumION and CesiumJS platforms for handling 3D data and metadata

​

• Import of Tiled Cesium models compatible in both local and remote modes with the support of CesiumION.
• Import and visualization of models on the Cesium geospatial map, with OpenStreetMap building overlay.
• Management and integration of landslide models through the terrain subtraction approach at the model locations. Pre-existing surface portions that would otherwise cover the model (due to the non-faithful conformity of the geospatial map) are removed.
• Study and comparison of different rendering quality resolutions of 3D tilesets thanks to the LOD (Level of Detail) system.
• Comparison between Tiled model formats and OBJ models (allow the management of a single texture file)
• Integration of interactive KML and GeoJSON structures, with the possibility to interact on these areas to display associated information directly on the map via dedicated menus.
• General study of custom widgets to improve navigation and exploration of models in CesiumJS.

​

KPI:

• Code Availability: https://github.com/Fliki1/CesiumDemo

Main Results

T2.1: Demo experiments on importing classification into 3D models

Example of a Classification Primitive on CesiumJS starting from a point cloud coordinates applied to a Tiled model

​

75k points in 1x1 meter

​

No Mask

​

KPI:

• Code Availability

https://github.com/Fliki1/ClassificationPrimitive

Main Results

• Large landslide damages plant nursery (Quarrata, Pistoia - 16 marzo 2025)�
• Flooding of the Lamone river �(Brisighella, Ravenna - 14 marzo 2025)

https://www.localteam.it/

Main Results

https://www.localteam.it/

Main Results

https://www.localteam.it/

Core Data:

• Georeferenced UAV images
• GPS coordinates of Ground Control Points (GCPs)
• Point clouds generated through image alignment
• 3D Model
• High-resolution texture images projected onto 3D models
• Tiled Model with high LOD
• Semantic Segmentation Mask
• 3D Model and/or Point Cloud classification

​

​

The results are complete

We are proceeding to upload the processed data to the Data Platform Archive (May) �with possible updates or refinements in the following months if necessary

Auxiliary Data:

• Automatic workflow report: parameters and steps used
• Digital Elevation Model for topographic analysis
• Orthophotos: Orthorectified and georeferenced images
• Orthomosaic: Complete mosaic generated from the orthophotos
• (where possible) coverage areas in polygonal format (kml)

​

T2.1: Processing of digital twins of the area of Tredozio (FC): results

Technical Objectives, Methodologies and Solutions

Final Steps

• Evaluation of new methods for importing classification into 3D models

​

• Refining 3D Models

Improve the quality of the 3D models by removing non-relevant parts, ensuring a cleaner and more focused final output

​

• Planning a New Survey�Organize and execute a new mission in a new area affected by natural disasters (Bagnacavallo Ravenna)

​

• Automatic (or semi-automatic) analysis�We have discontinued the Tiramisu experiment and started training the Attention U-Net, which leverages attention mechanisms to better capture contextual information—crucial for understanding the role of specific items in relation to others (e.g., water in natural reservoirs versus water on roadways). Other model architectures are also being considered.
• We are also exploring the idea of generating synthetic datasets to support generalization to non-nadir imagery
• We will study machine learning algorithms that operate directly on 3D artifacts such as point clouds, meshes, and similar structures

Thank you for your time and attention!

Final Steps

T2.4: Using segmentation masks for noise reduction in 3D model generation