1 of 42

�The Smithsonian Institution

Digitization Program Office

Office of the Chief Information Officer

Pushing Productivity

Vince Rossi

3D Program Supervisor �rossiv@si.edu

�A Case Study Using Standard Turntable Photogrammetry

1

@Smithsonian3D

2 of 42

2

@Smithsonian3D

3 of 42

National Museum of Natural History - Paleobiology Dept.

3

Unlock research quality data for the world

Ideal subject matter for photogrammetry

- Focused on holotype specimens

- Affordable & scalable

@Smithsonian3D

4 of 42

2D vs 3D

2D

3D

4

Complexity

Simplicity

@Smithsonian3D

5 of 42

2D vs 3D

5

2D digitization is a commodity

Capture: mass-digi affordably contracted
Processing: solved by mature software tools
Preservation: many DAMs solutions available
Authoring: solved with… a website!
Viewing: solved and standardized

3D digitization. It’s really hard…

Capture: many modalities
Processing: high computation + advanced skill set
Preservation: no reference in the marketplace
Authoring: limited options in the market
Viewing: no interoperability (yet)

.

@Smithsonian3D

6 of 42

3D = Complexity

6

Diffuse

Metallic

Normal

Occlusion

Rough

Geometry

Camera

Lights

Experience

Materials

@Smithsonian3D

7 of 42

Why 3D?

→ Engagement!

7

Unlike other media formats, such as 2D imagery or video where users simply click to view or watch a movie, 3D scenes demand active engagement.

Users need to interact with the model by manipulating it, rotating it, zooming in or out, and clicking on specific story elements to explore further. �
Empowering users to dynamically control their exploration and delve deeper into content.�
Longer dwell time for 3D scenes (2-7m)

@Smithsonian3D

8 of 42

Managing Complexity

Software!

8

Getting complexity out of your way; getting you to your end goal faster.

… a lot faster

Asset Management Project

@Smithsonian3D

9 of 42

Accessible Capture Hardware

9

@Smithsonian3D

10 of 42

The Smithsonian Paleobiology Kit

10

@Smithsonian3D

11 of 42

The Kit in Action

11

big softboxes for diffuse light

peeking inside

@Smithsonian3D

12 of 42

The Kit + Staff Support

12

Collections manager - Pulling objects, object size grouping, barcodes

Object handler - Placing objects

3D technician - Running capture

@Smithsonian3D

13 of 42

Physical Capture Workflow

13

4.

Object Placed

6.

Object Flip

8.

Remove Object

Lens Selection

1.

Barcode

.

3.

2.

Object Staging

5.

Top Scan

Bottom Scan

7.

@Smithsonian3D

14 of 42

Object size vs. Depth of Field Guide

14

Object Size vs DoF Guide�

@Smithsonian3D

15 of 42

Photogrammetry Abstraction

15

Alignment

1.

2.

Reconstruction

Cleaning

.

3.

Texturing

4.

Abstraction provides flexibility to our pipeline tools allowing other solutions such as RC, Meshroom, or any other software to be built into the automation process

@Smithsonian3D

16 of 42

Processing Workflow

16

Batch pickup

Sever picks up images from local machine sends to cook

2.

Agisoft

Optimizes and aligns cameras

3.

Agisoft

Aligned cameras used to reconstruct geometry

4.

Meshlab

Mesh clean-up and object isolation

5.

QC (manual)

Technician manually inspects photogrammetry model

Meshlab

Geometry level of detail (LOD) assets generated

8.

Rizom UV

Create UVs on LOD assets

9.

X-Normal

Projecting textures from high to low geometry

10.

Image Magick

Resizing texture maps per LOD .obj file

11.

Blender

Create GLBs for Voyager deployment

12.

7.

Agisoft

6.

Texture reprojection

Preprocessing

1.

Batch operation for demosaicing, creating photogrammetry images

→→→→→AUTOMATION →→→→→

Alignment

Reconstruction

Cleaning

Texturing

@Smithsonian3D

17 of 42

There is no single solution

17

Meshroom

Metashape

Rizom UV

Blender

Meshlab

Image Magick

Reality

Capture

XNormal

Rapid Compact

Seven Zip

Raw Therapee

3D Slicer

Use the BEST tool for the task

Third-party software downloads required!�

@Smithsonian3D

18 of 42

Cook is an Orchestration Tool

18

Third-party software downloads required!�

Meshroom

Metashape

Rizom UV

Blender

Meshlab

Image Magick

Reality

Capture

XNormal

Rapid Compact

Seven Zip

Raw Therapee

3D Slicer

18

Cook is the orchestration tool that connects and automates third-party software

@Smithsonian3D

19 of 42

Keys to Automation Success Standard Capture Format

19

Image Alignment Optimization: Identifying images that are out of bounds of the expected ring of camera positions and iteratively aligning them with their neighbors.�

Since we are using fixed cameras and a turntable we can make some assumptions about the alignment of the image sets and their relation to the subject of digitization.�

Out of bounds image

@Smithsonian3D

20 of 42

Keys to Automation Success Standard Capture Format

20

Model Axial Alignment: Assuming the capture rings are parallel to the turntable surface, we can deduce vertical axis, and then center the model using it’s bounding box.

Since we are using fixed cameras and a turntable we can make some assumptions about the alignment of the image sets and their relation to the subject of digitization.�

@Smithsonian3D

21 of 42

Keys to Automation Success Standard Capture Format

21

Object Isolation: Identifying the object of interest by shooting a ray down the vertical axis, identifying the first intersected piece of geometry as the object.

Since we are using fixed cameras and a turntable we can make some assumptions about the alignment of the image sets and their relation to the subject of digitization.�

@Smithsonian3D

22 of 42

Automation Success

22

Turntable Capture Standardization

Enabled optimization of photogrammetry data processing.
Reduced need for manual intervention.�

Data Capture in Concentric Circles

Improved automatic detection and realignment of out-of-alignment images.�

Assumptions for Axis Alignment

Images captured parallel to a horizontal plane.
Simplified automatic axis alignment of data.�

Ray Casting for Object Identification

Easier isolation of the object of interest by shooting rays through capture circles.
Eliminated additional geometry effectively.

@Smithsonian3D

23 of 42

Paleobiology Production Project Achievements

23

Capturing 7-17 objects per day.

408 Holotype specimens at the National Museum of Natural History in ~3 months.�
Automatic Photogrammetry via Cook

Currently 84% ‘no-touch’ success rate!
24 hour turnaround�

Human time saved

~228 hours
~28.5 working days�

@Smithsonian3D

24 of 42

24 hour turn-around

24

Platygonus intermedius Gidley, 1920

@Smithsonian3D

25 of 42

Automation Challenges - Object Isolation

25

@Smithsonian3D

26 of 42

Automation Challenges - Object Isolation

26

SUCCESS

FAIL

@Smithsonian3D

27 of 42

Automation Challenges - Object Isolation

27

Multi-ray approach to come

@Smithsonian3D

28 of 42

Automation Challenges - Compute Resources

28

Load Balancing

Future mitigation

@Smithsonian3D

29 of 42

Automation Challenges - Foam Support Ghosting

29

Ghosted

Mitigated

Alternative Supports

Masking

@Smithsonian3D

30 of 42

Automation Challenges - Future Mitigation

30

Isolating Geometry of Interest

Use multi-ray casting from different axes to identify geometry hit by most rays.
Define bounding boxes using object record dimensions.�

Compute Resources

Streamline tool deployment across more machines.
Explore workflow orchestration tool for load balancing.�

Foam Supports Ghosting

Investigate different types of supports for the physical workflow.
Implement intelligent automatic masking, potentially with AI.�

Capture Speed

Replace the (slow) Ortery turntable to significantly increase capture speed.

@Smithsonian3D

31 of 42

Building Capacity at NMNH Invertebrate Zoology Dept.

31

Staffer new to photogrammetry
Turntable 3D capture
Leveraging our software pipeline

3D models generated: 440

Biodiversity & Conservation

Corals: 270

Crustaceans: 76

Bivalves: 157

All Holotypes

@Smithsonian3D

32 of 42

Building Capacity at NMNH Invertebrate Zoology Dept.

32

3 years, 6 months

3 months

Contractor new to photogrammetry!

@Smithsonian3D

33 of 42

Affordable photogrammetry kit

33

@Smithsonian3D

34 of 42

Close to an Open Source Pipeline

34

We are one stage away from a fully open source processing pipeline (Rizom UV)�

Alignment

1.

2.

Reconstruction

Cleaning

.

3.

Texturing

.

4.

Kintsugi 3D

We used Metashape w/ Paleobiology

we do support Meshroom in Cook

…but Meshroom recipe w/ automation optimization is to come

@Smithsonian3D

35 of 42

Open Source Software

35

Asset Storage & Preservation Environment
Workflow Management

Asset Creation & Automated Processing
Photogrammetry Automation

QC, Authoring, Publishing
Viewing, Storytelling, & XR enabled

@Smithsonian3D

36 of 42

36

Voyager Updates

New Contributors
New Accessibility Features
IIIF Involvement

Help us build the community

@Smithsonian3D

37 of 42

Voyager Accessibility Updates

37

Multilingual support
Audio narration with captions
Keyboard control for UI & navigation
Screen reader support for UI & navigation
Exploration into sonification of 3D surfaces
More to come!

38 of 42

Discover Smithsonian 3D models

38

39 of 42

Download 3D Data @ 3d.si.edu

39

Download files in glTF, glb, obj, usd (and some stl’s)

https://3d.si.edu/object/3d/neil-armstrong-spacesuit:d8c63ba6-4ebc-11ea-b77f-2e728ce88125

@Smithsonian3D

40 of 42

Download 3D Data @ API

40

Files available via API: glTF, glb, obj, and USD

https://3d-api.si.edu/api-docs/

@Smithsonian3D

41 of 42

Open Source Tools

Please star/follow the repository to stay informed.�Contributors welcome!

41

github.com/smithsonian/dpo-voyager

github.com/smithsonian/dpo-cook

github.com/smithsonian/dpo-packrat

@Smithsonian3D

42 of 42

� Thank you!

Vince Rossi

Smithsonian 3D Program Supervisor �rossiv@si.edu

Link to slides

Sign up for the Smithsonian 3D Storytelling workshop on Friday!

Contributors:

Jamie Cope
Jon Blundell

@Smithsonian3D