Challenges & Risks in�Evolving 3D Standards for the Metaverse

IEEE Metaverse Congress Session 5

​

Challenges & Risks in Evolving�3D Standards for the Metaverse

QR link to Slides

Vincent Marchetti

​

X3D Graphics Working Group - Co-Chair

Web3D Consortium

1994

2013

2005

2007

2016

1997

Evolution Collaboration Standardization Democratization 3D Printing HTML5 AR/VR

2014

2018

2019

Model + hyperlink

VR devices

XML – Programmable

Animation, Scripting

Interactivity

Interchange

Tool chain support

Binary Streaming

AR - 3D - VR

In browser Declarative 3D

Annual Web3D Conference

1995-2022 is our 27^th Year!

Web3D Evolution

1997-2022

​

V 3.0

2022

V4.0

V 2.0

V 1.0

V 2.0

V 1.0

V 3.3

JavaScript Implementations

HTML5

glTF

3D Print

3D Scanning

WebVR

AR/VR

Next Generation VRML

Open-source

Experiment/Example - X3D & glTF Objects Combined

Authoring Interactive Dynamic X3D and glTF Together (Web3D Consortium)

​

• Examples using ‘X3D to add visual annotation to glTF assets, powerful navigation and exploration aids, and combine assets from multiple sources in a single scene’

- Vincent Marchetti

​

https://www.web3d.org/event/webinar-authoring-interactive-dynamic-x3d-and-gltf-together

See: A Gathering of Goblets - https://www.kshell.com/pages/webinar_20220531/drinking/goblets_inline_html.html

Catharine Dass

​

UK Parliament - Architect

BIM4Heritage

Image credits: Parliamentary Heritage Collections / UK Parliament

Brief History of Palace of Westminster (1530s)

Westminster Hall

Westminster Abbey

• Palace of Westminster was a royal palace in the middle ages.
• Westminster Hall is the oldest building.
• The hall was built in 1097 – 1099 under Wiliam II (Rufus), the son of William the Conqueror.
• Largest hall in England and probably in Europe at that time. Measuring 73 by 20 metres (240 by 67 feet).
• The English Parliament has met here since 1295.

Brief History of Palace of Westminster

Image Credits: Google Earth

Westminster Hall

Victoria Tower (Archives)

Westminster Abbey

Elizebeth Tower (Big Ben)

• October 1834, a great fire.
• Westminster Hall, St. Stephen’s hall and cloister court survived.
• Palace of Westminster today, reconstructed by architect Charles Barry, with Augustus Pugin in Gothic Revival style.
• WH has been through 900 years of fascinating years British history.
• Various key events: State openings, trials, banquets, welcoming international addresses and lying in state.

​

​

Image credits: Purcell

Elizabeth Tower (Big Ben): Multi-Disciplinary linked models, Clash Detection

Image credits: Donald Insall Associates

Define information requirement

Stonework Conservation: Conservation Data for Heritage Digital Construction

2. Non graphical data is pushed into the Revit model

1. Typical information delivery for conservation project

3. Metadata reformatted into COBie schema

Image credits: Donald Insall Associates

Stonework Conservation: Legacy 2D Information Translated into 3D BIM Model

1. 3D Revit model as existing

2. 2D AutoCAD annotated drawing identifying masonry repairs (colour coded) linked into 3D model

3. Annotations converted / embedded into 3D Revit model

4. Annotations replicated (orange) and modelled to represent masonry repair object

5. Two methodologies = embedded annotations and modelling. Information is exported into Cobie scheme

Federico Ponchio

​

Visual Computing Lab - Researcher

Institute of Information Science & Technologies

Italian National Research Council

Karina Rodriguez Echavarria

​

University of Brighton - Reader

Computing and Cultural Heritage - Researcher

Ronald Haynes

​

IIIF 3D Technical Specification Group - Co-Chair

University Senior Computer Officer

University of Cambridge

IIIF – International Image Interoperability Framework - iiif.io

IIIF Implementations and Consortium Map - https://iiif.io/community/map

• Open standards for delivering high-quality, attributed digital objects online at scale
• Rich access to the world's image and audio/visual files
• An international community developing and implementing the IIIF open APIs
• https://iiif.io

IIIF 3D Groups

The 3D Community Group meets monthly to share:

• 3D projects, e.g.: The Met Unframed, Deutsches Museum, Mozilla Hubs, Open Access (Smithsonian, Morphosource, Sketchfab), Google <model-viewer>
• Use cases <https://github.com/IIIF/iiif-3d-stories/issues>

​

The Technical Specification Group (TSG) meets monthly to:

• Implement the charter, address user needs, expand collaborations
• Explore suitable options for 2D image, A/V, and 3D model integration
• Develop guidance for ‘digital dioramas’ (mini-metaverse)� - scenes with many files/formats (e.g. glTF, X3D, DICOM, OBJ, HBIM, RTI, 360)

IIIF 3D Groups

Some challenges facing the 3D Groups:�

• Considerations for sustainability
• e.g. with Digital Preservation Coalition, CS3DP (Community Standards for 3D Data Preservation)
• ‘Decolonisation’ - virtual and physical options for sharing and repatriation�
• Reference workflows in the Community
• clarify suitable equipment, processes, interoperable options
• identify people and capabilities needed for 3D production�
• Options for smaller/less well-funded institutions
• support for viable 3D workflows and production
• joint funding, means to ensure growth of interoperability�
• Metaverse modularity
• provide structures for a metaverse (digital dioramas, or multi-metaverse?)

3D TSG Community Support

• British Library
• Cyprus University of Technology� / Digital Heritage Research Lab
• Deutsches Museum
• Duke University
• Harvard University Library
• ISTI - CNR (Institute of Information Science & Technologies - National Research Council of Italy)
• MorphoSource
• Mnemoscene

• sketchfab.com
• The Smithsonian Institution
• University of Basel
• University of Brighton
• University of California� San Diego
• University of Cambridge
• University of Edinburgh
• University of Oxford�
• … your name here?

• Collaborations: Europeana, UNESCO World Heritage, IEEE SA

IIIF 3D groups’ work

Technical and Community Documentation

Design Sketching & Funding

Demos & Collaborations

Proposals

Interoperable 3D!

2022-2023

2023 and onwards

Develop Technical Standards Group...

IIIF 3D - Current Uses

Royal Pavilion & Museums Collection Explorer

Exhibit.so Jane Austen Digital Exhibit

MorphoSource (models + CT/MRI volumes)

Let’s call these IIIF-ish...

User Stories

• 6 core user stories (I want to display): https://github.com/IIIF/iiif-3d-stories/issues

​

1. a 3D model, specifying position, orientation, and scale
2. multiple 3D models in a shared space
3. 3D model alongside 2D images and AV content in a single viewing experience
4. 3D models with options to annotate with commentary
5. options for sharing camera position, orientation, and target
6. multiple 3D models that can be rotated and manipulated independently

It’s complicated...

Tweet from @FreyaHolmer - update by Ed Silverton (Mnemoscene)

Factors for interoperability

​

• Need 3D objects nested inside other 3D models?

​

• Enable model viewing across repositories, in scenes, VR/AR/XR?

​

• Universal Persistent IDs to track components & multiple model versions?

Interoperability / Sustainability Challenges

Post production?

• Optimise/Version?
• Add 2D, Audio, Video, Animation?
• Annotation?

​

• Which axis is up?
• What scale units?
• Where is the pivot?

​

Open Access?

• Copyright
• Licensing

Varied 3D Workflows

• Photogrammetry
• Laser, Structured Light
• CT, MRI
• 3D modelling�

Varied Output formats

• Mesh
• Pointclouds
• Volumetric
• + Metadata, Paradata

Factors for interoperability

​

• Need 3D objects nested inside other 3D models?

​

• Enable model viewing across repositories, in scenes, VR/AR/WebXR?

​

• Universal Persistent IDs to track components & multiple whole models?

Mozilla Hubs – with 360 video & spatial audio ‘bubbles’

See: Jeffrey Pomerantz - proximalvr.com & North Carolina Museum of Art - ncartmuseum.org & ncmaexplore.org/duQ3Qzs/iiif-peru-demo

​

Related References

• Initial IIIF 3D TSG Glossary�https://docs.google.com/document/d/1wCtQxbfr9xV6CenaN88OfZbCc5RxgLx-r_vzO0OHodc�
• Sketchfab Cultural Heritage User Survey�
• IIIF & Europeana collaboration - evolving integration of 3D content
• see: EuropeanaTech Insight magazine - Issue 14: 3D

​

• Basic principles and tips for 3D digitisation of cultural heritage | Shaping Europe’s future�https://digital-strategy.ec.europa.eu/en/library/basic-principles-and-tips-3d-digitisation-cultural-heritage

​

• Study on quality in 3D digitisation of tangible cultural heritage | Shaping Europe’s digital future�https://digital-strategy.ec.europa.eu/en/library/study-quality-3d-digitisation-tangible-cultural-heritage�
• MorphoSource 2.0 - morphosource.org
• MS2 Data Dictionary 2/15/2020 - https://docs.google.com/spreadsheets/d/1nC_p2dHxWZCWVoobTzdHC_8Dn1MU9ptk1qZSZvCzaLo

Selected References

• BIM4Heritage - https://bim4heritage.org
• Centre for Secure, Intelligent and Usable Systems� - https://www.brighton.ac.uk/csius
• IIIF 3D Technical Specification Group� - https://iiif.io/community/groups/3d/tsg
• Visual Computing Lab - http://vcg.isti.cnr.it
• Web3D Consortium - https://www.web3d.org

Questions & Comments