1 of 16

Distributed Data Access �in the National Security Complex

Peer-Timo Bremer

Center for Applied Scientific Computing

Center for Extreme Data Management Analysis and Visualization

​

​

LLNL-PRES-847297

This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

1

LLNL-PRES-xxxxxx

2 of 16

LLNL is one of the three NNSA laboratories focusing on national security and nuclear safety

  • Officially LLNL is mainly known for
    • Founded as part of the Manhattan Project
    • Premier research institution for high energy density physics
    • Designing, building, and certifying the nuclear stockpile
    • Hosting some of the largest supercomputers
  • Unofficially LLNL is mainly known for
    • Having the most powerful laser in the world
    • Maintaining the warp-drive of the Enterprise in Start Trek Into Darkness (2013)
    • Contributing to the original TRON (1982)

2

LLNL-PRES-xxxxxx

3 of 16

New Use Cases are Driving the Need for More Distributed and Smarter Data Access across the Livermore and the Complex

  • Larger, distributed teams need tighter coupling and faster turn around times
    • Science: Remote experiments, distributed sample preparation, etc.
    • Production: Design-Develop-Manufacture-Deploy loop is spread across agencies
  • Self-driving experiments
    • Tight coupling between experiments and simulations enabled by ML driven by data
    • Non-standard compute patterns and hardware
  • New user profiles for massive data requiring new tools
    • High-resolution metrology in additive manufacturing, energetic materials, NIF, etc.
    • Live cell imaging

​

NSDF Technology adapted to LLNL problems is raising significant interest

3

LLNL-PRES-xxxxxx

4 of 16

Data Access Pattern, Analysis Needs, and User Profiles are Rapidly Changing to Becoming More NSDF-Like

  • Traditionally much of the large-scale data management, analysis, and visualization has been tied to high-performance computing
    • Massively parallel simulations producing tera-, peta-, exa-byte sized data
    • Matching in-situ or post-hoc parallel analysis and visualization, i.e. VisIt, Ascent, Paraview, etc.
    • Enabled by HPC experts and domain scientists turned HPC experts
  • Many recent use cases have markedly different characteristics
    • Proliferation of high-resolution diagnostics, sensing, imaging, etc., hardware
    • Continuous production of many mid-size datasets vs. small number of massive data collections
    • Distributed facilities and diverse teams
    • Limited or no HPC skills or connections to HPC resources

NSDF Technology with an LLNL twist is raising significant interest

4

LLNL-PRES-xxxxxx

5 of 16

Use Case 1: High-Resolution Metrology of Additive Manufacturing Parts, NIF Targets, Energetic Materials, …

  • High-resolution CT scans provide insights into internal structures
    • Additive manufacturing: Printing artifacts and part certification
    • NIF targets: Voids and high Z inclusions that disturb the implosion
    • High energy materials: Material aging and performance characteristics
    • …
  • These applications pose a number of new challenges
    • Massive collections of large (2Kx2Kx6K and above) scans exceeding local resources
    • Diverse group of stakeholders: CT technicians, scientists, decision makers, etc.
    • Distributed teams and data collection
    • Data access restrictions
    • Wide variety of analysis needs often based on scripting, matlab, etc.

5

LLNL-PRES-xxxxxx

6 of 16

OpenShift-Based Deployment of OpenViSUS Data Server Provides Simple, Flexible, and Scalable Central Repository

  • Livermore Computing maintains a OpenShift/ Kubernetes controlled cloud environment
    • Simple OpenShift deployment of the OpenViSUS docker container
    • Easy setup for per-project deployments
    • Access to very large parallel file systems

​

Livermore Computing

Spark

OpenViSUS

6

LLNL-PRES-xxxxxx

7 of 16

Remaining Challenges are Mitigated by Setup and Deployment Scripts Providing Immediate Access to All Authorized Users

  • Data upload requires privileged access and data transfers
    • Data transfers are inside Livermore’s site network which makes them manageable
    • Server-side datasets are integrate semi-automatically but require an (instantaneous) server restart
  • Data access consists of three components
    • Where is the data: server URL
    • What datasets are on the server: visus.config
    • How can we access them: authentication
  • All access requirements are integrated into
    • Per-project visus.config configuration files
    • Providing immediate access through programmatic interfaces

​

7

LLNL-PRES-xxxxxx

8 of 16

Use Case 2: Immersive Metrology �Virtual Reality to Improve Advanced Manufacturing

  • Validation of both design and manufacturing of AM components is challenging
    • Complex internal geometries
    • Custom designed density gradients, dopants, gratings, etc.
    • Boutique design and printing technology for production of low volume parts
  • Interactive visualization could provide solutions yet is not widely adopted
    • Hardware and software requirements for traditional solutions are high
    • Understanding complex 3D geometries on 2D screens is difficult
    • Interacting with objects is difficult, i.e., measuring distances, aligning parts, etc.
  • Virtual reality provides intuitive solutions augmented by super-powers
    • Visualization, selection, manipulation, etc., match the real world
    • Augmented abilities, i.e., X-ray vision, arbitrary magnification, non-destructive cutting, etc.

​

​

​

Challenge: VR requires high framerates but manufacturing produces massive data

8

LLNL-PRES-xxxxxx

9 of 16

Fast and Progressive Data Access Enables the Design of Virtual Metrology Tools Currently being Explored at LLNL

Print and scan

Manually align

Automatically align

Select a defect

Change design

CAD modeling (nTopology)

CAD & CT scan visualization

Initial CAD to CT registration

Defect navigation

Inspection and measurements

Precisely registered CAD and CT

Automatically�find defects

9

LLNL-PRES-xxxxxx

10 of 16

Fast and Progressive Data Access Enables the Design of Virtual Metrology Tools Currently being Explored at LLNL

10

LLNL-PRES-xxxxxx

11 of 16

Fast and Progressive Data Access Enables the Design of Virtual Metrology Tools Currently being Explored at LLNL

11

LLNL-PRES-xxxxxx

12 of 16

Some Applications Benefit from On-the-Fly Data Access, Are Less well Connected, but have Fewer Security Concerns

  • The new Lattice Light Sheet Microscope enables live cell imaging for hours or days, but requires sample prep, bio-containment, etc. in a non-computing friendly location

12

LLNL-PRES-xxxxxx

13 of 16

Some Applications Benefit from On-the-Fly Data Access, Are Less well Connected, but have Fewer Security Concerns

  • The new Lattice Light Sheet Microscope enables live cell imaging for hours or days, but requires sample prep, bio-containment, etc. in a non-computing friendly location
  • Network Attached Storage (NAS) based deployment provides immediate access to anyone inside the VPN

ViSUS

13

LLNL-PRES-xxxxxx

14 of 16

Some Applications Benefit from On-the-Fly Data Access, Are Less well Connected, but have Fewer Security Concerns

  • The new Lattice Light Sheet Microscope enables live cell imaging for hours or days, but requires sample prep, bio-containment, etc. in a non-computing friendly location
  • Network Attached Storage (NAS) based deployment provides immediate access to anyone inside the VPN
  • Real time analysis will enable automatic feedback and steering

ViSUS

14

LLNL-PRES-xxxxxx

15 of 16

OpenViSUS is Providing Novel Capabilities That will Significantly Accelerate Common Workflows and Enable New Processes

  • OpenViSUS fills an important gap in Livermore’s current data science infrastructure
    • Light-weighed and easy to deploy
    • Fast and flexible data exploration of massive images both locally and remotely
    • Low barrier of entry for both hardware and software
  • Prototype deployments of OpenViSUS are active for multiple different projects
  • Significant interest from additional partners

Questions?

15

LLNL-PRES-xxxxxx

16 of 16

Disclaimer

This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes.

16

LLNL-PRES-xxxxxx