Developing with vtk.js

Acknowledgments

• Presented By:
• Forrest Li (Kitware)�
• Contributors:
• Forrest Li (Kitware)
• Trinity Urban (MGH and OHIF)
• Steve Pieper (Isomics)
• Erik Ziegler (OHIF)
• Gordon J. Harris (MGH and OHIF)
• James A. Petts (OHIF)
• Danny Brown (OHIF)
• Stephen Aylward (Kitware)

Funding Sources

• VTK.js / Kitware
• NIH NINDS R44NS081792
• NIH NINDS R42NS086295
• NIH NIBIB and NIGMS R01EB021396
• NIH NIBIB R01EB014955
• Isomics
• NIH P41 EB015902
• OHIF
• NIH U24 CA199460

Course Outline

1. Introduction to vtk.js and OHIF
2. Starting a vtk.js project
3. vtk.js: Architecture, itk.js, widgets, and more
4. OHIF + vtk.js integration
5. Advanced vtk.js: WebGL and GLSL

Requirements (basic)

• A modern web browser that supports WebGL
• Don’t know? Test it out here: https://get.webgl.org/
• Good graphics card is preferred
• A text editor

WebGL2 ✓

(recommended)

WebGL 1 ✗

(not recommended)

Requirements (development)

• node.js (https://nodejs.org/en/)
• npm (in node.js install) or yarn (https://yarnpkg.com/en/)
• A terminal emulator
• Familiarity with working in a shell/command line
• Familiarity with browser devtools

Source Files for this Demo

• Find them here: https://github.com/KitwareMedical/vtk.js-examples

The super simple example

Only browser + text editor needed

🌟 Only a single

HTML file needed!

​

File: standalone.html

Import VTK.js

Set up the VTK.js RenderWindow and Renderer

Attach WebGL RenderWindow implementation

Attach interactive trackball controls

Construct our rendering pipeline for a cone

Center camera on cone and trigger the render pipeline

Super simple example

• Great for one-off demos
• Only need a browser!
• No need to pull in node.js/npm/entire development ecosystem
• Ease of prototyping

​

• 💡 Exercises
• Use a different surface source, like vtkCylinderSource, vtkSphereSource, or vtkPlaneSource
• Modify some properties of the sources! e.g. cone.setHeight(3.0)

Project Creation Example

node + npm + everything else

Project Creation: Application Structure

• Example application structure
• We will make these files as we go�through the example
• src/: source code
• dist/: final bundled code

​

• See the examples for final project

Project Creation

​

​

� $ npm init

​

​

Installing tooling + dependencies

​

​

​

$ npm install --save vtk.js�$ npm install --save-dev kw-web-suite

Brings in:�

• webpack
• babel
• eslint
• prettier
• webpack loaders (css, shaders, etc.)
• webpack-dev-server
• ...and more

Creating initial files

• Main javascript entry point: src/index.js
• Browser entry point: dist/index.html
• Webpack build configuration: webpack.config.js

Creating files: dist/index.html

Creating files: src/index.js

• Start with a simple import to verify the build worked

Creating files: webpack.config.js

Creating files: webpack.config.js

vtk.js webpack rules required

Building our example

Two ways:

• npx webpack --progress
• Requires installing “npx”: npm install -g npx
• npm run build or npm run dev
• Requires the following changes to package.json:

Build results

You should now have a file: dist/app.js

Open index.html in your browser, open devtools (F12 on most browsers):

Success!

Cone Example: revisited

Let’s “rewrite” our old code

Rewriting our Cone Example

• vtkGenericRenderWindow does all of the setup for the RenderWindow, Renderer, and Interactor.

Rewriting our Cone Example

Rewriting our Cone Example

​

​

​

​

​

Example source file: vtk.js-examples/cone.js

Explanation: Rendering Pipeline

Explanation: Rendering Pipeline

• ConeSource supplies data
• Data goes through Mapper
• Mapper is attached to Actor
• Renderer holds many Actors
• RenderWindow holds many�Renderers

Examining ConeSource

• vtkConeSource is a VTK filter
• Algorithm that outputs vtkPolyData
• A collection of vertices, lines, and surfaces
• A basic vtk.js datatype
• ConeSource algorithm: 0 inputs, 1 output

VTK.js Filters: Example

• Simple example: vtkOutlineFilter
• vtkOutlineFilter transforms input PolyData into an outline box

VTK Filters: Example

VTK Filters: Pipelines

• InputData/OutputData
• Static: need to call getOutputData()�to get updated data
• InputConnection/OutputPort
• Reactive: getting the output data of�a chain of filters will fetch the latest�data given the existing parameters
• Data can go through multiple pipelines

VTK Filters: Example Result

​

​

​

​

​

Example source file: vtk.js-examples/cone-filter.js

Volumes and Images

3D volumes and 2D slices

Volume and Slice Rendering

• 3D volumes rendered in scene
• Including oriented images!
• Color Transfer Functions
• Opacity
• Slicing
• Loading remote datasets

Volume Rendering Example

Three steps:

1. Setting up the volume rendering pipeline
2. Loading a remote dataset
3. Rendering the volume

Volume Rendering Pipeline

• Set up the vtkVolume and vtkVolumeMapper
• Very similar to the cone setup
• What data goes into the mapper?

Loading a Remote Dataset

• vtk.js/Sources/IO/Core/HttpDataSetReader
• Loads a custom vtk.js-specific file format
• Data is automatically loaded into a polydata or image
• Example
• Browser Fetch API with I/O readers
• Downloads raw data from an endpoint
• Must manually load that data into usable vtk.js polydata or image
• XMLImageDataReader, XMLPolyDataReader, itk.js, etc.
• Alternative for older browsers: XMLHttpRequest API

Fetch Example

​

​

​

​

• 💡 Throughout these examples, feel free to try fetching and rendering your own data! Make sure the data is CORS accessible.
• https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Loading a Remote Volume

• HttpDataSetReader is an algorithm that outputs a dataset

Volume Rendering: 1st result

• Black/grey/white
• No opacity
• Not terribly useful - cannot see�Inside

​

​

Source file: volume.js

Volume Rendering: Opacity and Color

• Opacity function: determines which voxels are transparent
• Color transfer function: Determines which voxels map to what color

Volume Rendering: Opacity

• Create a vtkPiecewiseFunction
• Linear opacity function maps [0, 256] to [0, 1]
• Actor properties: Opacity, color, lighting, etc.

Volume Rendering: Color Transfer Function

• Create a vtkColorTransferFunction
• Apply a colormap, or set points manually
• Cool to Warm, Matlab colors, ERDC colors, etc.
• Set mapping range to be volume data range

Volume Rendering: 2nd result

• Pixels close to 0 are now transparent
• Pixels are mapped according to the�colormap “Warm to Cool”
• Can see internal volume structures

​

​

Source file: volume-transfer.js

Piecewise Gaussian Widget

• Interactive opacity function editing
• Multiple Gaussian functions
• Check it out: https://kitware.github.io/vtk-js/examples/PiecewiseGaussianWidget.html

Volume Rendering

• MPR
• Max, Min, Avg Intensity Projection
• Multiple component volumes
• Interpolation types

​

• 💡 Exercises!
• Change the scalar opacity function. More advanced: use vtkPiecewiseGaussianWidget
• Change up the color transfer function with a different preset: All presets
• Try rendering an oriented image dataset

Volume Slicing

• vtkVolume -> vtkImageSlice
• vtkVolumeMapper -> vtkImageMapper
• Parallel camera projection
• 2D image interactor
• window/level
• slicing

Volume Slicing

Volume Slicing

Use image slicing interactivity (slice scroll and mouse window/level)

Volume Slicing

Need to set slice at focal point

Volume Slicing

Sets default window/level

Volume Slicing

• Play around with the mouse for window/level�controls
• Scroll through slices with mouse wheel or�trackpad scrolling

​

Source file: image-slicing.js

Widgets and Interactions

• Add lines and controls to a scene
• Paint or contour an image
• Place landmarks
• Select regions
• ...and more!

Widget Example: Cropping Widget

• Interactive widget to crop a volume
• Demonstrates handle and dataset interactions in 2D and 3D
• Requires the vtkCropFilter
• This performs the actual cropping of the dataset

Widgets Overview

• Widget Manager
• Is associated with a renderer
• Manages widgets in that renderer
• Widgets
• Consists of a widget state and a widget factory
• Widgets constructed via factory are bound to renderers
• Representations derive from widget state
• MVC architecture

Adding Crop Widget

Adding Crop Widget

Attaches renderer to widget manager

Adding Crop Widget

A view widget is an instance of a widget in a renderer.

Adding Crop Widget

When the crop widget updates, we crop our image.

Adding Crop Widget

• vtkHttpDataSetReader 🠆 vtkCropFilter 🠆 cropped image

Initialize the crop filter with the volume

Crop Widget

• Crop your volume via the green handles
• 💡 Check out other vtk.js widgets on the�vtk.js examples page.
• Play with the widget API by turning off�certain handles

​

Source file: widgets.js

Conclusion

• Demonstrated examples for:
• Geometry rendering
• Filters
• Volume rendering
• Slice rendering
• Interactive widgets
• More: https://kitware.github.io/vtk-js/examples/

​

Next Time

​

Next course (#3):

Digging into vtk.js: Architecture and more