1 of 16

Faster data loading

More tricks

Erik Ylipää

Linköping University

AIDA Data Hub AI Support Lead

National Bioinformatics Infrastructure Sweden (NIBS)

SciLifeLab

2 of 16

AIDA Data Hub Support�Training a model

Support for how to split data (independence, resampling etc.)

Support for how to load data (efficiency, compatibility)

Support for designing and debugging machine learning model

Support for evaluating model

Support for secure data storage and compute

Task specific dataset

Training data

Test data

Machine Learning model

Trained Machine Learning model

Experiment results

3 of 16

Outline

Overview of NVIDIA DALI
Tutorial on converting a pytorch dataloader pipeline to DALI
Hands-on converting a pytorch dataloader pipeline to DALI

4 of 16

CPU utilization

GPU utilization

btop visualization of CPU vs GPU utilization. Training is bottlenecked by data augmentation on CPU.

6 of 16

Defining a pipeline

7 of 16

Defining a pipeline

While still there, doing standard any randomized augmentation only works with the decorator

8 of 16

Basic example

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/getting_started.html

9 of 16

What is a decorator

Decorators are functions which takes other functions as inputs and returns a function as output

It’s an “simple” way from a user point of view of changing the behaviour of a function

Used in many frameworks to “magically” inject themselves in your python code

10 of 16

NVIDIA Dali does more than most

The pipeline_def decorator actually transforms the source code of the function (or the Abstract Syntax Tree to be more precise) to generate a new function using Dali primitives

This means that errors in your pipeline definitions can get weird error messages

11 of 16

DALI AutoGraph

In many ways similar to the computational graph built by TensorFlow or PyTorch
When you use DALI functions, these will become nodes in your pipeline graph
Certain nodes might be pruned from this graph, which we will need to be mindful of.

12 of 16

Readers

The “source” of your data.

Analogous to pytorch Dataset in a DataLoader pipeline

13 of 16

External source

Customizable function to generate data as input to the pipeline
Like creating your own Pytorch Dataset

Either an iterable (implementing the __iter__ method) similar to or
a callable (implement __call__) taking a sample information object

14 of 16

Pytorch integration

By default DALI uses its own ndarray format

There are wrappers which convert the arrays to pytorch Tensors

Here will will use the DALIGenericIterator