1 of 40

Hangar;PyTorch;RedisAI

Toolkits for the complete deep learning workflow

2 of 40

me?

  • hhsecond in internet
    • twitter.com/hhsecond
    • linkedin.com/in/hhsecond
    • github.com/hhsecond
    • www.hhsecond.com
  • Senior software architect @tensorwerk
  • I created turingnetwork.ai

3 of 40

Deep learning workflow

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

Data collection, unit testing etc.

4 of 40

Deep learning workflow

What do we use

  1. Jira / Notion
  2. Dropbox Paper / Google Doc
  3. Sublime Text / Atom / VSCode

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

5 of 40

Deep learning workflow

What do we use

  • Matplotlib / Seaborn
  • Scikit-learn / SciPy / etc
  • PyTorch / TensorFlow etc

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

6 of 40

Deep learning workflow

What do we use

  • PyTorch / Tensorflow / etc

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

7 of 40

Deep learning workflow

What do we use

  • PyTorch / Tensorflow / etc

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

8 of 40

Deep learning workflow

What do we use

  • TFServing / Clipper / MMS
  • TensorRT

​

Problems

  1. TFServing is complicated than it needs to be
  2. Other’s doesn’t really work all the time
  3. What do we do for PyTorch models

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

9 of 40

Deep learning workflow

What do we use

  • Arbitrary compression for data
  • Filesystem/s3 folders for versioning
  • Cloud storage for backup
  • DVC

Problems

  1. Version controlling on Binary
  2. Diffing doesn’t work well
  3. No partial cloning

Design

&

Experimentation

Ideation

&

Planning

Model

Implementation

Training

&

Validation

Shipping

To

Production

10 of 40

Hangar

11 of 40

Hangar

Question: Why hasn’t the open-source software ethos translated to open-source datasets?

​

​

​

Hangar is git for data; Version control tensors, not the blobs

12 of 40

13 of 40

What did we have

14 of 40

What went wrong?

  • How do we connect some piece of “Data” with a meaning in the real world?
  • How do we diff and merge large collections of data samples?
  • How can we resolve conflicts?
  • How do we make data access (reading and writing) convenient for both user-driven exploratory analyses and high performance production systems operating without supervision?
  • How can we enable people to work on huge datasets in a local (laptop grade) development environment?

15 of 40

Fundamentals

  • Data
    • Numerical arrays
    • Data samples - smallest unit possible
  • Datasets: Grouping of similar data pieces
    • Shape, type & name
  • Repository
    • Analogous to git repository
  • Operations
    • Checkout
    • Branch
    • Diff
    • etc..

16 of 40

Backend

  • LMDB for bookkeeping data
  • HDF5 & TileDB for dataset
  • More on the way..

https://github.com/tensorwerk/hangar-py

​

17 of 40

Hangar: Advantages

  • Saving data as tensors
  • A flexible CLI & programmatically accessible python APIs
  • Eliminate pipelines for dataset on disk
  • Time travel through the historical evolution of a dataset
  • Zero-cost Branching to enable exploratory analysis and collaboration
  • Cheap Merging to build datasets over time (with multiple collaborators)
  • Partial cloning ability while still maintaining complete historical record
  • Ability to push and pull changes directly to collaborators or a central server (ie a truly distributed version control system)

18 of 40

Some Examples & Demo

>>> repo = Repository(path='/path/to/hangar-dev-repo/')

>>> repo.init(user_name='Sherin Thomas', user_email='foo@email.com')

>>> co = repo.checkout(write=True)

>>> co.datasets.init_dataset(name='mnist_training_images', prototype=trimgs[0])

>>> train_dset = co.datasets['mnist_training_images']

>>> train_dset['0'] = trimgs[0]

>>> out = train_dset['0']

>>> plt.imshow(out.reshape(28, 28))

​

​

19 of 40

PyTorch

20 of 40

PyTorch

  • Using Hangar Dataset to load data
  • Building a graph
  • Training
  • Exporting; Using both PyTorch & RedisAI client
  • Benefits of PyTorch
  • Problems faced by developers pushing PyTorch to production

21 of 40

RedisAI

22 of 40

RedisAI

  • Backend; Tensorflow, PyTorch & ONNXRuntime
  • Device; CPU & GPU
  • Redis Ecosystem

23 of 40

RedisAI: Backend a.k.a Framework

  • Tensorflow
  • PyTorch
  • ONNXRuntime
    • MxNet
    • CNTK
    • Chainer
    • DyNet
    • Flux
    • Scikit-learn

24 of 40

Production Strategies

  1. Python code behind e.g. Flask
  2. Execution service from cloud provider
  3. Runtimes like TFServing, Clipper
  4. Bespoke solutions (C++, ..)

25 of 40

RedisAI: What is it?

A Redis module providing

  1. Tensors as a data type
  2. Runtime on top of LibTorch, LibTensorflow & ONNXRuntime

​

It turns Redis into a full-fledged deep learning runtime while still being Redis

26 of 40

Install Redis

wget http://download.redis.io/redis-stable.tar.gz

tar xvzf redis-stable.tar.gz

cd redis-stable

make

sudo make install

​

​

Install RedisAI

git clone https://github.com/RedisAI/RedisAI.git

cd RedisAI

bash get_deps.sh cpu

mkdir build

cd build

cmake -DDEPS_PATH=../deps/install -DCMAKE_BUILD_TYPE=Release ..

make

​

redis-server --loadmodule build/redisai.so

27 of 40

28 of 40

RedisAI: Tensor

29 of 40

Model & Script

  • Binary export from Tensorflow / PyTorch / ONNX
  • model.pb / model.pt / model.onnx
  • Running on corresponding runtime
  • Can execute on CPU / GPU
  • Pure code with python syntax (SCRIPT is a subset of python language)
  • script.txt
  • Running on libtorch runtime
  • Can execute on CPU / GPU
  • Vast library of tensor ops
  • Can do pre-proc/post-proc regardless of model runtime

30 of 40

Redis Ecosystem: Persistence

  • RDB and AOF
    • Tensors are serialized (meta + blob)
    • Models are serialized as protobuf
    • Scripts are serialized as string

31 of 40

Redis Ecosystem: Client Libraries

  • Client libraries
    • Redis clients in all languages
    • Specific RedisAI client for Python and Java (NodeJS & Go is on the way - community could help us?)

32 of 40

Redis Ecosystem: Replication

  • Sentinel
  • Cluster

33 of 40

Redis Ecosystem: All other modules

  • Stream
  • TimeSeries
  • Gear
  • and a lot more ...

34 of 40

RedisAI: Advantages

  1. Keep the data local
  2. Keep stack short & optimize use of resources
  3. Run everywhere Redis runs
  4. Run multi-backend
  5. Stay language-independent
  6. Keep models hot
  7. HA with sentinel, clustering
  8. Autobatching (upcoming)
  9. Running Async (upcoming)

https://github.com/RedisAI/redisai-examples

35 of 40

Example 1:

Golang

36 of 40

Example 2: NodeJS

37 of 40

Example 3: Python

38 of 40

Example 4: Python client for RedisAI

​

Checkout the Demo

39 of 40

Demo

  • Fundamentals of Hangar
  • Setting up Hangar Repository
  • Partial cloning from Hangar repository
  • Use hangar data loaders and load data into PyTorch
  • Train PyTorch
  • Fundamentals of RedisAI, Setting tensor, model, script etc.
  • Export PyTorch model to optimized binary (RedisAI client)
  • Load model to RedisAI cluster (RedisAI client)
  • High availability even when master goes down

40 of 40

Thank you

We need

feedback

users

contributors

github stars