1 of 33

Exploring Inner Workings of Neural Networks Approximating 1D Functions with NNVisualiser

Hands On Workshop

Caxton Emerald S

Research Scholar, PU

2 of 33

Agenda

Introduction
Setting up environment
Training simple neural networks with Keras
Visualise a Neuron with NNVisualiser
Extending Visualisation to Layer and Network
Practice and QA
Conclusion

3 of 33

Introduction

4 of 33

Objectives

Introduce NNVisualiser.
Demonstrate training simple neural networks.
Visualise neuron transformations.
Extend visualisations to layers and entire networks.
Provide hands-on experience with the NNVisualiser package.

5 of 33

Motivation

Neural Networks

Universal Function Approximators

Do we understand what we build?

NNs as black boxes
Reliability

6 of 33

NNVisualiser - A python package

Key Features

Built upon versatile Matplotlib
Open source
Supports simple neural networks
Support for multiple plots for detailed visualisation
Supports custom visualisation packages of your choice

7 of 33

NNVisualiser - An Overview

A Neural Network Visualisation package in Python
Supports Neural Network models from Keras library
60 Public methods offering versatile functionalities

18 getters and 6 setters for customisations
36 plot methods for detailed study and visualisation

Supports saving generated plots in PNG or PDF formats

8 of 33

2. Setting Up the Environment

9 of 33

Requirements

Python Packages:

NNVisualiser

Neural Network Visualisation package in python

Tensorflow Keras

For handling Neural Network Models

Numpy

Numerical Python for input data provisioning feature
Optional

A few additional packages for the purpose of demonstration

10 of 33

Installing Requirements

Google Colaboratory comes pre-installed with Tensorflow, Keras and Numpy
Install NNVisualiser

!pip install NNVisualiser

11 of 33

3. Training Simple Neural Networks

12 of 33

Artificial Neural Networks

Models inspired by the biological neural networks of human brains
Network of interconnected nodes, or "neurons."
Designed to recognize patterns and solve complex tasks
Layered - MLP
Directional - Feed Forward
Deep Neural Networks - Number of Layers > 2

13 of 33

Learning Unknown from Known

Unknown functional mapping
For simplicity, let’s pick simple mathematical functions

Quadratic Function

f(x) = x*x {-1 ≤ x ≤ 1 }

Simple Neural Networks - Regression ANNs with Single Input and Single Output

14 of 33

Training a SNN with Keras

CODE

15 of 33

4. Visualising Neuron Transformations

16 of 33

Neuron Transformations

Input
Scaling
Accumulation of Scaled Terms
Translation
Activation

Introduction of Non Linearity

17 of 33

Loading a Keras model

CODE

18 of 33

2. Plotting Input Plots

CODE

19 of 33

3. Plotting Scale Plots

CODE

20 of 33

4. Plotting Cumulative Scale Plots

CODE

21 of 33

5. Individual Vs Consolidated Plots

CODE

22 of 33

6. Plotting Translation Plots

CODE

23 of 33

7. Plotting Activation Plots

CODE

24 of 33

8. Customising the Plots

CODE

25 of 33

5. Extending Visualisation to

Layers and Network

26 of 33

1. Flow Plots for Layer

CODE

27 of 33

2. Flow Plots for Network

CODE

28 of 33

5. Practice Session and QA

29 of 33

Try this

Train a SNN of 1_10ReLU_1 architecture over a dataset generated from a cubic function defined over the range of -1 and 1 in R, with a sample size of 100K, Batch size of 64 and 10 epochs. Leave the rest to default. Store and load the model into a new variable and plot the flow for fifth neuron in hidden layer 1, and flow for layer 1 and the entire network

30 of 33

Questions

31 of 33

6. Conclusion

32 of 33

Let’s remember

NNVisualiser package
Visualising

Neurons - Input, Scaling, Translation, Activation
Layers
Networks

Customisations

33 of 33

Feedbacks are most welcome!

Thank You