1 of 11

Neural Networks IV: Implementation & Review!

Vatsal Sivaratri

TJ Machine Learning Club

Slide 1

2 of 11

  • Forward Propagation is...
  • The process where input data is passed through the neural network layers to generate an output.
  • Involves the computation of the weighted sum of inputs, followed by the application of an activation function.

​

So far you’ve learned….

TJ Machine Learning Club

Slide 2

3 of 11

  • Backward Propagation is...
  • A method used to calculate the gradient of the loss function with respect to each weight in the network.
  • Essential for optimizing the weights by iteratively adjusting them in the direction that minimizes the loss.

So far you’ve learned….

TJ Machine Learning Club

Slide 3

4 of 11

  • Activation Functions are...
  • Functions applied to the output of a neural network layer, introducing non-linearity to the model.
  • Examples include ReLU (Rectified Linear Unit), Sigmoid, and Tanh, each having distinct effects on the model's learning capability.

​

So far you’ve learned….

TJ Machine Learning Club

Slide 4

5 of 11

  • Hyperparameters include...
    • Settings like learning rate, batch size, and number of epochs, which define the network's learning process and structure.
    • Critical for model performance, requiring tuning to optimize the model's ability to learn and generalize.

So far you’ve learned….

TJ Machine Learning Club

Slide 5

6 of 11

What is Tensorflow?

  • An open-source machine learning library developed by Google.
  • Provides both high and low-level APIs for building and training machine learning models.
  • Known for its powerful capabilities in deep learning, but also supports general numerical computation.
  • Allows for easy deployment of computation across various platforms (CPUs, GPUs, TPUs).

​

TJ Machine Learning Club

Slide 6

7 of 11

What is Keras?

  • An open-source neural network library written in Python.
  • Designed to enable fast experimentation with deep neural networks.
  • Runs on top of TensorFlow, making it user-friendly with simpler, higher-level API.
  • Focuses on being modular, extensible, and easy to work with, ideal for beginners.

​

TJ Machine Learning Club

Slide 7

8 of 11

Pytorch vs. Tensorflow??

Pytorch:

  • Dynamic Graphs: Great for dynamic inputs and easy debugging.
  • Research-Friendly: Intuitive for experiments and rapid prototyping.

Tensorflow:

  • Static Graphs: Efficient for fixed data, ideal for production models.
  • Broad Ecosystem: Extensive tools and community support for deployment.

​

TJ Machine Learning Club

Slide 8

9 of 11

Neural Networks with Libraries

  • Less Math! - Libraries like TensorFlow and Keras abstract away the intricate math involved in neural networks, making the implementation more accessible.
  • Less Code! - Libraries simplify coding, but now the complexity shifts towards data handling - preprocessing, augmentation, and ensuring data quality.
  • Significant effort is still required in tuning for optimal performance.

TJ Machine Learning Club

Slide 9

10 of 11

Let’s Code a Neural Network!

Grab a Laptop, some paper, whatever you need to participate.

TJ Machine Learning Club

Slide 10

11 of 11

Review Time!

Form 3 Big Groups and Have a Piece of Paper Ready!

TJ Machine Learning Club

Slide 11