1 of 12

FUNDAMENTALS OF NEURAL NETWORKS

Presented by: OLUSANYA JOY

Date: March 28th

​

2 of 12

OVERVIEW & OBJECTIVES

Objective: Learn the building blocks of neural networks.

​

  • Topics:
    • What are neurons? Understanding the basic unit of neural networks.
    • Introduction to activation functions: ReLU, sigmoid, and tanh.

​

3 of 12

WHAT ARE NEURONS?

Neurons in deep learning were inspired by neurons in the human brain.

They are also called called nodes through which data and computations flow

4 of 12

HOW DO NEURONS FUNCTION IN DEEP LEARNING

  • Neurons receive one or more input signals from either the raw dataset or neurons in a previous layer of the neural network.
  • They perform calculations on these inputs.(Weighted Sum and Activation)
  • They transmit the output signals to deeper neurons in the network through a synapse.(Forward Propagation or Feedforward).

​

​

​

  • Video Link: https://www.youtube.com/watch?v=VhRtaziEWd4

5 of 12

Neuron Functionality

6 of 12

7 of 12

ACTIVATION FUNCTIONS�

Activation functions help neurons decide whether to activate or not.

Activation functions are mathematical operations applied to the output of each neuron in a neural network. They introduce non-linearity, allowing neural networks to learn complex patterns beyond simple linear transformations.

An activation function is a mathematical operation applied to a neuron's output. It adds non-linearity to the model, enabling the network to recognize and learn complex patterns in data. Without activation functions, even a deep neural network would function like a simple linear regression model, regardless of the number of layers.

The activation function determines whether a neuron should be activated by computing the weighted sum of inputs and incorporating a bias term. By introducing non-linearity, it allows the model to make decisions and improve prediction accuracy.

​

8 of 12

Types of Activation Functions

9 of 12

10 of 12

Linear Activation Function

  • Formula: f(x)=xf(x) = xf(x)=x
  • Output is proportional to the input (no non-linearity).
  • Limitation: Cannot handle complex patterns; multiple layers behave like a single-layer model.

11 of 12

NON-LINEAR ACTIVATION FUNCTIONS

  • Threshold function
  • Sigmoid function
  • ReLU Function
  • Hyperbolic Tangent function(Tanh)

​

​

  • Video Link:https://www.youtube.com/watch?v=icZItWxw7AI

https://www.youtube.com/watch?v=Mubj_fqiAv8&list=PLeo1K3hjS3uu7CxAacxVndI4bE_o3BDtO

12 of 12

DEEP LEARNING FRAMEWORKS

TensorFlow

Developed by Google.� Supports both high-level (Keras) and low-level operations.

​

​

PyTorch

Developed by Facebook AI.

Dynamic computation graph, making debugging easier.�Used widely in research and production (especially for NLP and CV).

​

​