2 of 79

Intelligent Computing

A new Computing Paradigm that is reshaping the traditional computing.
Computing paradigms are fundamental approaches to how we design, build, and use computer systems to solve problems and achieve specific goals.
They represent different ways of organizing computation, data, and computer systems.

3 of 79

Computing Paradigm

Core Paradigms: (Computing before Cloud and Modern Computing)

Sequential/Imperative
Functional
Object-Oriented

Distributed Computing and Parallel Computing

Networked Computers solve a particular problem.
Multiple processors within a single machine.

4 of 79

Modern Computing Paradigm

Cloud Computing: Computing resources as a service over the internet.
Edge Computing: Processes data closer to the source (e.g., IoT devices, sensors) to reduce latency and bandwidth usage, often complementing cloud computing.
Quantum Computing: Uses quantum mechanics to perform computations.
Cognitive Computing: Aims to create systems that mimic human thought processes, including learning, problem-solving, and decision-making.

5 of 79

Intelligent Computing

Definition(Zhu et al, 2022): Intelligent computing is the area that encompasses the new computing theoretical methods, architecture systems, and technical capabilities in the era of digital civilization that supports the interconnection of all the world. Intelligent computing targets computational tasks with the minimum cost according to the specific actual needs, matching adequate computational power, invoking the finest algorithm, and obtaining optimal results.

6 of 79

Intelligent Computing

Intelligent computing is a field that encompasses emerging computational theories, system architectures, and technical capabilities tailored for the digital age.

Its goal is to provide efficient, secure, autonomous, reliable, and transparent computing services to support large-scale and complex computational tasks.
In Intelligent Computing, human society, the physical world, and information space are integrated.

7 of 79

Intelligent Computing

Figure 1: An overview of intelligent computing based on the fusion of human social space, physical

space, and information space.

8 of 79

Evaluation Criteria

Sessional Assessment: 10%
Mid Semester Exam: 30%.
End Semester Exam: 50%.
Quiz: 10%

9 of 79

Evaluation Criteria(Sessional Assessment: 10%)

Each student is required to spend approximately 12 hours over the semester—typically around one hour per week—at the Coforge Data & AI Lab: 10%. [10.00-16.00]

Each student needs to register his(her) biometric @ Coforge Data & AI Lab.

10 of 79

Mode of Learning

Online Mode:

Everyone will get access to the online learning platform Percipio (ai-datalab.percipio.com)

AI for Programmers:

Offline Classes.

11 of 79

Major Topics to be covered in this class

Artificial Neural Networks
Deep Neural Network (Mixed Mode)
Recommender System.
Anomaly Detection System
Spiking Neural Networks.

12 of 79

Online Learning (AI for Programmers)

AI & ML Foundations: Classification, Clustering, Evaluation

Deep Learning: CNNs, RNNs, GANs, Attention Models

Generative AI: ChatGPT, Prompting, Transformer models

Prompt Engineering: Introduction, categories, task-based prompting

Responsible AI: Ethics, Risks, Mitigation strategies

13 of 79

Artificial Neural Networks

The term neural network derives its origin from human brain that consists of a massively parallel architecture of large number of neurons.
Biological neural network can perform various tasks: perceptual, recognition, etc., much faster than even today’s high-performance computing systems.
Biological neural network inspired researchers to think if there is any way to make the computer mimic the networking and the complex interconnections that exist between the nerve cells in the human brain.

14 of 79

Biological Neuron

15 of 79

History: Artificial Neural Networks

McCulloch and Pits (1943): Introduced the idea of Neural Networks as Computing Machines.

Hebb (1949): Postulated the first rule for self-organized learning

Rosenblatt (1958): Proposed the perceptron as the first model for learning with a teacher

16 of 79

McCulloch-Pitt’s Model

Proposed by Warren McCulloch and Walter Pitts in 1943.

This model imitates the functionality of a biological neuron, thus is also called Artificial Neuron.

An artificial neuron accepts binary inputs (i.e., 0 or 1) and produces a binary output based on a certain threshold value which can be adjusted. This can be mainly used for classification problems.

17 of 79

McCulloch-Pitt’s Model

Different parts of McCulloch-Pitts Neuron Model

Neuron: It is a computational unit which accepts incoming input signals. The input signals are computed and an output is fired. The neuron further consists of following two elements:

Summation function: Computes summation of the inputs.
Activation function: This is basically a threshold function which checks if the summation is greater than or equal to a preset value. If yes, the neuron should fire (i.e., produce output 1), and if no, the neuron should not fire (i.e., produce output 0).

18 of 79

McCulloch-Pitt’s Model

Different parts of McCulloch-Pitts Neuron Model

2. Weighted Inputs and Summation: Each input has an associated weight. Positive weights represent excitatory inputs, encouraging the neuron to fire, while negative weights signify inhibitory inputs, potentially preventing the neuron from firing.

3. Output: This is simply the output of the neuron which again can take only binary values of 0 or 1. The value of 0 indicates that the neuron does not fire, the value of 1 indicates the neuron does fire.

19 of 79

McCulloch-Pitt’s Model

20 of 79

McCulloch-Pitt’s Model

21 of 79

Realizing Simple Functions with McCulloch-Pitt’s Model

22 of 79

McCulloch-Pitt’s Model

23 of 79

Disadvantages of MCP Neuron

Binary Output Limitation
Fixed Weights
Lack of Learning
Inability to Handle Non-Linearities
Limited Complexity.
Scalability.

24 of 79

Donald Hebb's Hebbian Learning (1949)

"Neurons that fire together, wire together".

In 1949, Donald Hebb proposed one of the key ideas in biological learning, commonly known as Hebb’s Law.
Hebb’s Law states that if neuron i is near enough to excite neuron j and repeatedly participates in its activation, the synaptic connection between these two neurons is strengthened and neuron j becomes more sensitive to stimuli from neuron i.

25 of 79

Donald Hebb's Hebbian Learning (1949)

The Hebbian rule states that if two interconnected neurons are simultaneously and persistently active, the strength of the synaptic connection (weight) between them should increase.

⇒ If two neurons on either side of a connection are activated synchronously, then the weight of that connection is increased.

If two neurons on either side of a connection are activated asynchronously, then the weight of that connection is decreased.

26 of 79

Donald Hebb's Hebbian Learning (1949)

27 of 79

Donald Hebb's Hebbian Learning (1949)

32 of 79

Apply Hebbian Rule

x₁	x₂	y (Output)
-1	-1	-1
-1	1	-1
1	-1	-1
1	1	1

33 of 79

Initialize all weight w_i=0; including Bias=0; W₀ =[0 0 0];
Set activations for input units with the input vector Xi for i = 1 to n.
Set the corresponding output value to the output neuron, i.e. y_i.
Update weight and bias by applying Hebb rule for all i = 1 to n:

w_new=w_old+ l.x_i.y , l=learning rate (Assume 1.0 here)

b=b_old + y

34 of 79

First Iteration:

w(new) = w(old) + x⁽¹⁾y⁽¹⁾ = [ 0 0 0 ]^T + [ -1 -1 1 ]^T . [ -1 ] = [ 1 1 -1 ]^T

2nd Iteration:

w(new) = [ 1 1 -1 ]^T + [ -1 1 1 ]^T . [ -1 ] = [ 2 0 -2 ]^T

3rd Iteration:

w(new) = [ 2 0 -2]^T + [ 1 -1 1 ]^T . [ -1 ] = [ 1 1 -3 ]^T

4th Iteration:

w(new) = [ 1 1 -3]^T + [ 1 1 1 ]^T . [ 1 ] = [ 2 2 -2 ]^T

35 of 79

Decision Boundary:

x₁w₁+x₂w₂+B=y

2x₁+2x₂-2=0

x₁+x₂=1

36 of 79

Table

x₁	x₂	Y
-1	-1	-1
-1	1	1
1	-1	1
1	1	-1

37 of 79

Single Layer Perceptron (1958)

39 of 79

Perceptron is the simplest form of neural network used for classification of linearly separable patterns.
It consists of a single neuron with adjustable weights and bias.
The algorithm used to adjust the free parameters first appeared in a learning procedure developed by Rosenblatt (1958) for his perceptron brain model
He proved that if objects are drawn from two linearly separable classes, then the perceptron algorithm converges and positions the decision surface in the form of a hyper-plane between the two classes.

40 of 79

The perceptron built around a single neuron is limited to performing classification with only two classes.
Expanding the output layer by including more neurons will help is performing multi-class classification, but the classes must be linearly separable for the perceptron to work properly.
As long as we look upon perceptron as an object classifier, we need to consider the case of a single output neuron only. Extension of the theory to the case of more than one neuron is trivial.
A single neuron forms the basis of an Adaptive Filter.

41 of 79