UNIT-5�ASSOCIATIVE MEMORY NETWORKS

CONTENTS

• Introduction
• Auto Associative Memory
• Hetero Associative Memory
• Bidirectional Associative Memory(BAM)-theory and architecture
• BAM Training Algorithm
• Hopfield Network: Introduction , Architecture

​

INTRODUCTION

• Associative memory is also known as content addressable memory (CAM) or associative storage or associative array.
• It is a special type of memory that is optimized for performing searches through data, as opposed to providing a simple direct access to the data based on the address.
• It is a hardware search engine, a special type of computer memory used in certain very high searching applications.

CONT…

• Applications of Associative memory:
• It can be only used in memory allocation format.
• It is widely used in the database management systems, etc.
• Advantages of Associative memory :
• It is used where search time needs to be less or short.
• It is suitable for parallel searches.
• It is often used to speedup databases.
• It is used in page tables used by the virtual memory and used in neural networks.

​

CONT…

• Disadvantages of Associative memory :
• It is more expensive than RAM.
• Each cell must have storage capability and logical circuits for matching its content with external argument.

​

AUTO ASSOCIATIVE MEMORY�

• This is a single layer neural network in which the input training vector and the output target vectors are the same.
• The weights are determined so that the network stores a set of patterns.
• Architecture:the architecture of Auto Associative memory network has ‘n’ number of input training vectors and similar ‘n’ number of output target vectors.

​

​

HETERO ASSOCIATIVE MEMORY

• Similar to Auto Associative Memory network, this is also a single layer neural network.
• In this network the input training vector and the output target vectors are not the same.
• The weights are determined so that the network stores a set of patterns.
• Hetero associative network is static in nature, hence, there would be no non-linear and delay operations.

CONT…

Architecture:

• The architecture of Hetero Associative Memory network has ‘n’ number of input training vectors and ‘m’ number of output target vectors.

​

​

​

BIDIRECTIONAL ASSOCIATIVE MEMORY(BAM)

• Bidirectional Associative Memory (BAM) is a supervised learning model in Artificial Neural Network.
• This is hetero-associative memory, for an input pattern, it returns another pattern which is potentially of a different size.
• This phenomenon is very similar to the human brain.
• Human memory is necessarily associative. 

CONT…

• Why BAM is required?
• The main objective to introduce such a network model is to store hetero-associative pattern pairs.
• This is used to retrieve a pattern given a noisy or incomplete pattern.
• A BAM contains two layers of neurons, which we shall denote X and Y.
• Layers X and Y are fully connected to each other.

CONT…

• Once the weights have been established, input into layer X presents the pattern in layer Y, and vice versa.
• The layers can be connected in both directions (bidirectional) with the result the weight matrix sent from the X layer to the Y layer is  W and the weight matrix for signals sent from the Y layer to the X layer is W^T.
• Thus, the weight matrix is calculated in both directions.

​

BAM ARCHITECTURE

• When BAM accepts an input of n-dimensional vector X from set A then the model recalls m-dimensional vector Y from set B.
• Similarly when Y is treated as input, the BAM recalls X.

​

ALGORITHM

• Storage (Learning): In this learning step of BAM, weight matrix is calculated between M pairs of patterns (fundamental memories) are stored in the synaptic weights of the network following the equation 
• Testing: We have to check that the BAM recalls perfectly  Y_m for corresponding X_m and recalls  X_m  for corresponding  Y_m. Using,

CONT…

• Retrieval: For an unknown vector X (a corrupted or incomplete version of a pattern from set A or B) to the BAM and retrieve a previously stored association:

​

• Initialize the BAM:  X(0)=X, p=0

​

• Calculate the BAM output at iteration p :

​

​

​

​

​

CONT…

• Update the input vector X(p):

​

​

​

• Repeat the iteration until convergence, when input and output remain unchanged.

​

​

CONT…

• Limitations of BAM:
• Storage capacity of the BAM: In the BAM, stored number of associations should not be exceeded the number of neurons in the smaller layer.
• Incorrect convergence: Always the closest association may not be produced by BAM.
• �

HOPFIELD NETWORK

• INTRODUCTION:
• Hopfield neural network was invented by Dr. John J. Hopfield in 1982.
• It consists of a single layer which contains one or more fully connected recurrent neurons.
• The Hopfield network is commonly used for auto-association and optimization tasks.

​

CONT…

• Discrete Hopfield Network:
• A Hopfield network which operates in a discrete line fashion or in other words, it can be said the input and output patterns are discrete vector, which can be either binary 0,1 or bipolar +1,−1 in nature.
• The network has symmetrical weights with no self-connections i.e., w_ij = w_ji and w_ii = 0.

​

ARCHITECTURE

​

​

​

​

​

​

​

• The output from Y₁ going to Y₂, Y_i and Y_n have the weights w₁₂, w_1i and w_1n respectively. Similarly, other arcs have the weights on them.

CONT…

• This model consists of neurons with one inverting and one non-inverting output.
• The output of each neuron should be the input of other neurons but not the input of self.
• Weight/connection strength is represented by w_ij.
• Connections can be excitatory as well as inhibitory.
• It would be excitatory, if the output of the neuron is same as the input, otherwise inhibitory.
• Weights should be symmetrical, i.e. w_ij = w_ji

�