Welcome to AI Club!

Please sign in on the computer and join the Remind (text @westwoodai to 81010) and Google Classroom (code is c6kktb) if you haven’t done so already

​

​

​

​

Introduction (Glen)

Project Ideas

“gun” detection social media

Trigger word detection

Generate faces yearbook data

Trash vs recycling

Minimizing traffic flow

Identifying location of virus

​

Do we want one big project everyone is working on

Or one big project we break up into teams and compete

Or individual projects for teams - you get to pick your own project?

​

What would you like to do in Flex Tomorrow?

Types of Learning

• Supervised
• If given a X and Y data we want to build a classifier that learns to predict Y based on X
• Unsupervised
• Find patterns on unlabeled data.
• Recurrent

​

Quick Review of Neural Networks

• What is a neural network?
• Essentially a function - takes in input and produces output
• You “train” it so it produces the output you want by feeding it large amounts of labeled data (data you already know the output for)
• Compare predicted output of network to the actual output/label and go back and adjust network so it gets closer to the actual output

​

​

Common Neural Networks

ANN - the most simple neural network

CNN- a neural network with suited to classify images, due to convolutional algorithm

RNN - class of neural networks that allow previous outputs to be used as inputs while having hidden states

GAN - Generates data via a discriminator and a generator(both are usually one of the previous NN )

Loss and Error

Loss functions and optimizations. Machines learn by means of a loss function. It's a method of evaluating how well specific algorithm models the given data. If predictions deviates too much from actual results, loss function would cough up a very large number.

Epochs and Batch Size

Batch Size - how many training examples the model considers when deciding to update the weights. If you have 1000 training examples, you don’t have to use all 1000 of them to compute the loss and take a step in gradient descent. You can use anything from 1 to 1000, but like anything there are tradeoffs.

Larger batches compute more accurate losses, and converge more steadily onto a minimum loss. Smaller batches may converge more unsteadily. However, you can see in the graphs below that lower batch sizes are much faster.

Using all the examples(batch)

Using only one example(stochastic)

A happy middle

Epochs and Batch Size ct.

Epochs - the number of times the model will make a pass over all the examples in the data set.

-The model does not immediately converge on a small loss after one pass of the training set, so often we will need to have multiple epochs.

-Too many epochs, the model may memorize the dataset, and know the dataset a little bit too well so that when it sees other examples, it’s unable to classify them correctly. This is what we call overfitting, and we want to avoid that.

Back to Batches: Differences in how different batch sizes converge onto the optimal solution

model.fit(trainX, trainy, validation_data=(testX, testy), epochs=200, verbose=0, batch_size=len(trainX))

Learning Rate and Optimizer

• This is the most important hyperparameter
• When adjusting the weights in the neural network, we try to minimize the loss
• We use an algorithm called gradient descent
• There are tradeoffs when considering learning rate -
• too high - the model won’t converge - it will “jump over” the minimum
• too small - the model will not learn anything for a very long time
• Momentum is another important concept - are there many minimums or just one?
• Is a hyperparameter you can change to make sure you don’t get stuck in local minimums - you update the weights by learning_rate*derivative of weight + momentum * size of last update
• Higher momentum means you can overshoot minimum, low momentum means you can get stuck in a suboptimal minimum
• Optimizers in keras change the learning rate over time - Adagrad, RMSProp, Adam, SGD

Overfitting

​

​

Large neural nets trained on relatively small datasets can overfit the training data creates statistical noise resulting in poor performance

DropOut

​

Dropout is a regularization technique patented by Google for reducing overfitting in neural networks.

Randomly drops Nodes

​

Activation Functions

• Introduces non linearity into the network. Why is this important?
• Activation in the hidden layers - usually some form of ReLU (Rectified Linear Unit)
• Final Activation is usually sigmoid, linear, softmax, or tanh

​

Regression vs Classification

Regression - the problem where we are trying to predict continuous outputs. For example, trying to predict the price of a house, where there’s lots of different answers that come in a range.

Classification - the problem where we are trying to predict a label for something. Unlike regression, there’s a defined number of classes. For example, trying to predict if someone has diabetes or not. There’s no range of diabetes there’s just yes or no.

How do we evaluate the performance of models (not specific to neural networks)?

When do we need to evaluate performance (training, validation, testing)?

Is accuracy always the best metric?

Binary Classification vs Multiclass Classification

• Binary Classification: Classification into two categories
• Usually a Positive/Negative (Yes/No) situation
• Multiclass Classification: Classification into multiple (more than 2) categories
• Any grouping problem that can’t be solved with one “yes/no” question

TP, TN, FP, FN and Confusion Matrices

• TP (True Positive): The prediction is Positive; the actual value is Positive
• TN (True Negative): The prediction is Negative; the actual value is Negative
• FP (False Positive): The prediction is Positive; the actual value is Negative
• FN (False Negative): The prediction is Negative; the actual value is Positive

Confusion Matrix:

Way of displaying model output as a

relation of True and False Values

​

Accuracy, Precision, Recall, F1

• Accuracy: Measures overall accuracy of the model: #Correct/#Total
• Works well when the cost for True Negatives is high
• Essentially requires data to be balanced
• Precision: Measures accuracy of positive predictions: #TP/(#TP + #FP)
• Works well when the cost for False Positives is high
• Recall: Measures accuracy of negative predictions: #TN/(#TN + #FN)
• Works well when the cost for False Negatives is high
• F1: A balance between Precision and Recall: 2 * (P * R)/(P + R)
• Works well when the costs for False Values are high AND data is unbalanced