CSE 163

Machine Learning, cont’d.�

Suh Young Choi�

🎶 Listening to: Tetris soundtrack

💬 Before Class: What are some ways you think ML might be used in the future?

​

Announcements

• Take-Home Assessment 2 due tomorrow!

​

• New resubmission cycle has opened today and will close next Tuesday
• Can resubmit either Startup or Processing

2

This Time

• Categorical Features
• Assessing Performance
• Overfitting
• Model Complexity
• Hyperparameters
• When to use ML

Last Time

• Machine Learning
• Terminology
• Types of ML
• ML Code (scikit-learn)
• Decision Trees

3

One-Hot Encoding

• Most ML models can’t handle categorical features by default
• Mapping usually doesn’t work, one-hot encoding can!

4

Overfitting

• The most important problem in science you’ve never heard of
• Overfitting: When your model matches the training set so well, that it fails to generalize
• Memorizing answers to Multiple Choice test
• Tall trees are likely to overfit if you don’t have enough data
• Can learn very complex boundaries
• Very few points at the leaves

​

​

5

Assessing Performance

• Training is cool, but we want to know its future performance
• Training data can’t give an accurate evaluation
• “I got 100% on the practice test I have been studying for 4 hours, therefore I will get 100% on the real exam”
• Must hold out data called a test set to evaluate at the end
• Unbiased estimate of performance in the wild

​

Never ever ever train or make decisions based on your test set.

If you do, it will no longer be good estimate of future performance.

6

Code Recap

General ML pipeline - For classification tasks w/ categorical features

7

# Separate data

features = data.loc[:, data.columns != 'target']

features = pd.get_dummies(features)

labels = data['target']

# Train/test split

feat_train, feat_test, lab_train, lab_test = \

train_test_split(features, labels, test_size=0.2)

# Create and train model on train set

model = DecisionTreeClassifier()

model.fit(feat_train, lab_train)

# Predict on test data

predictions = model.predict(feat_test)

accuracy_score(lab_test, predictions)

​

Model Complexity

• One hyperparameter to control complexity of decision tree is the max depth (or height) of tree

8

Visualize the split

9

feat_train

lab_train

feat_test

lab_test

fit

Empty model

Trained model

Visualize the split

10

feat_train

lab_train

feat_test

lab_test

fit

Empty model

Trained model

predictions

accuracy_score

predict

Ground Rules

• Talking about social impact and ethics in data science can be challenging since its effects can be deeply personal or harmful
• Many reasonable people have differing opinions on how to draw the line between okay/not okay, there isn’t always an easy yes/no answer.
• Brought up a lot of questions, but this is not the only set of questions we should ask. I’m just one person with one perspective!

​

Productive Discussions:

• Listen with intention to understand first and forming an opinion only after you fully understand.
• Take responsibility for the intended and unintended effects of your words and actions on others.
• Mindfully respond to others’ ideas by acknowledging the unique value of each contribution.

​

11

Group Work:

Best Practices

When you first working with this group:

• Introduce yourself!
• If possible, angle one of your screens so that everyone can discuss together

​

Tips:

• Starts with making sure everyone agrees to work on the same problem
• Make sure everyone gets a chance to contribute!
• Ask if everyone agrees and periodically ask each other questions!
• Call TAs over for help if you need any!

12

Practice Session 1

• Discussion (3 questions)

​

TAs will walk around to answer questions! Raise your hand!

​

​

Come back at TBD!

13

Practice Session 2

• Coding practice

​

TAs will walk around to answer questions! Raise your hand!

​

​

Come back at TBD!

14

Before Next Time

• Be sure to add to the discussion megathread
• Go to section tomorrow!
• THA 2 due tomorrow night

Next Time

• Data Literacy and Communications

​

15