Human AI interaction - Quiz 1
Thank you for your interest.
You are asked to compare two different machine learning models for classification. On the target data set, Model 1 has 95% accuracy, with a 10% false positive rate. Model 2 has a 70% accuracy with a 5% false positive rate. Which model would you prefer to use?
Model 1 because it has higher accuracy
Model 1 because it has a higher false positive rate
Model 2 because it has a lower false positive rate
Either Model 1 or Model 2 might be better. It really depends on what is being classified and the result is used for.
What's the point of a validation set?
A validation set forces you to learn with less data than you have, so you learn a simpler model
A validation set allows you to see how well your model performs on data it hasn't seen before
A validation set allows you to see what's the worst case performance of your model
A validation set allows you to see what's the best case performance of your model
Should you always use a separate validation and test set?
No: if you only have a small amount of data, it's OK to not use a validation set.
No: if you have a small amount of data, it's OK to first get a good model with separate validation/test, and then merge validation set with training set, and train again before finally checking performance on test set.
Yes: it ensures you don't overfit to your dataset but can still iterate on your model.
Yes: it ensures you can train with the maximum amount of data.
This form was created inside of Carnegie Mellon University.
Terms of Service