Lecture 31

Working with Images, Computer Vision

Summer 2023

Working with Image Data

Image Data

• Images are just two-dimensional arrays
• Each entry represents the value of that pixel
• Values are between 0 and 255
• 0 is completely black, 255 is completely white

​

​

​

(demo)

But what about color…?

Color Images

• Each pixel has three color channels
• Red, Green, Blue (RGB)
• A value of 255 represents full intensity
• We can create all other colors by mixing intensities of RGB values

​

​

​

Image Data

(demo)

Video Data

• A video is just a sequence of images
• Frames Per Second (FPS) measures how fast frames are processed

​

(demo)

Computer Vision

CV in the Real World

(self-driving cars)

(commerce)

(medical imaging)

A.I. For Tennis

SwingVision

Object Detection

• Object detection models locate and classify objects in an image
• A bounding box is used to locate the object
• The model predicts an object class for each bounding box

Object Detection

person

person

person

person

ball

How do you build the model?

Step 1: Collect Data

Step 2: Label Data

• Use Computer Vision Annotation Tool (CVAT)
• Labeled by our data labelers!
• Use models to automatically label data

Step 3: Train Model

• “Show” the model an example video
• Make a prediction
• Calculate how bad the model’s predictions are
• Update the model to learn from its mistakes
• Repeat

Some of these (GPUs) might help, too

Step 4: Make Predictions

• Use shot heatmap to show location predictions
• Bounding boxes converted into a single location
• Locations used to predict ball velocity

Cutting-edge Object Detection

YOLO v8

• Stands for “You Only Look Once”
• Real-time Object Detection Model
• Original YOLO: Joseph Redmon (2015)
• YOLO v8: Ultralytics (2023)

(demo)

Thank you!