1 of 7

Object Detection with TensorFlow in FTC

Windsor Robotics

2 of 7

What is TensorFlow?

  • TensorFlow is a Google API that can sense and track objects. It was trained by FTC to train Rover Ruckus Gold and Silver last year, and this year we can use it to detect Skystones.
  • It tracks the Top, Bottom, Left and Right pixel values based on the orientation and position of your camera.
  • We’ll be taking a look at the modified example OpMode that Windsor Robotics used last year for Rover Ruckus.

3 of 7

Initialization Procedures

  • Vuforia forms the base of Tensorflow, so we need to call that first.
  • We also need to initialize the ObjectDetection method.

4 of 7

How TensorFlow decides what is what

  • We set the recognitions list to the getRecognitions method from TensorFlow, which is based off the Left pixel.
  • When the list has three objects within it, the OpMode will cycle through a set of instructions to decide what is what.

5 of 7

How TensorFlow decides what is what (cont)

  • The Op Mode then checks to see if it has three valid X (or Left) coordinate values for the Gold and two Silver Minerals. If it does have three valid values, then it compares the Gold X coordinate value with the other two values:
  • If the Gold X value is the lowest (i.e., leftmost) of the three, then the Gold Mineral is in the "Left" position.
  • If the Gold X value is the highest (i.e., rightmost) of the three, then the Gold Mineral is in the "Right" position.
  • Otherwise, the Gold Mineral is in the "Center" position.

​

6 of 7

Applying the Locations

  • Using the if/then statements that decide where the locations of the minerals are, we can insert telemetry that informs us of the location.
  • Generally speaking, you also want to include a break loop block to exit the loop if you’ve found something.

7 of 7

Closing Thoughts

  • Orientation of your phone matters. If your phone was in Portrait mode when TensorFlow started, then the pixel values will be tracked as Portrait.
  • This is an interpretation of the example program provided by FTC (which can be created as an example OpMode in Blocks or Java), so experimentation is key.
  • This is not an in-depth tutorial of TensorFlow. For additional information, see https://github.com/ftctechnh/ftc_app/wiki/Java-Sample-TensorFlow-Object-Detection-Op-Mode.

​