Lab 6: Do What I Say!

JC Hu

CS 123

Let’s make Pupper listen!

CS 123

Credit: DALL-E

Lab Overview

1. ROSify an interface with GPT for textual understanding and Whisper for speech-to-text
2. Create a voice input pipeline by using the onboard speaker and microphone
1. Microphone is wireless with a receiver and two transmitters
3. Command Pupper with the KarelPupper API and voice input, experimenting with different prompts
4. Benchmark this method against keyword spotting algorithm
• Using a pre-trained CNN/Resnet model
5. Extensions opportunity to improve the current solution

3

Dependencies

• Dependencies for the lab are listed at the top of the lab spec
• We recommend starting those installs, and then working on the first part of your code while those run
• Should take <5 minutes for all to complete
• Let a TA know if you have any issues, or if you are still missing any packages

4

Quick Tips

Safety

• The KarelPuper API moves Pupper very fast if you command it to move forward
• Make sure to test any commands that move the Pupper on the ground

General

• Follow the lab spec to make sure that the Microphone receiver fits on top of the Pupper’s ports
• Work together and experiment with making this method even better!

SSH Tip

• To remove all aliases of pupper.local, run: ssh-keygen -R pupper.local

​

​

5

Final Project Announcement

Project Ideas

• We have compiled a list of final project ideas you can base off of here

Final Project Details

• Description page on website here
• Schedule here

​

Summary

• 5 parts
• Meet with TA
• Proposal
• Milestone 1
• Milestone 2
• Demo day/Presentations: 7/9 at 7pm

​

6

Please ask the TAs if you have any questions, and come to OH if you need help/more time

​

Thank you, and have fun!

7